Article Bot

Article Bot
by velocd


Article Bot is a script that allows you to assign a user account (preferably an exclusive one) to an article bot that will post RSS feeds in select forums at designated intervals. These articles are derived from RSS (Really Simple Syndication) feeds.

RSS files are assigned to article bots, as well to individual forums, which in turn directs what, when and where your article bot will post.

Features

• Works with RSS versions: 0.9x, 2.0, 1.0, and Atom 0.3
• AdminCP interface for adding/modifying/deleting article bots & rss feeds
• Designate intervals (in minutes) to each RSS feed for when your article bot will post them.
• Requires only 1 file edit (admincp/index.php), allowing ease of upgrading your vBulletin
  without having to remodify files.
• Easy to install. Works best in conjunction with crontab (contab help is inside install.html)

News

• 7/25/04 - Version 1.1 Released
  • Bug Fixes
    • Replaced htmlspecialchars() with htmlentities() in articlebot.php to fully convert special entities from descriptions.
    • Added trim() to thread titles, to prevent double threads from being posted that are merely off by whitespace.
    • New table articlebot_rss_cache for storing a copy of article bot threads, for quicker checking of doubles when posting new threads.
    • Threads are checked in articlebot_rss_cache, so you can permanently remove them from the thread table (you don't have to "soft delete") without worrying about the bot replicating them.
    • Other minor bugs.
  • Features
    • New installer/upgraders for doing the SQL part automatically.
    • Added "label" field to add/modify RSS interface, that allows you to tag a label, e.g. [yahoo], to the beginning of threads, unique to each RSS feed. This might help for better distinction of RSS feeds.
    • Added an option to toggle on/off of building similar threads per RSS feed. Some RSS feeds could hault the script if the building of similar thread results for that feed takes too long (maybe the article thread title is too vague, and matches a lot of results--my guess is it's only a problem on large forums).
    • Added an option to toggle on/off skipping of articles that don't contain descriptions.
    • Regular expression search and replace for article titles and descriptions. This grants you much flexability to filter out things you don't want in your titles or descriptions (given you have some regex experience, I will try to post some patterns in due time.. feel free to add yours )
• 7/11/04 - Version 1.0 Released

Upgrading

• To upgrade, reupload the new PHP and MagpieRSS parser files to their places on your server. Upload upgrade_vx-x (replace the x's with the version you are upgrading to) to /path/to/forums/admincp/, and then execute that script.

Files

• 3 screenshots attached
• articlebot_v1.zip
  • install.html, logo.gif
  • articlebot_admin.php
  • articlebot/
    • rss_cache.inc
    • rss_fetch.inc
    • rss_parse.inc
    • rss_utils.inc
    • extlib/
      • Snoopy.class.inc

RSS Feeds

There are several in the install.html, but if you're looking for more than you could ever want, check out NewsIsFree's directory:

http://www.newsisfree.com/sources/bycat/


Remember to click install if you use this hack.

Also be sure to read through the what you haven't of the thread before asking a question, for it may have been answered.

[Frzzy]

Seem like the bot doesn't work with this RSS...

http://www.theinquirer.net/inquirer.rss

I know why, look like the bot won't do anything if the description tag is missing.

[romeshomey]

Originally Posted by Velocd

Normally the article title shouldn't be inside the description, unless you modified the articlebot_post template.

Take a look and you'll see what I mean. You'll see how where the text in bold ends is where there needs to be a space before the rest of the desctiption is posted.

http://www.insidersportszone.com/for...6602#post26602

It keeps doing this:

Odom, Butler, Grant and pick go to LakersMIAMI -- It's official: The Shaquille O'Neal era in Los Angeles is over, and his new team is the Miami Heat.The Lakers and Heat finalized a trade Wednesday sending O'Neal to Miami. Los Angeles gets Caron Butler, Lamar Odom, Brian Grant and a first-round draft pick.The deal had been on the verge of completion since Saturday, when O'Neal met in Orlando with Heat president Pat Riley and agreed to the trade. NBA attorneys approved it on the same day the league's two-week moratorium... (Associated Press)

See how there is no space between the work Lakers and the word MIAMI. It does that in all of them and the space is there in the RSS feed itself.

<pubDate>Fri, 9 Jul 2004 20:48:37 CEST</pubDate> 
  <guid isPermaLink="false">i,43741943</guid> 
  </item>
- <item>
  <title>Shaq traded to Heat in blockbuster deal</title> 
  <link>http://www.newsisfree.com/click/i,44177556,1851,rss/</link> 
  <description><b>Odom, Butler, Grant and pick go to Lakers</b>MIAMI -- It's official: The Shaquille O'Neal era in Los Angeles is over, and his new team is the Miami Heat.The Lakers and Heat finalized a trade Wednesday sending O'Neal to Miami. Los Angeles gets Caron Butler, Lamar Odom, Brian Grant and a first-round draft pick.The deal had been on the verge of completion since Saturday, when O'Neal met in Orlando with Heat president Pat Riley and agreed to the trade. NBA attorneys approved it on the same day the league's two-week moratorium... (Associated Press)</description>

[romeshomey]

Originally Posted by Frzzy

[high]* Frzzy clicks install
[/high]

A question, can the bot recognize and ignore already posted item?

I wonder the same thing as it keeps posting the same articles again and again, and if I remove it, itll post it as the next one again.

[romeshomey]

Originally Posted by romeshomey

See how there is no space between the work Lakers and the word MIAMI. It does that in all of them and the space is there in the RSS feed itself.

My mistake, the RSS doesnt provide a space in the description which is why there is no space.

Theres no way to add one is there?

IM just glad I got the HTML to work. That was a big plus. If anyone else has the same problem with the HTML showing up as code and not being executed.

The fix for the HTML is:

In the forums/articlebot/articlebot.php change:

$article['description'] = strip_tags(htmlspecialchars($article['description']));

too

$article['description'] = $article['description'];

Then just make sure that HTML is enabled in that forum section.

[ranger2kxlt]

Thanks for the link, but not 100% how to use this?

Instead of making the bot link to a .rss or .xml file, i now link him to the one generated by that link? Because all the pages it shows me seem to show up as html....

Thanks

[Velocd]

The fix for the HTML is:

In the forums/articlebot/articlebot.php change:

romeshomey, it's working for you because you have HTML enabled for your posts (which is not a good idea). For anyone else, they will only get HTML tags, so it's good to have the strip_tags() normally in there.

I know why, look like the bot won't do anything if the description tag is missing.

Yes, as I said not too earlier here, RSS feeds without descriptions will be skipped.

My mistake, the RSS doesnt provide a space in the description which is why there is no space.

Theres no way to add one is there?

Not really. It's the mistake of the RSS provider to forget the space, but there isn't really a way to add it, given the arbitrary data.

I wonder the same thing as it keeps posting the same articles again and again, and if I remove it, itll post it as the next one again.

As said, the bot starts at the end of the file (oldest articles) and counts its way up. It makes sense that if you remove a thread, the bot (which does check for existing threads) will re-create it. Instead of removing the thread from the database, if you wish to get rid of it, "soft" delete it.

Also note there is a "initial article trim" to skip a number of articles when first starting on an RSS feed.

[romeshomey]

Thanks Velocd. Very useful script...

[sjau]

Can someone test out this feed? I can't get it to work with either bot:

http://www.swissinfo.org/sde/swissin...?siteSect=1111

[romeshomey]

Originally Posted by sjau

Can someone test out this feed? I can't get it to work with either bot:

http://www.swissinfo.org/sde/swissin...?siteSect=1111

It might only work with feeds with date tags. Not sure though, but I notice your feed has no dates in it..

[ranger2kxlt]

Ok, got my search query going good....but! it seems to pull the same info from different sources, and i have repeat posts.....is there a way to moderate the bot? so all i have to do is look at the threads and take out the dups.?

Its an extra step, but at least i don't have to find the news!

Thanks