Article Bot

Article Bot
by velocd


Article Bot is a script that allows you to assign a user account (preferably an exclusive one) to an article bot that will post RSS feeds in select forums at designated intervals. These articles are derived from RSS (Really Simple Syndication) feeds.

RSS files are assigned to article bots, as well to individual forums, which in turn directs what, when and where your article bot will post.

Features

• Works with RSS versions: 0.9x, 2.0, 1.0, and Atom 0.3
• AdminCP interface for adding/modifying/deleting article bots & rss feeds
• Designate intervals (in minutes) to each RSS feed for when your article bot will post them.
• Requires only 1 file edit (admincp/index.php), allowing ease of upgrading your vBulletin
  without having to remodify files.
• Easy to install. Works best in conjunction with crontab (contab help is inside install.html)

News

• 7/25/04 - Version 1.1 Released
  • Bug Fixes
    • Replaced htmlspecialchars() with htmlentities() in articlebot.php to fully convert special entities from descriptions.
    • Added trim() to thread titles, to prevent double threads from being posted that are merely off by whitespace.
    • New table articlebot_rss_cache for storing a copy of article bot threads, for quicker checking of doubles when posting new threads.
    • Threads are checked in articlebot_rss_cache, so you can permanently remove them from the thread table (you don't have to "soft delete") without worrying about the bot replicating them.
    • Other minor bugs.
  • Features
    • New installer/upgraders for doing the SQL part automatically.
    • Added "label" field to add/modify RSS interface, that allows you to tag a label, e.g. [yahoo], to the beginning of threads, unique to each RSS feed. This might help for better distinction of RSS feeds.
    • Added an option to toggle on/off of building similar threads per RSS feed. Some RSS feeds could hault the script if the building of similar thread results for that feed takes too long (maybe the article thread title is too vague, and matches a lot of results--my guess is it's only a problem on large forums).
    • Added an option to toggle on/off skipping of articles that don't contain descriptions.
    • Regular expression search and replace for article titles and descriptions. This grants you much flexability to filter out things you don't want in your titles or descriptions (given you have some regex experience, I will try to post some patterns in due time.. feel free to add yours )
• 7/11/04 - Version 1.0 Released

Upgrading

• To upgrade, reupload the new PHP and MagpieRSS parser files to their places on your server. Upload upgrade_vx-x (replace the x's with the version you are upgrading to) to /path/to/forums/admincp/, and then execute that script.

Files

• 3 screenshots attached
• articlebot_v1.zip
  • install.html, logo.gif
  • articlebot_admin.php
  • articlebot/
    • rss_cache.inc
    • rss_fetch.inc
    • rss_parse.inc
    • rss_utils.inc
    • extlib/
      • Snoopy.class.inc

RSS Feeds

There are several in the install.html, but if you're looking for more than you could ever want, check out NewsIsFree's directory:

http://www.newsisfree.com/sources/bycat/


Remember to click install if you use this hack.

Also be sure to read through the what you haven't of the thread before asking a question, for it may have been answered.

[mrpotatohead]

Hi,

I'm having a problem that has only occured recently.

See:
http://www.bigbrotherwebsite.net/for...splay.php?f=34

The bot is starting to post up news stories from the RSS feed that are days or sometimes weeks old as new... why is this? It is not coming up in the actual RSS feed as recent news stories, only in the forum...

Can anyone help?


- Joe

[Torqued]

Originally Posted by mrpotatohead

Hi,

I'm having a problem that has only occured recently.

See:
http://www.bigbrotherwebsite.net/for...splay.php?f=34

The bot is starting to post up news stories from the RSS feed that are days or sometimes weeks old as new... why is this? It is not coming up in the actual RSS feed as recent news stories, only in the forum...

Can anyone help?


- Joe

Check the RSS feed/origin site - I have found that some feeds/sites will recycle older stories if there has not been much traffic.

[mrpotatohead]

Nope it's not that, because it's my own website that I am including the feed from in to the forum... it's very strange!?

- Joe

[kyouens]

Originally Posted by mrpotatohead

Hi,

I'm having a problem that has only occured recently.

See:
http://www.bigbrotherwebsite.net/for...splay.php?f=34

The bot is starting to post up news stories from the RSS feed that are days or sometimes weeks old as new... why is this? It is not coming up in the actual RSS feed as recent news stories, only in the forum...

Can anyone help?


- Joe

Have you noted that the old posts are duplicates, e.g. that they are identical in every way to prior posts? Same post title, content, etc?

Have you noted that the old articles have exceptionally long titles?

Mine was doing that and I edited the code to fix it. If you're having the same problem let me know and I'll post (re-post? can't remember . ..) the fix.

Kenny

[mrpotatohead]

Originally Posted by kyouens

Have you noted that the old posts are duplicates, e.g. that they are identical in every way to prior posts? Same post title, content, etc?

Have you noted that the old articles have exceptionally long titles?

Mine was doing that and I edited the code to fix it. If you're having the same problem let me know and I'll post (re-post? can't remember . ..) the fix.

Kenny

Yup, yes to both those Kenny... they do have long titles and yes the duplicates are exactly the same... if you could would you be able to post the fix, I'd be extremely grateful!!


- Joe

[kyouens]

Yup, yes to both those Kenny... they do have long titles and yes the duplicates are exactly the same... if you could would you be able to post the fix, I'd be extremely grateful!!

Hey Joe, below I've attatched my description of and workaround for this problem. The shorthand answer, which does involve mucking with the database (so make a backup):

Go into PHPmyadmin (or whatever mySQL client you've got and change the column type of the "title" row of the articlebot_rss_cache table from Varchar(255) to "text". That eliminates a 255 character limit for titles in the feed cache and stops the doubleposting. Note that the thread title ON THE POST will still be limited to 254 characters, but this shouldn't be a problem at all for most people.

I did this about a month ago and I've had no problems at all since that time with doubleposting.

Here's the original stuff below:

Originally Posted by kyouens

Check this out: In my forum, the longest threads have names like this:

Immunohistochemical Stains for p63 and alpha-Methylacyl-CoA Racemase, Versus a Cocktail Comprising Both, in the Diagnosis of Prostatic Carcinoma: A Comparison of the Immunohistochemical Staining of 430 Foci in Radical Prostatectomy and Needle Biopsy

However, the feed itself gives the title as this (ignore the html characters):

Immunohistochemical%20Stains%20for%20p63%20and%20
alpha-Methylacyl-CoA%20Racemase%2C%20Versus%20a%20
Cocktail%20Comprising%20Both%2C%20in%20the%20Diagn osis%20
of%20Prostatic%20Carcinoma%3A%20A%20Comparison%20o f%20
the%20Immunohistochemical%20Staining%20of%20430%20 Foci
%20in%20Radical%20Prostatectomy%20and%20Needle%20
Biopsy%20Tissues.

These are not identical. If you look, the last part of the feed's title has a few extra words. For some reason, the title is getting truncated as it is being written into the forum, which is why the script, which I believe uses the title and postdate to determine identical-ness, fails to see these as identical.

Now to find the responsible code and drop a major load of hurt on this stupid bug.

Update: The problem is that the title row in the post table (of vbulletin) is of type varchar(250), and some of the full titles are >250 characters. To include the full titles, I may need to change the column type to TEXT. I wonder if this will wreck my vbulletin due to unforeseen badness...I will ask around before I try it and will update the thread.

Update: Fixed the doubleposting of articles with long titles issue. Go into PHPmyadmin and change the column type of the "title" row of the articlebot_rss_cache table from Varchar(255) to "text". That eliminates the 255 character limit for titles in the feed cache and stops the doubleposting. Note that the thread title ON THE POST will still be limited to 254 characters, but this shouldn't be a problem at all for most people.

[mrpotatohead]

Thanks kyouens!! Fingers crossed it works... thanks for your help, much appreciated!

- Joe

[Benj]

my cron gets these errors any ideas tried changing the files permissions

/home/benj/www/articlebot/articlebot.php: line 1: ?php: No such file or
directory
/home/benj/www/articlebot/articlebot.php: line 5: /aquota.user: Permission
denied
/home/benj/www/articlebot/articlebot.php: line 6: syntax error near unexpected
token `|'
/home/benj/www/articlebot/articlebot.php: line 6: ` | articlebot.php, v1.1,
07/25/04

[Benj]

bump

[zachlee]

I am getting the cron error, even though I have set the script to report to /dev/null

/usr/bin/php /var/www/html/football-forums/forums/articlebot/articlebot.php >> /dev/null

my error email I get is:

junk after document element at line 161, column 16

I followed earlier instruction, to change the table type of 'title' in 'articlebot_rss_cache' from 'varchar' to 'text' which eleminated 1 line of the error. What am I missing to fix this?

Thanks
-Zach