Article Bot

Article Bot
by velocd


Article Bot is a script that allows you to assign a user account (preferably an exclusive one) to an article bot that will post RSS feeds in select forums at designated intervals. These articles are derived from RSS (Really Simple Syndication) feeds.

RSS files are assigned to article bots, as well to individual forums, which in turn directs what, when and where your article bot will post.

Features

• Works with RSS versions: 0.9x, 2.0, 1.0, and Atom 0.3
• AdminCP interface for adding/modifying/deleting article bots & rss feeds
• Designate intervals (in minutes) to each RSS feed for when your article bot will post them.
• Requires only 1 file edit (admincp/index.php), allowing ease of upgrading your vBulletin
  without having to remodify files.
• Easy to install. Works best in conjunction with crontab (contab help is inside install.html)

News

• 7/25/04 - Version 1.1 Released
  • Bug Fixes
    • Replaced htmlspecialchars() with htmlentities() in articlebot.php to fully convert special entities from descriptions.
    • Added trim() to thread titles, to prevent double threads from being posted that are merely off by whitespace.
    • New table articlebot_rss_cache for storing a copy of article bot threads, for quicker checking of doubles when posting new threads.
    • Threads are checked in articlebot_rss_cache, so you can permanently remove them from the thread table (you don't have to "soft delete") without worrying about the bot replicating them.
    • Other minor bugs.
  • Features
    • New installer/upgraders for doing the SQL part automatically.
    • Added "label" field to add/modify RSS interface, that allows you to tag a label, e.g. [yahoo], to the beginning of threads, unique to each RSS feed. This might help for better distinction of RSS feeds.
    • Added an option to toggle on/off of building similar threads per RSS feed. Some RSS feeds could hault the script if the building of similar thread results for that feed takes too long (maybe the article thread title is too vague, and matches a lot of results--my guess is it's only a problem on large forums).
    • Added an option to toggle on/off skipping of articles that don't contain descriptions.
    • Regular expression search and replace for article titles and descriptions. This grants you much flexability to filter out things you don't want in your titles or descriptions (given you have some regex experience, I will try to post some patterns in due time.. feel free to add yours )
• 7/11/04 - Version 1.0 Released

Upgrading

• To upgrade, reupload the new PHP and MagpieRSS parser files to their places on your server. Upload upgrade_vx-x (replace the x's with the version you are upgrading to) to /path/to/forums/admincp/, and then execute that script.

Files

• 3 screenshots attached
• articlebot_v1.zip
  • install.html, logo.gif
  • articlebot_admin.php
  • articlebot/
    • rss_cache.inc
    • rss_fetch.inc
    • rss_parse.inc
    • rss_utils.inc
    • extlib/
      • Snoopy.class.inc

RSS Feeds

There are several in the install.html, but if you're looking for more than you could ever want, check out NewsIsFree's directory:

http://www.newsisfree.com/sources/bycat/


Remember to click install if you use this hack.

Also be sure to read through the what you haven't of the thread before asking a question, for it may have been answered.

[gmtalk]

Originally Posted by Velocd

As said, the bot starts at the end of the file (oldest articles) and counts its way up. It makes sense that if you remove a thread, the bot (which does check for existing threads) will re-create it. Instead of removing the thread from the database, if you wish to get rid of it, "soft" delete it.

Also note there is a "initial article trim" to skip a number of articles when first starting on an RSS feed.

Well I have been testing this one out and it is going nuts on me. The feeds I have are from Racing sites and they come in every few hours. Bad thing is that it is grabbing even the old stories over and over. 8 stories have been repeated on the site 12 times now.

This is nuts. Is there a better way to have it check for dupes? I know in the other RSS feed hack they have sorted that out. I would like to see that in this one. :\

[scotty]

i had the problem of reoccuring threads by my bot too...

the reason was an additional space in the "title-tag"!

instead of <title>here the headline</title> there was <title>here the headline </title> in the rss-feed.

so the bot couldn't recognize, that the thread (without the whitespace) still exists.

my sollution:

find:

if (!$articlebot['lastpost'] && (--$articlebot['initialtrim'] > 0))

and just add above:

$article['title'] = trim($article['title']);

now it is working fine again!

[Floris]

now it is working fine again! [/QUOTE]The .php file is ready for cron, to save a query could we setup this through the scheduler from vbulletin?

[Bobby S]

Originally Posted by floris

The .php file is ready for cron, to save a query could we setup this through the scheduler from vbulletin?

I set it up as a scheduled task and it appears to work fine: except some feeds duplicating. I'm not sure why but the post two up may hold the answer...

[Velocd]

I've been meaning to make an update fixing the bugs discussed so far, but just haven't had the time.

Probably at the weekend I will do so.

Do the trimming of the thread title mentioned here, and hopefully that erases some duplicate threads that are only off by whitespace.

This hack checks for threads of the same title that already exist, and if they do, it wont post them again. Unless there is a bug.

[Chris-FH]

Originally Posted by Chris-FH

Please help me, I'm getting this error:

Warning: main(./global.php) [function.main]: failed to create stream: No such file or directory in /xxx/html/board/articlebot/articlebot.php on line 43

Fatal error: main() [function.main]: Failed opening required './global.php' (include_path='.:/usr/lib/php') in /xxx/html/board/articlebot/articlebot.php on line 43

@romeshomey:

What was your solution?

C.

Nobody??? :ermm:

C.

[scotty]

Originally Posted by Chris-FH

Nobody??? :ermm:

C.

do you have changed that:

chdir('/path/to/forums/');
require_once './global.php';

(step8 installation)

[Chris-FH]

Yes I have...

If this path (chdir) is incorrect, the error is in line 42...

But it is correct...

If I set the full path to the global.php , I get this error:

Warning: main(./includes/init.php) [function.main]: failed to create stream: No such file or directory in /xxx/html/board/global.php on line 18

Fatal error: main() [function.main]: Failed opening required './includes/init.php' (include_path='.:/usr/lib/php') in /xxx/html/board/global.php on line 18

C.

[Kru]

Well I installed it. I went on add bot and entered the user ID 1.

After it I got following message:

Database error in vBulletin 3.0.3:

Invalid SQL:
SELECT articlebot.userid,
articlebot.articlebotid, user.username
FROM articlebot
LEFT JOIN user
USING (userid)
WHERE articlebot.userid = 0

mysql error: Table 'help.articlebot' doesn't exist

mysql error number: 1146

Date: Tuesday 20th of July 2004 12:12:16 PM
Script: http://www.spiritual-connections.com...ebot_admin.php
Referer: http://www.spiritual-connections.com...min.php?do=add
Username: xxx
IP Address: xxxx

every time if I am trying to open my forum this appears.

Can anybody help me, please

[scotty]

Originally Posted by Chris-FH

Yes I have...
If this path (chdir) is incorrect, the error is in line 42...
But it is correct...

it can't be, sorry.

please provide:
- where is your forum located? (full path!) you can xxx some chars to mask your path
- where is your articlebot.php located? (full path!)
- how do you call articlebot.php? cron? direct?

If I set the full path to the global.php , I get this error:

you can't edit line 43 - the only line to edit is 42!