Article Bot

Article Bot
by velocd


Article Bot is a script that allows you to assign a user account (preferably an exclusive one) to an article bot that will post RSS feeds in select forums at designated intervals. These articles are derived from RSS (Really Simple Syndication) feeds.

RSS files are assigned to article bots, as well to individual forums, which in turn directs what, when and where your article bot will post.

Features

• Works with RSS versions: 0.9x, 2.0, 1.0, and Atom 0.3
• AdminCP interface for adding/modifying/deleting article bots & rss feeds
• Designate intervals (in minutes) to each RSS feed for when your article bot will post them.
• Requires only 1 file edit (admincp/index.php), allowing ease of upgrading your vBulletin
  without having to remodify files.
• Easy to install. Works best in conjunction with crontab (contab help is inside install.html)

News

• 7/25/04 - Version 1.1 Released
  • Bug Fixes
    • Replaced htmlspecialchars() with htmlentities() in articlebot.php to fully convert special entities from descriptions.
    • Added trim() to thread titles, to prevent double threads from being posted that are merely off by whitespace.
    • New table articlebot_rss_cache for storing a copy of article bot threads, for quicker checking of doubles when posting new threads.
    • Threads are checked in articlebot_rss_cache, so you can permanently remove them from the thread table (you don't have to "soft delete") without worrying about the bot replicating them.
    • Other minor bugs.
  • Features
    • New installer/upgraders for doing the SQL part automatically.
    • Added "label" field to add/modify RSS interface, that allows you to tag a label, e.g. [yahoo], to the beginning of threads, unique to each RSS feed. This might help for better distinction of RSS feeds.
    • Added an option to toggle on/off of building similar threads per RSS feed. Some RSS feeds could hault the script if the building of similar thread results for that feed takes too long (maybe the article thread title is too vague, and matches a lot of results--my guess is it's only a problem on large forums).
    • Added an option to toggle on/off skipping of articles that don't contain descriptions.
    • Regular expression search and replace for article titles and descriptions. This grants you much flexability to filter out things you don't want in your titles or descriptions (given you have some regex experience, I will try to post some patterns in due time.. feel free to add yours )
• 7/11/04 - Version 1.0 Released

Upgrading

• To upgrade, reupload the new PHP and MagpieRSS parser files to their places on your server. Upload upgrade_vx-x (replace the x's with the version you are upgrading to) to /path/to/forums/admincp/, and then execute that script.

Files

• 3 screenshots attached
• articlebot_v1.zip
  • install.html, logo.gif
  • articlebot_admin.php
  • articlebot/
    • rss_cache.inc
    • rss_fetch.inc
    • rss_parse.inc
    • rss_utils.inc
    • extlib/
      • Snoopy.class.inc

RSS Feeds

There are several in the install.html, but if you're looking for more than you could ever want, check out NewsIsFree's directory:

http://www.newsisfree.com/sources/bycat/


Remember to click install if you use this hack.

Also be sure to read through the what you haven't of the thread before asking a question, for it may have been answered.

[akiy]

Originally Posted by Nordinho

I have this weird permission issue...the article bots post are always published...I've made a new usergroup for the bot, with limited permissions and 'always moderate posts', but the articlebot just ignores those settings and keeps on posting...any other ideas on moderating it's posts???

I just hacked the code so that all posts made by the Article Bot get put into the moderation queue.

In articlebot.php, find:

             $DB_site->query("                                                                                                                        
                 INSERT INTO ".TABLE_PREFIX."thread                                                                                                    
                     (title, lastpost, forumid, open,                                                                                                    
                     replycount, postusername, postuserid, lastposter, dateline, iconid,                                                                
                     visible, attach, similar, is_rss)                                                                                                
                 VALUES                                                                                                                                
                     ('".addslashes($article['title'])."', ".TIMENOW.", $articlebot[forumid],                                                            
                         1, 0, '".addslashes($articlebot['username']) . "', $articlebot[userid],                                                            
                         '".addslashes($articlebot['username']) . "', ".TIMENOW.", $articlebot[iconid], 1, 0,                                            
                         '".addslashes($similarthreads)."', 1)                                                                                            
                ");

... and replace with:

             $DB_site->query("                                                                                                                        
                 INSERT INTO ".TABLE_PREFIX."thread                                                                                                    
                     (title, lastpost, forumid, open,                                                                                                    
                     replycount, postusername, postuserid, lastposter, dateline, iconid,                                                                
                     visible, attach, similar, is_rss)                                                                                                
                 VALUES                                                                                                                                
                     ('".addslashes($article['title'])."', ".TIMENOW.", $articlebot[forumid],                                                            
                         1, 0, '".addslashes($articlebot['username']) . "', $articlebot[userid],                                                            
                         '".addslashes($articlebot['username']) . "', ".TIMENOW.", $articlebot[iconid], 0, 0,                                            
                         '".addslashes($similarthreads)."', 1)                                                                                            
                ");

The above is a one character change -- look for the colored "0" in the second code segment...

Also, after:

              /* +---------------------------------+                                                                                                    
                 | Query: update thread         |                                                                                                    
                 +---------------------------------+ */


             $DB_site->query("                                                                                                                        
                 UPDATE ".TABLE_PREFIX."thread                                                                                                        
                 SET firstpostid = $post_id                                                                                                            
                 WHERE threadid = $thread_id                                                                                                            
                ");

... add:

             $p = $DB_site->query_first("SELECT firstpostid from ".TABLE_PREFIX."thread                                                                
                                         where threadid = $thread_id");

             $DB_site->query("                                                                                                                        
                 INSERT INTO ".TABLE_PREFIX."moderation                                                                                                
                     (threadid, postid, type)                                                                                                            
                 VALUES                                                                                                                                
                     ($thread_id, $p[firstpostid], \"thread\")");

Once again, this will cause all posts done by all article bots to be put into the moderation queue.

Let's hope that VeloCD can now use this in the future so that we can specify which bots get moderated and which ones do not...

[ragintajin]

Since there is no support for the "There is no file located" error...how do I go about uninstalling this hack? (Specifically, the database changes)

[berberber]

I have the articlebot running via a vB Scheduled Task. The thing you have to remember about the scheduled tasks is that it's not a "real" cron job - it depends on activity on your forum to kick it off.

Thank you, i finally found how to set up a common cron task and I believe it is better (not to move a forum, but thats rare actually).

One other thing, after a server out time of a few hours, many articles were indexed a second time. (it is quite rare too, but perhaps that could be improved ?)
Thanks you for the hack.

[jzewatsky]

I have installed this hack and am loving it.

I could use some help writing the cron to run articlebot.php. This whole idea is new to me and I dont do that well with this type of stuff to begin with. My database management tool provided by my server is called phpMyAdmin (I believe) and I have downloaded PUttY. I unfortunately have no idea what to do with either of these to set up the cron. Any help (especially examples) would be great!

[poolking]

Are you actually running cron jobs to get the feed?

[bishop93]

Originally Posted by poolking

Are you actually running cron jobs to get the feed?

Wait wait. I deleted my post because I *just* went back, changed to the use of the phpinclude_end option, and suddenly had incredible PHP errors at the top of the forums, hit refresh and *bing* there was slashdot.

That tells me that the cron job isn't doing it's job. Or so I would think.

But, the other RSS that I was really after still isn't working ... BUT, I don't think that is a problem of this mod. I think that's a problem of the source.

[bishop93]

Okay ... this is the error that keeps coming back. Is this the source or the mod?

Warning: MagpieRSS: Failed to parse RSS file. (not well-formed (invalid token) at line 3579, column 35) in /home/xxx/public_html/xxx/articlebot/rss_fetch.inc on line 230

Warning: array_reverse(): The argument should be an array in /home/xxx/public_html/xxx/articlebot/articlebot.php on line 137

Warning: Invalid argument supplied for foreach() in /home/xxx/public_html/xxx/articlebot/articlebot.php on line 141

It would appear to me that it's the source. However, using the same feed with other readers (we pick up the feed for our website), works just fine without error.

Any thoughts?

Thanks in advance.

[Tenny]

Originally Posted by bishop93

Wait wait. I deleted my post because I *just* went back, changed to the use of the phpinclude_end option, and suddenly had incredible PHP errors at the top of the forums, hit refresh and *bing* there was slashdot.

That tells me that the cron job isn't doing it's job. Or so I would think.

But, the other RSS that I was really after still isn't working ... BUT, I don't think that is a problem of this mod. I think that's a problem of the source.

Very confused what you did? Can you explain it again/ What you deleted? Where is phpinclude_end?

Thanks, hit me up on AIM: Tendollaz, maybe I can figure it out and try to explain to everyone else...

[bishop93]

I did manage to get things worked out. Sorta. I finally got the cron job to function properly as well. (I didn't read carefully what I was inputing)

However, the cron job now kicks back the same exact error as the phpinclude_end option did when I tried that way.

Warning: MagpieRSS: Failed to parse RSS file. (not well-formed (invalid token) at line 3657, column 35) in /home/xxx/public_html/xxx/articlebot/rss_fetch.inc on line 230

Warning: array_reverse(): The argument should be an array in /home/xxx/public_html/xxx/articlebot/articlebot.php on line 137

Warning: Invalid argument supplied for foreach() in /home/xxx/public_html/xxx/articlebot/articlebot.php on line 141

I'm still thinking this could be the source issue since that parsing RSS file error looks familiar. But, if this returns an error, I'm not sure why my other newsfeed module (in Mambo CMS) is pulling the exact same feed without any errors.

(I'm at work and IMs are blocked, but I'd be more than willing to open things up and let you in to check around if you want to drop me an email. Currently the forums are in test mode anyway before we switch the main boards from SMF to vBulletin (a different headache, but oh well).)

[Playboy69]

I am not sure what I am doing wrong but I cant seem to get this to work. I set it all up and it even shows the control panel in my admincp but I cant seem to get this booger to post nothing. I even set up a cron job and still nothing. any possable clue on what i need to do please.