Article Bot

Article Bot
by velocd


Article Bot is a script that allows you to assign a user account (preferably an exclusive one) to an article bot that will post RSS feeds in select forums at designated intervals. These articles are derived from RSS (Really Simple Syndication) feeds.

RSS files are assigned to article bots, as well to individual forums, which in turn directs what, when and where your article bot will post.

Features

• Works with RSS versions: 0.9x, 2.0, 1.0, and Atom 0.3
• AdminCP interface for adding/modifying/deleting article bots & rss feeds
• Designate intervals (in minutes) to each RSS feed for when your article bot will post them.
• Requires only 1 file edit (admincp/index.php), allowing ease of upgrading your vBulletin
  without having to remodify files.
• Easy to install. Works best in conjunction with crontab (contab help is inside install.html)

News

• 7/25/04 - Version 1.1 Released
  • Bug Fixes
    • Replaced htmlspecialchars() with htmlentities() in articlebot.php to fully convert special entities from descriptions.
    • Added trim() to thread titles, to prevent double threads from being posted that are merely off by whitespace.
    • New table articlebot_rss_cache for storing a copy of article bot threads, for quicker checking of doubles when posting new threads.
    • Threads are checked in articlebot_rss_cache, so you can permanently remove them from the thread table (you don't have to "soft delete") without worrying about the bot replicating them.
    • Other minor bugs.
  • Features
    • New installer/upgraders for doing the SQL part automatically.
    • Added "label" field to add/modify RSS interface, that allows you to tag a label, e.g. [yahoo], to the beginning of threads, unique to each RSS feed. This might help for better distinction of RSS feeds.
    • Added an option to toggle on/off of building similar threads per RSS feed. Some RSS feeds could hault the script if the building of similar thread results for that feed takes too long (maybe the article thread title is too vague, and matches a lot of results--my guess is it's only a problem on large forums).
    • Added an option to toggle on/off skipping of articles that don't contain descriptions.
    • Regular expression search and replace for article titles and descriptions. This grants you much flexability to filter out things you don't want in your titles or descriptions (given you have some regex experience, I will try to post some patterns in due time.. feel free to add yours )
• 7/11/04 - Version 1.0 Released

Upgrading

• To upgrade, reupload the new PHP and MagpieRSS parser files to their places on your server. Upload upgrade_vx-x (replace the x's with the version you are upgrading to) to /path/to/forums/admincp/, and then execute that script.

Files

• 3 screenshots attached
• articlebot_v1.zip
  • install.html, logo.gif
  • articlebot_admin.php
  • articlebot/
    • rss_cache.inc
    • rss_fetch.inc
    • rss_parse.inc
    • rss_utils.inc
    • extlib/
      • Snoopy.class.inc

RSS Feeds

There are several in the install.html, but if you're looking for more than you could ever want, check out NewsIsFree's directory:

http://www.newsisfree.com/sources/bycat/


Remember to click install if you use this hack.

Also be sure to read through the what you haven't of the thread before asking a question, for it may have been answered.

[Velocd]

Is there a way to get it to read the html in the RSS feeds?

I have HTML enabled in the forums, but when my Bot posts the HTML in the feeds stil show as <b> and </b> etc...

Is there a way to enable it to read the HTML so the bold is bold and etc?

I will look into this. I would have suspected that enabling HTML on the forums would show the HTML, but guess not. It's a security hazard to enable HTML for posts, btw, and not recommened.

I'm thinking I'll probably throw in strip_tags() to remove the HTML tags. Good RSS feeds don't provide HTML anyway, so you might look for others.

[romeshomey]

All of ESPN's feeds have HTML, and seeing as Im a sports forum, then I dont have alot of choices to choose from as far as getting the latest up to date feeds in those areas.

Thanks though, I'd love to get those tags either removed or working, either or. Let me know if or when you figure out how to remove or enable them

I love the addition of the Bot, although he may surpass the world posting records! lol

[Gryphon]

Originally Posted by Velocd

Ah, although I don't believe the RSS parser I use (MagpieRSS) grabs that information.

I'll take a second look though.

I mean in addition to adding the URL and the time limit and the fetch limit and initially skip options, add a field that the board admin can specify as prefix for the titles, like Poll: or Moved: or Sticky:, in this case [News] or [Blues]. Something that people can use to identify that it is an automated post, or for myself, use to parse out onto other pages.

BTW romeshomey, you should really edit previous posts rather than posting a stream of them.

[romeshomey]

Originally Posted by Blackjack

BTW romeshomey, you should really edit previous posts rather than posting a stream of them.

What do you mean by that? The purpose of the addition was so that I or my other moderators didnt have to spend time looking for and posting articles for readers into the forums.

Im not sure what you are trying to tell me to edit..

[rsuplido]

Just wondering what version of RSS can this hack handle. The other hack can only handle RSS 0.91. Can it parse RSS 2.0, like that of VB3's?

Thanks.

[romeshomey]

Originally Posted by rsuplido

Just wondering what version of RSS can this hack handle. The other hack can only handle RSS 0.91. Can it parse RSS 2.0, like that of VB3's?

Thanks.

Yes it can. All of my RSS feeds are 2.0 and are working fine. Just wish the html special characters didnt show as well.

[romeshomey]

I don't see anything on this one, it wont parse anything if nothing is there for it to parse.. You sure the link is right? http://www.mini2.com/news/rss.php

Then this one should work fine. http://www.motoringfile.com/rss.xml

Everytime I make an edit in my ACP to add or update a feed for my bot, I have to run the forums/articlebot/articlebot.php manually by loading the page before my feeds will start posting.

So try that. But your 1st feed link is completely blank.. It might have something to do with it not being an XML extension.

[rsuplido]

Originally Posted by romeshomey

Yes it can. All of my RSS feeds are 2.0 and are working fine. Just wish the html special characters didnt show as well.

Thanks for the info.

Suggestion: can this hack handle ATOM in the future?

[Velocd]

Just wondering what version of RSS can this hack handle. The other hack can only handle RSS 0.91. Can it parse RSS 2.0, like that of VB3's?

Thanks.

MagpieRSS handles the following, as noted in the install.html:

RSS 0.9x, RSS 2.0, RSS 1.0, and Atom 0.3

Very nice hack, installed and testing at the moment, however I've got a problem (and it started off so well too!).

Anyway, I have several feeds, all but two work. If I exclude these two feeds, the other feeds (3 other feeds) work 100%.

The error I get is this (when I fun from command line):

How strange. articlebot.php requires functions_search.php only if you have the similar thread feature enabled, so that similar threads are generated for the threads.

Your error would probably disappear if you commented the fetch_similar_threads() function call in articlebot.php, like so:

PHP Code:

// $similarthreads = fetch_similar_threads($article['title']); 


I'll make a modification to that hack that allows you to toggle whether to grab similar threads per RSS entry (so you can disable it for RSS files that give errors).

I mean in addition to adding the URL and the time limit and the fetch limit and initially skip options, add a field that the board admin can specify as prefix for the titles, like Poll: or Moved: or Sticky:, in this case [News] or [Blues]. Something that people can use to identify that it is an automated post, or for myself, use to parse out onto other pages.

BTW romeshomey, you should really edit previous posts rather than posting a stream of them.

yeah, I'll incorporate this feature.

Everytime I make an edit in my ACP to add or update a feed for my bot, I have to run the forums/articlebot/articlebot.php manually by loading the page before my feeds will start posting.

This isn't a bug, but just how it is. When you add an RSS feed, the "next post" time will be the time it was added, plus the interval in minutes. So you have to wait.

If you set the interval to 1 minute, then it should proceed automatically. I'm assuming you are using a cron job.

Thanks though, I'd love to get those tags either removed or working, either or. Let me know if or when you figure out how to remove or enable them

I must have forgot, but the code is currently using strip_tags() to remove basic HTML tags. Whether this removes XHTML or more extensive, I'm not sure. I'll look into a preg_replace() expression that will remove HTML tags.

[romeshomey]

Originally Posted by Velocd

I must have forgot, but the code is currently using strip_tags() to remove basic HTML tags. Whether this removes XHTML or more extensive, I'm not sure. I'll look into a preg_replace() expression that will remove HTML tags.

Thanks..

I also noticed, it wont post an article unless the article has a 'discription' in it. It wont post just a title and link. If the feed doesnt include discription tags, it seems to skip those articles.

Not that I need it too, but just pointing that out.