Article Bot
 

Article Bot
by velocd


Article Bot is a script that allows you to assign a user account (preferably an exclusive one) to an article bot that will post RSS feeds in select forums at designated intervals. These articles are derived from RSS (Really Simple Syndication) feeds.

RSS files are assigned to article bots, as well to individual forums, which in turn directs what, when and where your article bot will post.

Features

• Works with RSS versions: 0.9x, 2.0, 1.0, and Atom 0.3
• AdminCP interface for adding/modifying/deleting article bots & rss feeds
• Designate intervals (in minutes) to each RSS feed for when your article bot will post them.
• Requires only 1 file edit (admincp/index.php), allowing ease of upgrading your vBulletin
  without having to remodify files.
• Easy to install. Works best in conjunction with crontab (contab help is inside install.html)

News

• 7/25/04 - Version 1.1 Released
  • Bug Fixes
    • Replaced htmlspecialchars() with htmlentities() in articlebot.php to fully convert special entities from descriptions.
    • Added trim() to thread titles, to prevent double threads from being posted that are merely off by whitespace.
    • New table articlebot_rss_cache for storing a copy of article bot threads, for quicker checking of doubles when posting new threads.
    • Threads are checked in articlebot_rss_cache, so you can permanently remove them from the thread table (you don't have to "soft delete") without worrying about the bot replicating them.
    • Other minor bugs.
  • Features
    • New installer/upgraders for doing the SQL part automatically.
    • Added "label" field to add/modify RSS interface, that allows you to tag a label, e.g. [yahoo], to the beginning of threads, unique to each RSS feed. This might help for better distinction of RSS feeds.
    • Added an option to toggle on/off of building similar threads per RSS feed. Some RSS feeds could hault the script if the building of similar thread results for that feed takes too long (maybe the article thread title is too vague, and matches a lot of results--my guess is it's only a problem on large forums).
    • Added an option to toggle on/off skipping of articles that don't contain descriptions.
    • Regular expression search and replace for article titles and descriptions. This grants you much flexability to filter out things you don't want in your titles or descriptions (given you have some regex experience, I will try to post some patterns in due time.. feel free to add yours ;))
• 7/11/04 - Version 1.0 Released

Upgrading

• To upgrade, reupload the new PHP and MagpieRSS parser files to their places on your server. Upload upgrade_vx-x (replace the x's with the version you are upgrading to) to /path/to/forums/admincp/, and then execute that script.

Files

• 3 screenshots attached
• articlebot_v1.zip
  • install.html, logo.gif
  • articlebot_admin.php
  • articlebot/
    • rss_cache.inc
    • rss_fetch.inc
    • rss_parse.inc
    • rss_utils.inc
    • extlib/
      • Snoopy.class.inc

RSS Feeds

There are several in the install.html, but if you're looking for more than you could ever want, check out NewsIsFree's directory:

http://www.newsisfree.com/sources/bycat/


Remember to click install if you use this hack. ;)

Also be sure to read through the what you haven't of the thread before asking a question, for it may have been answered.

Ah yeah, enjoy the modification. ;)

This looks great well done and thanks, Does this mean we have to like write articles or can it automatically pull them from another site or something?

It pulls articles from RSS feeds. RSS is a xml format primarily used for syndicating news. Many news related sites provide RSS files of their recently posted articles so their readers can use an RSS aggregator (e.g. http://www.sharpreader.net) to read the data.

With PHP's XML libraries, parsing RSS is done very easily.

Sample RSS:
http://rss.news.yahoo.com/rss/topstories

Anyhow, you first create an article bot, by assigning it a user it will control.

You add an RSS feed by specifying the path to the RSS file, the article bot it applies to, the forum the thread will be posted in, and an interval (in minutes) that a new thread will be posted.

It's preferred that you use a crontab to execute the articlebot.php file, every couple hours or so. The script will work from the bottom of the RSS file (oldest articles) to the top (latest articles). It will post the title of the article as the thread subject, and then the RSS description as the body content. It then tags the URL to the actual full article.

I've had it running on my forum for about 2 weeks, and it works quite well. Make the intervals quite large, like every 5 hours, or else you'll have a flood of bot threads on your forum. ;) This of course depends on the RSS feed and how often the news source updates it.

Hopefully I didn't make things more convoluted. :rolleyes:

Nice I like this a lot :)

There is another hack just like this one that I installed a month or two ago. :) How is this different to that one? Just wondering.

Hm, I didn't know others existed.

Well, it's different in that it's coded by another person. ;)

Feature wise, you'll just have to try it out. Article bots can have unlimited RSS feeds assigned to them, and RSS feeds can be in multiple forums. Meaning, 1 article bot can post in many forums. Not sure if the other RSS mods are restricted of this.

It's very query efficient also.

cool

Is there a way to create an RSS feed from your site, for other sites with this hack? (ie: new events on my forum automaticaly get RSS'd to other sites)

alkatraz - I could be mistaken but I believe thats a default feature of vb3. Check the online manual for it.

crazy, just checked the vb3 manual..

thx

would be good from Erwin to post a link, so we can all compare!

Erwin is talking about Slapyo's New Modifies RSS feed hack - there is a search button for just such an occasion ;) :D

This is a great hack (y)

Ok stupid question, where do you get RSS feeds from? I mean what site will this info be pulled from? I run a F0RD website, where would i find info on stuff like that?

*sorry disregard opened the zip and you have lots of feeds for me :-D

Silly question time, but would this work on 3.01 as iam not planning on updating yet

nice job! *clicks install*

This hack will work on all vBulletin 3 versions.

It doesn't seem to like table prefixes. :ermm:

Appears to be line 377 in articlebot_admin.php

Replace
With

How do we find RSS sites :/ Think you could post a list of sites please?

It would depend on what you are interested in. If you want gaming, try your favorite fansite and look for their RSS link. If you want world news, head to a news website and find their link.

There are tons of links, hard to just post some that you may or may not like. He already gave examples in the install file.

*edit
Velocd, how about the option to prefix the titles with something we can use to identify the different feeds. Like a feed from Blues could be [Blues] Morning Q&A's and ShackNews could show as [Shack] Game Reviews.

The cron also shows up in the whois online as the base path.

Great - love RSS!

Ford blogs? are they out there?

Google is your best bet to find blogs. There are thousands out there, search around. use keywords like Movabletype, Los Angeles Blogs, Xanga.com, Blogger, etc. If you want more traditional sources, find large news sites, like BBC, that provide RSS files.

Hm, thought I got them all. Will make the change.

Ah, although I don't believe the RSS parser I use (MagpieRSS) grabs that information.

I'll take a second look though. ;)

Given the cron has the same IP as your server, you could probably do a comparison in online.php with $_SERVER["SERVER_ADDR"] and exclude that entry. I'll look into this also. Expect an update probably Sunday.

Im having a problem and not sure what it is.
Im really not sure what to put in the articlebot.php which Im sure is why Im having that error when I try to execute it.

Anyone see what I might have wrong?

I tried this
Still getting an error, but not sure what I have wrong...

Ok, no errors now, but also don't see anything when the php runs... Is that normal?

http://www.insidersportszone.com/for...articlebot.php

Hmm, something isnt right. Its either not reading the feed I have in there or isnt working altogether. Not sure which.

I have this feed running on other sites, so should work here as well, but isnt.

http://xml.newsisfree.com/4/43/4306f...6d8c1dd74c.xml

Ok, well its working as far as showing a feed on the forums.

Problem now is, it looks like this..

How do I get it to not show the <br clear=all> ??

Is there a way to get it to read the html in the RSS feeds?

I have HTML enabled in the forums, but when my Bot posts the HTML in the feeds stil show as <b> and </b> etc...

Is there a way to enable it to read the HTML so the bold is bold and etc?

We will be having news feeds from two of our sites by the end of this month.

I have not seen any others as of yet that are Ford Specific. I have not been looking too hard, but i have had a look.

I will look into this. I would have suspected that enabling HTML on the forums would show the HTML, but guess not. It's a security hazard to enable HTML for posts, btw, and not recommened.

I'm thinking I'll probably throw in strip_tags() to remove the HTML tags. Good RSS feeds don't provide HTML anyway, so you might look for others.

All of ESPN's feeds have HTML, and seeing as Im a sports forum, then I dont have alot of choices to choose from as far as getting the latest up to date feeds in those areas.

Thanks though, I'd love to get those tags either removed or working, either or. Let me know if or when you figure out how to remove or enable them :)

I love the addition of the Bot, although he may surpass the world posting records! lol

I mean in addition to adding the URL and the time limit and the fetch limit and initially skip options, add a field that the board admin can specify as prefix for the titles, like Poll: or Moved: or Sticky:, in this case [News] or [Blues]. Something that people can use to identify that it is an automated post, or for myself, use to parse out onto other pages.

BTW romeshomey, you should really edit previous posts rather than posting a stream of them.

What do you mean by that? The purpose of the addition was so that I or my other moderators didnt have to spend time looking for and posting articles for readers into the forums.

Im not sure what you are trying to tell me to edit..

Just wondering what version of RSS can this hack handle. The other hack can only handle RSS 0.91. Can it parse RSS 2.0, like that of VB3's?

Thanks.

Yes it can. All of my RSS feeds are 2.0 and are working fine. Just wish the html special characters didnt show as well.

I don't see anything on this one, it wont parse anything if nothing is there for it to parse.. You sure the link is right? http://www.mini2.com/news/rss.php

Then this one should work fine. http://www.motoringfile.com/rss.xml

Everytime I make an edit in my ACP to add or update a feed for my bot, I have to run the forums/articlebot/articlebot.php manually by loading the page before my feeds will start posting.

So try that. But your 1st feed link is completely blank.. It might have something to do with it not being an XML extension.

Thanks for the info.

Suggestion: can this hack handle ATOM in the future? ;)

MagpieRSS handles the following, as noted in the install.html:

RSS 0.9x, RSS 2.0, RSS 1.0, and Atom 0.3

How strange. articlebot.php requires functions_search.php only if you have the similar thread feature enabled, so that similar threads are generated for the threads.

Your error would probably disappear if you commented the fetch_similar_threads() function call in articlebot.php, like so:

I'll make a modification to that hack that allows you to toggle whether to grab similar threads per RSS entry (so you can disable it for RSS files that give errors).

yeah, I'll incorporate this feature. ;)

This isn't a bug, but just how it is. When you add an RSS feed, the "next post" time will be the time it was added, plus the interval in minutes. So you have to wait.

If you set the interval to 1 minute, then it should proceed automatically. I'm assuming you are using a cron job.

I must have forgot, but the code is currently using strip_tags() to remove basic HTML tags. Whether this removes XHTML or more extensive, I'm not sure. I'll look into a preg_replace() expression that will remove HTML tags.

Thanks..

I also noticed, it wont post an article unless the article has a 'discription' in it. It wont post just a title and link. If the feed doesnt include discription tags, it seems to skip those articles.

Not that I need it too, but just pointing that out.