NEW Modified RSS News Feed Hack

Hack originally posted by: Xyphen - RSS News Feed Hack
Modified by: wolfstream - Modified RSS News Feed Hack

I am posting this up as a new hack because of the major changes made to how the script works. Also, I didn't want people to get confused as to which one to download since there were 3 different versions on the original post.

Xyphen released the original version, then wolfstream modified the script and corrected some errors. After that, I modified wolfstreams version to change how posts were inserted. Now the post count will increase, posts will be able to be searched, and similiar threads will work.

RSS News Feed Hack for vB 3.0.x. As long as vBulletin doesn't change the database structure much, this should be good for the next release also, and those to come.

Allows you to have a live RSS news feed (such as one from Google or CNET) in a forum. The user that posts it will basically be a bot, which *should* only be used for posting the news, but you can even make it for a regular user.

Comments are welcome! Support provided through this thread.

Latest Version:
- 1.26 - for PHP 4.3.0 & above
- 1.26a - for PHP prior to 4.3.0

vBulletin Versions:
- 3.0.0
- 3.0.1

PHP Versions:
- > 4.3.0
- < 4.3.0

Installation Overview:
- Files to edit: (2) /admincp/index.php, /includes/functions_newpost.php
- Files to upload: (4) rss.php, admincp/rss_admin.php, includes/RSS/class.RSS.php, includes/RSS/rss_update.php
- Queries to run: (4) in rss_install.php

History:
Version 1.26 (23 June 2004 - Slapyo, Natch)
- listing of bot's more intuitive (Natch)
- hopefully fixed duplicate problem (Slapyo)
- stripped html tags from title and description (Slapyo)

Version 1.25 (12 May 2004 - Slapyo)
- fix for &quot; showing up
- logs action in scheduled task log if turned on

Version 1.24 (20 April 2004 - Slapyo)
- posts now deleted with delete_thread function
- post counts updated when using delete all news

Version 1.23 (14 April 2004 - Slapyo)
- added ability to delete bots
- cleaned up includes
- fix for PHP versions prior to 4.3.0 calling html_entity_decode()

Version 1.22 (13 April 2004 - Slapyo)
- duplicate posts problem fixed
- fix for problem with ' appearing as \'

Version 1.21 (12 April 2004 - Slapyo)
- bot will not be subscribed to posts now

Version 1.2 (11 April 2004 - Slapyo)
- posts now inserted with build_new_post function

Version 1.1 (28 March 2004 - wolfstream)
- file edits taken down to 1
- cron script included

Version 1.0 (28 January 2004 - Xyphen)
- original release

Common RSS Feeds:
- Yahoo!
- CNET.com
- BBC
- Wired News
- The Register
- Ars Technica
- All Headline News

[Slapyo]

Originally Posted by Bobby S

Slapyo: awesome hack many thanks

I have a couple of problems:

with the feed from:

http://www.masshibsteria.com/home/in..._rss&no_html=1

I get duplicates. Viz your comment above is it because the feed lacks a pub date?

Also from this feed:

http://sport.scotsman.com/football_h...cfm?format=rss

some of the stories have an apostrophe "Hibees' strike stars will be battling it out for Riordan's crown"

and these stories are repeating and the ' is being replaced by

"Hibees&#x27; strike stars will be battling it out for Riordan&#x27;s crown"

Any ideas?

ok bobby, here is what i can see.

your first feed has no pubDate field so it is not able to check if it repeats because it lacks that field.

your second feed having the ' and " in the title may be causing a problem because as you noted &#x27; does not equal ' when comparing strings.

i will look into the 2nd problem some more when i get a chance. i have 2 more finals today that are back to back and then i graduate on sunday. will have lots more time after that to work on it.

[Slapyo]

Originally Posted by andyhodges

I am using: http://rss.news.yahoo.com/rss/celebrity

And it does have a <pubDate> in it. It's grabbing the same top story over and over again.

-Andy

andy, your problem may be the same problem as bobby ... having the ' or " in the title may mess up when it checks for duplicates. because in the database they are not stored as " or ' but html encoded. while they appear the same, the string is different. i'm gonna look more into this.

[andyhodges]

Can anyone else get this one to work? http://news.search.yahoo.com/usns/yn...?p=half-life+2

It doesn't appear to be working at all.

-Andy

[Slapyo]

what does it not do? does anything show up? it appears to have all the proper fields, but it isn't rss, it is rdf which is slightly different, notice the CDATA tags around the title.

[andyhodges]

Yeah, it doesn't show up at all.. doesn't do anything. It would be cool if this would work, because you can search for specific news. Unless, you know of one?

-Andy

[proxyMX]

Originally Posted by yeloc

Is there a way not to add the news posts to the total posts number?

how can i moderate what news gets shown, Moderate Usergroup dosent seem to be working....

[Slapyo]

Originally Posted by andyhodges

Yeah, it doesn't show up at all.. doesn't do anything. It would be cool if this would work, because you can search for specific news. Unless, you know of one?

-Andy

http://www.syndic8.com

you can look around on there for news feeds.

[Slapyo]

Originally Posted by proxyMX

how can i moderate what news gets shown, Moderate Usergroup dosent seem to be working....

what exactly do you mean?

[proxyMX]

Originally Posted by Slapyo

what exactly do you mean?

i want to select exactly what appears on my site before the bot posts it

[Intex]

Slapyo - I've just had a thought about why it might be duplicating posts for certain forums. If you've included the slight code modification highlighted by yourself in Post 222 of this thread (which strips HTML codes from the thread title), then could it be that because tags are being removed, the titles are different. Therefore, when a comparison is made, they fundamentally look different and therefore get posted as a new thread?

The reason this happens for some forums and not others, is because some RSS feeds don't use HTML tags and some do?

What do you reckon?

[Slapyo]

Originally Posted by proxyMX

i want to select exactly what appears on my site before the bot posts it

oh you want to moderate which posts come in ... hrm, unless there is a built in option to moderate posts by forum, it's not built into the rss feed. sorry.

[Slapyo]

Originally Posted by Intex

Slapyo - I've just had a thought about why it might be duplicating posts for certain forums. If you've included the slight code modification highlighted by yourself in Post 222 of this thread (which strips HTML codes from the thread title), then could it be that because tags are being removed, the titles are different. Therefore, when a comparison is made, they fundamentally look different and therefore get posted as a new thread?

The reason this happens for some forums and not others, is because some RSS feeds don't use HTML tags and some do?

What do you reckon?

true, that could also be it. maybe a comparison of the title with html tags stripped to a comparison of the title from the feed with html tags stripped.

[dethfire]

Fatal error: Call to undefined function: html_entity_decode() in /home/street44/public_html/forum/includes/RSS/rss_update.php on line 77

[Bobby S]

I've had a think about my problem using feeds without a publication date.

Would it be possible to modify the code, so that it only checks for similarly titled threads? Without any date checking? I think that would solve my problems at least for the meantime

[Intex]

I think that's a good idea and probably would solve it. You're not likely to get the same headline again - so that would work for me also.

Slapyo - any ideas?