NEW Modified RSS News Feed Hack

[killervette]

me 2 ^

[Slapyo]

what news feeds are you using? the requirement from the news feed is there has to be a field <pubDate> in it ... otherwise it won't grab the date and it won't be able to check for duplicates since it checks for the title & the date.

post your news feeds that you are using.

[Bobby S]

Slapyo: awesome hack many thanks :)

I have a couple of problems:

with the feed from:

http://www.masshibsteria.com/home/index2.php?option=com_rss&no_html=1

I get duplicates. Viz your comment above is it because the feed lacks a pub date?

Also from this feed:

http://sport.scotsman.com/football_hibernian.cfm?format=rss

some of the stories have an apostrophe "Hibees' strike stars will be battling it out for Riordan's crown"

and these stories are repeating and the ' is being replaced by

"Hibees&#x27; strike stars will be battling it out for Riordan&#x27;s crown"

Any ideas? :)

[Bobby S]

I should add, I've added 3 other newsfeeds without any problem :)

[andyhodges]

what news feeds are you using? the requirement from the news feed is there has to be a field <pubDate> in it ... otherwise it won't grab the date and it won't be able to check for duplicates since it checks for the title & the date.

post your news feeds that you are using.

I am using: http://rss.news.yahoo.com/rss/celebrity

And it does have a <pubDate> in it. It's grabbing the same top story over and over again.

-Andy

[Slapyo]

Slapyo: awesome hack many thanks :)

I have a couple of problems:

with the feed from:

http://www.masshibsteria.com/home/index2.php?option=com_rss&no_html=1

I get duplicates. Viz your comment above is it because the feed lacks a pub date?

Also from this feed:

http://sport.scotsman.com/football_hibernian.cfm?format=rss

some of the stories have an apostrophe "Hibees' strike stars will be battling it out for Riordan's crown"

and these stories are repeating and the ' is being replaced by

"Hibees&#x27; strike stars will be battling it out for Riordan&#x27;s crown"

Any ideas? :)
ok bobby, here is what i can see.

your first feed has no pubDate field so it is not able to check if it repeats because it lacks that field.

your second feed having the ' and " in the title may be causing a problem because as you noted &#x27; does not equal ' when comparing strings.

i will look into the 2nd problem some more when i get a chance. i have 2 more finals today that are back to back and then i graduate on sunday. will have lots more time after that to work on it.

[Slapyo]

I am using: http://rss.news.yahoo.com/rss/celebrity

And it does have a <pubDate> in it. It's grabbing the same top story over and over again.

-Andy
andy, your problem may be the same problem as bobby ... having the ' or " in the title may mess up when it checks for duplicates. because in the database they are not stored as " or ' but html encoded. while they appear the same, the string is different. i'm gonna look more into this.

[andyhodges]

Can anyone else get this one to work? http://news.search.yahoo.com/usns/ynsearch/categories/news_story_search_rss/index.html?p=half-life+2

It doesn't appear to be working at all.

-Andy

[Slapyo]

what does it not do? does anything show up? it appears to have all the proper fields, but it isn't rss, it is rdf which is slightly different, notice the CDATA tags around the title.

[andyhodges]

Yeah, it doesn't show up at all.. doesn't do anything. It would be cool if this would work, because you can search for specific news. Unless, you know of one?

-Andy

[proxyMX]

Is there a way not to add the news posts to the total posts number?
how can i moderate what news gets shown, Moderate Usergroup dosent seem to be working....

[Slapyo]

Yeah, it doesn't show up at all.. doesn't do anything. It would be cool if this would work, because you can search for specific news. Unless, you know of one?

-Andy
http://www.syndic8.com

you can look around on there for news feeds.

[Slapyo]

how can i moderate what news gets shown, Moderate Usergroup dosent seem to be working....
what exactly do you mean?

[proxyMX]

what exactly do you mean?
i want to select exactly what appears on my site before the bot posts it

[Intex]

Slapyo - I've just had a thought about why it might be duplicating posts for certain forums. If you've included the slight code modification highlighted by yourself in Post 222 (https://vborg.vbsupport.ru/showpost.php?p=518449&postcount=222) of this thread (which strips HTML codes from the thread title), then could it be that because tags are being removed, the titles are different. Therefore, when a comparison is made, they fundamentally look different and therefore get posted as a new thread?

The reason this happens for some forums and not others, is because some RSS feeds don't use HTML tags and some do?

What do you reckon?

[Slapyo]

i want to select exactly what appears on my site before the bot posts it
oh you want to moderate which posts come in ... hrm, unless there is a built in option to moderate posts by forum, it's not built into the rss feed. sorry.

[Slapyo]

Slapyo - I've just had a thought about why it might be duplicating posts for certain forums. If you've included the slight code modification highlighted by yourself in Post 222 (https://vborg.vbsupport.ru/showpost.php?p=518449&postcount=222) of this thread (which strips HTML codes from the thread title), then could it be that because tags are being removed, the titles are different. Therefore, when a comparison is made, they fundamentally look different and therefore get posted as a new thread?

The reason this happens for some forums and not others, is because some RSS feeds don't use HTML tags and some do?

What do you reckon?
true, that could also be it. maybe a comparison of the title with html tags stripped to a comparison of the title from the feed with html tags stripped.

[dethfire]

Fatal error: Call to undefined function: html_entity_decode() in /home/street44/public_html/forum/includes/RSS/rss_update.php on line 77

[Bobby S]

I've had a think about my problem using feeds without a publication date.

Would it be possible to modify the code, so that it only checks for similarly titled threads? Without any date checking? I think that would solve my problems at least for the meantime :)

[Intex]

I think that's a good idea and probably would solve it. You're not likely to get the same headline again - so that would work for me also.

Slapyo - any ideas?

[Bobby S]

http://sport.scotsman.com/football_hibernian.cfm?format=rss

some of the stories have an apostrophe "Hibees' strike stars will be battling it out for Riordan's crown"

and these stories are repeating and the ' is being replaced by

"Hibees&#x27; strike stars will be battling it out for Riordan&#x27;s crown"

Any ideas? :)

I'm going to try manually editing the topic titles as they come in: that would hopefully solve that problem in the meantime. I'll let you know if that works. :)

[Slapyo]

I've had a think about my problem using feeds without a publication date.

Would it be possible to modify the code, so that it only checks for similarly titled threads? Without any date checking? I think that would solve my problems at least for the meantime :)
here you go, in rss_update.php:

FIND:

// filter out the old items
while($rss_thread_data = $DB_site->fetch_array($rss_query)) {
for($j = count($rss_allItems) - 1; $j >= 0; $j--) {
if($rss_thread_data[title] == $rss_allItems[$j][TITLE] &&
$rss_thread_data[rss_date] == $rss_allItems[$j][pubDate]) {
$rss_allItems[$j][OLD] = true;
}
}
}


REPLACE WITH:

// filter out the old items
while($rss_thread_data = $DB_site->fetch_array($rss_query)) {
for($j = count($rss_allItems) - 1; $j >= 0; $j--) {
if($rss_thread_data[title] == $rss_allItems[$j][TITLE]) {
$rss_allItems[$j][OLD] = true;
}
}
}

[Erwin]

Anyone seen this? I get this as a cron error every hour:


Fatal error: Maximum execution time of 30 seconds exceeded in /home/public_html/includes/functions_newpost.php on line 690


I suspect one of the news feed is timing out. Is there a way to prevent this error being emailed to me every hour if a news feed times out?

Just wondering, and thanks for the hack.

[Slapyo]

hm, i think you would have to add in a script to watch the time and if the execution time hit 29 seconds to stop. not sure exactly how to put that in though.

[checknme12]

Stange... :( I have the newest version of the hack installed but I am still getting duplicate stories. Anyone know why this might happen? I went to bed and now I have like 7 stories saying "Intel's 3D divorce rate" and another 7 duplicating another story and so on. The duplicates are not in a row.

Please help :) My bot is nuts.

[Slapyo]

what is the link to your RSS feed?

[Kraydl]

I found this mod for RSS feeds. The code may help you discover how others tackle the post duplication problem.

http://www.w3reports.com/forumfeeds.php

[dethfire]

I found this mod for RSS feeds. The code may help you discover how others tackle the post duplication problem.

http://www.w3reports.com/forumfeeds.php

looks like it's the same script... copyright issues?

[Slapyo]

that was one of the original rss feed scripts that doesn't insert the news with the function built into vB. no search works, no post counts go up, nada.

[smsmasters]

I get 30 second max execution errors on my forum at www.smsmasters.co.uk My members got so fed with 17 pages worth of news in Today's Posts. Is there a way to stop these posts appearing in Today's Posts or New Posts???

[Slapyo]

no because they are new posts, however this has been suggested and you can fix it by opening up the file and modifying the search criteria for today's posts or new posts.

there is a new field in the tableprefix_THREAD table and it is called rss_feed. just add this into the query:

WHERE rss_feed!=1

that should work. not at my computer right now so i could double check, but that should eliminate all news posts from being brought up since they will have a 1 in the column and normal fields won't.

[smsmasters]

Thanks, how about the max execution time of 30 seconds? It keeps timing out!

[smsmasters]

I added : http://www.smsmasters.co.uk/forum/search.php?do=getdaily&exclude=66,67,68,69,70,72,73,74,75,76

[Slapyo]

the time out could be related to your feed you are grabbing. what is the URL to the news feed. the script doesn't take long to run at all. but it could be timing out waiting for the rss file.

[smsmasters]

well, I have about <10 newsfeeds:

http://rss.news.yahoo.com/rss/software
http://rss.news.yahoo.com/rss/tech
http://rss.news.yahoo.com/rss/internet
http://rss.news.yahoo.com/rss/topstories
http://rss.news.yahoo.com/rss/wireless
http://news.bbc.co.uk/rss/newsonline_world_edition/front_page/rss091.xml
http://news.bbc.co.uk/rss/newsonline_world_edition/uk_news/rss091.xml
http://news.bbc.co.uk/rss/newsonline_world_edition/technology/rss091.xml
http://news.bbc.co.uk/rss/newsonline_world_edition/entertainment/rss091.xml
http://news.bbc.co.uk/rss/newsonline_world_edition/health/rss091.xml

[smsmasters]

Hmm, strange, if I run the cron job manually it times out, but if I let ir run automatically, it's fine!

[Slapyo]

it might time out because of the number of feeds.

[Intex]

The 30 second timeout is a PHP function. You can change this to whatever value you like in your PHP.INI I believe.



Thanks, how about the max execution time of 30 seconds? It keeps timing out!

[Slapyo]

I found this mod for RSS feeds. The code may help you discover how others tackle the post duplication problem.

http://www.w3reports.com/forumfeeds.php

ya, this problem has been resolved.

[rsuplido]

In their tutorial, there is a rssfeed.php file that is called from cron. Where can I get this?

Thanks.

[Slapyo]

you can call this script from cron if you want to.

call rss.php .... thats the file that triggers everything.

[Natch]

I have a small modification to the admincp/rss_admin.php to make the listing of Bot's more intuitive in the RSS News Feed Manager => Modify Bots ...

Find:// ########################## EDIT USER DISPLAY ############################
} else if ($_REQUEST['do'] == 'editUsers') {
print_form_header('rss_admin', 'update_editUser');
print_table_header('Edit a Bot');
$var = "<select name=\"rss_userid\" id=\"sel_rss_userid\" class=\"bginput\">";

$query = $DB_site->query("SELECT * FROM " . TABLE_PREFIX . "rss_user");
while($data = $DB_site->fetch_array($query)) {
$rss_userid = $data['rssuserid'];
$rss_username = $data['userid'];
Add after: $rss_source = $data['source'];
Then Find [four lines down from the last modification]: $var .= "<option value=\"$rss_userid\">$rss_username (id: $rss_userid)</option>";
Replace with: $var .= "<option value=\"$rss_userid\">$rss_source (feed id: $rss_userid)</option>";


HTH

[Intex]

Slapyo - I'm lost. How is the duplication issue being resolved?



you can call this script from cron if you want to.

call rss.php .... thats the file that triggers everything.

[Intex]

Natch - very useful, thx :).



I have a small modification to the admincp/rss_admin.php to make the listing of Bot's more intuitive in the RSS News Feed Manager => Modify Bots ...

Find:// ########################## EDIT USER DISPLAY ############################
} else if ($_REQUEST['do'] == 'editUsers') {
print_form_header('rss_admin', 'update_editUser');
print_table_header('Edit a Bot');
$var = "<select name=\"rss_userid\" id=\"sel_rss_userid\" class=\"bginput\">";

$query = $DB_site->query("SELECT * FROM " . TABLE_PREFIX . "rss_user");
while($data = $DB_site->fetch_array($query)) {
$rss_userid = $data['rssuserid'];
$rss_username = $data['userid'];
Add after: $rss_source = $data['source'];
Then Find [four lines down from the last modification]: $var .= "<option value=\"$rss_userid\">$rss_username (id: $rss_userid)</option>";
Replace with: $var .= "<option value=\"$rss_userid\">$rss_source (feed id: $rss_userid)</option>";


HTH

[Kraydl]

The post duplication problem.

ya, this problem has been resolved.

Slap, does the latest version have that fix in it?
Thanks.

[Slapyo]

nope, but if intex doesn't mind i would like to add this into the full release version. intex?

[Slapyo]

The post duplication problem.



Slap, does the latest version have that fix in it?
Thanks.
actually the post duplicate problem hasn't been resolved yet and that code wouldn't help since it was mine. it was a really old version of the code and still has the same stuff in it, someone changed the "created by" in the top of the rss_update.php. what i meant, was the problem had been resolved from them hosting the file off of their site.

[Kraydl]

Ah okay. I'm going to hold off installing that yet till the dup thing is fixed. Great work on the effort you have put into it so far.

[Slapyo]

i'm gonna work on the dup thing today. check back later on, might be updated.

[Slapyo]

ok, i am at work and made some changes but can't test it out.

Version 1.26 (23 June 2004 - Slapyo, Natch)
- listing of bot's more intuitive (Natch)
- hopefully fixed duplicate problem (Slapyo)
- stripped html tags from title and description (Slapyo)

for comparing the RSS feed with whats in the database, i just applied the same things i do before inserting the post. so it should be the same. i also applied Natch's hack to make listing the bots easier.

files changed:
- rss_admin.php
- rss_update.php

[dethfire]

ok, i am at work and made some changes but can't test it out.

Version 1.26 (23 June 2004 - Slapyo, Natch)
- listing of bot's more intuitive (Natch)
- hopefully fixed duplicate problem (Slapyo)
- stripped html tags from title and description (Slapyo)

for comparing the RSS feed with whats in the database, i just applied the same things i do before inserting the post. so it should be the same. i also applied Natch's hack to make listing the bots easier.

thanks!!! do we have to run any queries to update or just overwite the files?

[dethfire]

running the update

Database error in vBulletin 3.0.0:

Invalid SQL: ALTER TABLE rss_settings DROP forumid, DROP userid, DROP source
mysql error: Can't DROP 'forumid'. Check that column/key exists

mysql error number: 1091

[Slapyo]

sorry, no need to run the update. actually i should remove those from the install cause there is nothing to update from really. that was from the previous script.

the rss_admin.php and rss_update.php were the only files edited, so you just have to upload those.

[Natch]

Thanks - I'll let you know how this goes with the dup. posting ...

[Intex]

Slapyo - the only additional code I offered was excluding certain forums from the search (utilising 2 search scripts). If this is what you meant, go ahead, fine by me.



nope, but if intex doesn't mind i would like to add this into the full release version. intex?

[smsmasters]

Where's the code to exclude certain forums from search?

[Intex]

I posted it way back in this thread HERE (https://vborg.vbsupport.ru/showpost.php?p=519128&postcount=232).

[Slapyo]

sorry intex, i meant Natch, but thought it was you because i saw the code in your post, but it was a quote of Natch. and yes, actually i would like to include your code to remove the posts from search because i know some people have requested that. thanks.

[Blackbeard]

Top 100 Most-Subscribed-To RSS Feeds

http://radio.xmlstoragesystem.com/rcsPublic/rssHotlist

hope this is of some use

[Blackbeard]

Done the install, set up a bot etc, then ran the schedule and got this error

Parse error: parse error in /home/httpd/vhosts/l**********.com/httpdocs/forums/includes/functions_newpost.php on line 1187

line 1187 is

?>

whats happening here

[Slapyo]

look and see if there is any extra white space at the end of the file. ?> should be the very last thing, there shouldn't be anything else after it, not even spaces.

[Intex]

I believe this is the most comprehensive database of all RSS Newsfeeds websites: http://www.syndic8.com/. Hopefully you'll find everything there you need.

[Blackbeard]

look and see if there is any extra white space at the end of the file. ?> should be the very last thing, there shouldn't be anything else after it, not even spaces.

Just checked that and no spaces, still click run now in schedule manager and same error message comes up and no posts have been made in the forum i set the bot up to.

[Slapyo]

the only thing i could suggest to you is to go and download vbulletin again ... then take:

/forums/includes/functions_newpost.php

so you have a fresh new copy. apply the edits to the file that are listed in the instructions, then try it again.

[Blackbeard]

Ok Thanks

[Natch]

Great work Slapyo - the duplicates seem to be gone :)

[Slapyo]

whew! good to know. lemme know if they are still there, or if you notice any other problems.

[Intex]

Yep, gone for me too. Thx. Slapyo.

[Deimos]

I've got a problem with it to
Installed it AOK, ran the cron job and it picked up the news from the page I wanted it to...that was 2/3 days ago and it hasn't updated/got anymore news since then
Any ideas why?
I re ran the cron job and it didn't update it either.

[gmtalk]

Is the news feed updated?

Some of my feeds are only updated M-F

[Deimos]

Fixed it, for some reason it didn't like my user id being the reporter, as soon as I set it to a new user I setup, it started posting ok.

I now have a new problem (yay...)
I accidently overwrote my forum index file with the one from the vbadvanced page...no problem, I restored it
But as soon as I load the forums, I get....

Deimos: Fatal error: Call to undefined function: build_new_post() in /home/deimos/public_html/forums/includes/RSS/rss_update.php on line 103

Anyone know why? I haven't changed any files
If I disable the news bots, it loads ok, but if I enable it, it breaks
Er...

[Slapyo]

ya, like gmtalk said ... sometimes the news feeds are not updated very often, maybe once or twice a week. what is the rss feed you are using. which website is it coming from?

[Slapyo]

hrm trying getting a new copy of /includes/functions_newpost.php and re-editing the files.

[Deimos]

Thanks slap, it was my mistake, I uploaded an old version of the index file which called the rss_update file

Btw, I have a question for you
I've installed or setup an RSS type news ripper, which grabs news from a URL..
It then puts this info into an RDF file
I've tried pointing the RSS news feed to this file, but it doesn't add the contents to the forum
The layout of the RDF file is....

Content-Type: text/html

<?xml version="1.0"?>
<rdf:RDF
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns="http://my.netscape.com/rdf/simple/0.9/">

<channel>
<title>ORIGIN - Ultima Online - Main</title>
<link>http://update.uo.com/fyi.html</link>
<description>FYI</description>
</channel>
<item>



&nbsp; <b>Ultima Registration Server Downtime</b>


Jun 25 2004 6:02PM CST (GMT -06:00)&nbsp;


<item>



Beginning at 10:00PDT on Friday, June 25th the Ultima Registration and account management servers may experience approximately 1 hour of downtime for maintenance.



</rdf:RDF>

The script i'm using to grab the news is....

http://www.openconcept.on.ca/guide-html2rdf.phtml

Now, i'm guessing it doesn't like the layout of the RDF file somehow, as it just won't add what's in there to my forum
The RSS does work aok for other news things like yahoo news etc, but not this custom thing.....

Any ideas ?

[Deimos]

By the way, if i'm asking for to much, I apologise

[rsuplido]

Have you all tried adding a feed coming from a VB3 RSS2 feed? It can't extract it. :( Any workaround?

[rsuplido]

Thanks slap, it was my mistake, I uploaded an old version of the index file which called the rss_update file

Btw, I have a question for you
I've installed or setup an RSS type news ripper, which grabs news from a URL..
It then puts this info into an RDF file
I've tried pointing the RSS news feed to this file, but it doesn't add the contents to the forum
The layout of the RDF file is....



The script i'm using to grab the news is....

http://www.openconcept.on.ca/guide-html2rdf.phtml

Now, i'm guessing it doesn't like the layout of the RDF file somehow, as it just won't add what's in there to my forum
The RSS does work aok for other news things like yahoo news etc, but not this custom thing.....

Any ideas ?

I'm getting the same problem and it seems like the includes\class.RSS.php file, supports only RSS 0.91. Also, if you look at your example, <channel> and </channel> is declared above. Usually <item></item> are in between the <channel></channel> tags. You might want to do some tweaking on the class.RSS.php file and remove the <channel> test.

Regards.

[Deimos]

Remove the channel test? hmm

[Slapyo]

goes like this:


<channel>
<item>
<title></title>
<link></link>
<pubDate></pubDate>
<description></description>
</item>
</channel>


that is the format that it follows, if there are more items, they go between the <channel> tags.

for now, this is just a RSS feed. i haven't had time to look into adding different kinds of feeds like RDF or ATOM. i might if i get some time.

[Deimos]

Thanks Slap, it'd be awesome if you could..

[Bobby S]

Thanks Slap, it'd be awesome if you could..
Slapyo,

Just to let you know that I can know pick up the Scotsman feeds that were previously a problem - the ones with the ' in them. It doesnae remove the spurious characters, but it doesnae duplicate, and it picks up the new items. Godsend and Top work fellow!

I'm now having problems with other feeds not adding anything new in. I know there could be a million reasons for this, so I'll report back when I've investigated a bit more so as not to waste your time.

Thanks again, an appreciative php tube :)

[Slapyo]

bobby, go ahead and post the links. maybe someone else will know or can look at it, or i can check it out as well. get a few heads in on this thinking about it.

[gmtalk]

I have been playing around and testing this. For the longest I was not having duplicate posts/threads for certain feeds and was only having problems with one feed duplicating. Now it is going nuts. I downloaded the latest version, removed the version that was in there and put in this latest one.

Feeds I am having issues with.

http://rss.crash.net/crash_1.xml (F1 Racing feed)

and

http://www.just-auto.com/affiliates/features_rss.asp

Now I have had issues with that just auto feed before and it cleared up, but now just came back last night. The crash.net feed is wierd. I have 4 other crash feeds that are not having any issues and have updated through out the day. It is only screwing up on the one listed above and only repeating one thread over and over, not everyone off that feed. The one repeating has not been updated on their site at all and is actually a few days old.

[Natch]

Do other RSS readers have the same problem ? with it constantly updating ?

[gmtalk]

Do other RSS readers have the same problem ? with it constantly updating ?


I have been testing that out and I would have to say no on the crash.net. The Just-auto did it only 2 times in the last week(checked for updates hourly just like the forum), but on my test forums it hit every call.

[Deimos]

Has anyone else had a problem where the rss just stops updating?
It was working ok, then suddenly it stopped reporting on the 29th of June, yet nothing has changed?
The news feed I was getting was the bbc one.

[sjau]

Where can I actually change the "View the entire article" to something else?

[Bobby S]

bobby, go ahead and post the links. maybe someone else will know or can look at it, or i can check it out as well. get a few heads in on this thinking about it.

I've added seperate users for each newsfeed [a list of ex girlfriends, top idea I thought] and the all post into my forum here:

http://www.masshibsteria.com/vb/forumdisplay.php?f=2

Now the Scotsman feed [Gillian] works perfectly http://www.masshibsteria.com/vb/showthread.php?t=7560
http://sport.scotsman.com/football_hibernian.cfm?format=rss

*since the last updated release, it duplicated prior to that

The Official Hibs website feed [Caroline] used to work perfectly, but since the file update it doesnae work. I ran the first cron job and it took the feeds, but since then nothing
http://www.masshibsteria.com/vb/showthread.php?t=7504
http://www.hibs.org.uk/news/index.xml

I've also added a bbc feed [Belinda] which has the same problems as the Official feed.
http://www.masshibsteria.com/vb/showthread.php?t=7496
http://news.bbc.co.uk/rss/sportonline_uk_edition/football/teams/h/hibernian/rss091.xml

****Now I think about it, I edited some code earlier in the thread, so as not to check for time: That might be the problem, so I'll go back and change it.******

Edit: I don't think that's the problem https://vborg.vbsupport.ru/showpost.php?p=523199&postcount=272 - I've overwritten that file with the new one. :(

[Slapyo]

been busy this weekend, will have a better chance to look at all this on tuesday sorry.

[Bobby S]

been busy this weekend, will have a better chance to look at all this on tuesday sorry.
No rush Donnie: if you can when you can would be most appreciated.

Getting the scotsman one to work was a BIG help for me, thank yous.

[Slapyo]

ok bobby, i have looked at the feeds. the 2 that are not working do not have a pubDate field. the new code in 1.26 changed how a news feed was marked as "old". instead of having to match the pubDate AND the title. now it has to match the pubDate OR the title. so if there is no pubDate, no worries...it can still match the title.

it could be that those news feeds aren't updated frequently? not sure. the only thing i noticed was that the feeds not having pubDate.

[scottct1]

Ok I need help...

I run a SatelliteTV related Website, I only want to pull news relating to SatelliteTV, so in this regard lets say we want news on DirecTV.

I found the following search below which pulls all the DirecTV news.

http://www.rocketnews.com/news/RocketinfoXML.jsp?must=directv&age=0

The problem is it does not pull the description, instead all's that I see in the message is VIEW THE ENTIRE ARTICLE.

How do I get it to show the descrption that is included.

Thanks

[Slapyo]

that is using rss 2.0 and i think the version of rss that is supported in the script right now is .91.

rss .91:

<channel>
<item>
<title></title>
<link></link>
<pubDate></pubDate>
<description></description>
</item>
</channel>


rss 2.0:

<channel>
<item>
<title></title>
<link></link>
<pubDate></pubDate>
<description><![CDATA[ ]]></description>
</item>
</channel>


although it should still work and your description would include CDATA stuff in them. not sure why the description is not coming up.

[Bobby S]

ok bobby, i have looked at the feeds. the 2 that are not working do not have a pubDate field. the new code in 1.26 changed how a news feed was marked as "old". instead of having to match the pubDate AND the title. now it has to match the pubDate OR the title. so if there is no pubDate, no worries...it can still match the title.

it could be that those news feeds aren't updated frequently? not sure. the only thing i noticed was that the feeds not having pubDate.

Both feeds are updated daily, almost without fail. I'll have a look into anything I may have changed that might affect them.

Cheers D.

:up:

[Slapyo]

lemme know if you figure anything out. i'll keep lookin at it as well.

[sjau]

Has anyone else had a problem where the rss just stops updating?
It was working ok, then suddenly it stopped reporting on the 29th of June, yet nothing has changed?
The news feed I was getting was the bbc one.

Mine stopped also. It displays: Next update on July 08 2004 12:45

What message should appear if you run it manually from the admincp?

And where can I change that "View entire article"?

[notorious]

Would someone be able to help with this:
I'm getting the following errors whe I tr to run the schedule task:

Warning: main(./rss.php): failed to open stream: No such file or directory in /homepages/4*/d********/htdocs/forum/admincp/cronadmin.php on line 56

Fatal error: main(): Failed opening required './rss.php' (include_path='.:/usr/local/lib/php') in /homepages/4*/d*******/htdocs/forum/admincp/cronadmin.php on line 56

This is what I have on line 56:
require_once($nextitem['filename']);

and I'm getting this on the rss.php:

Fatal error: Call to undefined function: log_cron_action() in /homepages/4*/d********/htdocs/forum/Rss.php on line 7

Which is:
log_cron_action('RSS News Feed Posted', $nextitem);

Any Help would be appriciated

[smsmasters]

Mine stopped also. It displays: Next update on July 08 2004 12:45

What message should appear if you run it manually from the admincp?

And where can I change that "View entire article"?

Same here, I've checked the BBC news feeds url manually and there are new news on it. However the news feed bot doesnt update, it's been 1 week now! The yahoo news feed works ok tho.

[RS25com]

So does this work on 3.03? I'm thinking about installing, but want to be sure before I do! :D

[Slapyo]

check to see if the BBC news feed changed their format. if they have that might be the cause of the problem.

@notorious, the only thing i can tell you is to double and triple check that you uploaded the files to the proper locations as well as made the proper file edits.

@RS25.com, it should work on 3.03 ... as long as there are no major changes to the forum it should work on any 3.x i'm thinking.

[sjau]

@ slapyo

all I can say is that it fetched the first time data from "my" news feed and never since on 3.0.2. Not even when triggered manually from the admin center.
Furthermore where can I change that text "View entire article?"?

[mrboz]

i have the same problem with it not updating too.

i have tried 2 different feeds from the bbc news website and another feed from a different site.

the first time you get the news it works very well, but it doesn't update anymore. i have checked the bbc feeds and they have been updated and i tried another feed with news on it and that didn't work at all.

[Slapyo]

hrm, i'm not sure. everything should be working fine as no major updates have been done to anything that would affect the script. i'll look at it some more.

@sjau, you can edit the View entire article in rss_update.php

[rsuplido]

Here's a workaround that I did to parse VB RSS2 streams (item titles with CDATA):

Open <forum dir>/includes/RSS/class.RSS.php (please backup original first)

Look for:

$channelCount = preg_match_all("|<channel>(.*)</channel>|iUs",$data,$channels,PREG_SET_ORDER);


Replace with:

$channelCount = preg_match_all("|<channel*>(.*)</channel>|iUs",$data,$channels,PREG_SET_ORDER);


Look for :

if(preg_match_all("|<title>(.+)</title>|iUs",$itemData,$match,PREG_SET_ORDER))
{
$title = $match[0][1];
$this->CHANNELS[$channelID]['ITEMS'][$itemID]['TITLE'] = "$title";
} else {
$this->CHANNELS[$channelID]['ITEMS'][$itemID]['TITLE'] = "";
}


Replace with:

if(preg_match_all("|<title><\!\[CDATA\[(.+)\]\]></title>|iUs",$itemData,$match,PREG_SET_ORDER))
{
$title = $match[0][1];
$this->CHANNELS[$channelID]['ITEMS'][$itemID]['TITLE'] = "$title";
} else {
if(preg_match_all("|<title>(.+)</title>|iUs",$itemData,$match,PREG_SET_ORDER))
{
$title = $match[0][1];
$this->CHANNELS[$channelID]['ITEMS'][$itemID]['TITLE'] = "$title";
} else {
$this->CHANNELS[$channelID]['ITEMS'][$itemID]['TITLE'] = "";
}
}


Look for:

$itemCount = preg_match_all("|<item>(.*)</item>|iUs",$channelData,$items,PREG_SET_ORDER);


Replace with:

$itemCount = preg_match_all("|<item*>(.*)</item>|iUs",$channelData,$items,PREG_SET_ORDER);


I did the these changes to parse another VB3 enabled site and it's been working for several weeks now.

Regards.

[mrboz]

it seems to be only "BBC News" feeds that are not updating. i'm using a reuters feed now and it's working perfectly

[jribz]

Hello,

First I'd like to thank everyone invoilved in this hack, very nice work.

Secondly I have a question. I am using the hack flawlessly to post from a custom rss feed from my own site which delivers the entire article, not just a link to it.

The problem is the bot delivers the article in one large block of text without formatting it onto paragraphs and line breaks. When the rss feed is viewed in a reader such as Sharpreader it formats fine.

Is there any way to get this to format in a more user friendly fashion for those us of who actually have live rss feeds with the entire story inside them?

If someone is willing to take a look I will PM the feed url to you as well as my site url.

Thanks for your time.

[Slapyo]

you can PM me the feed if you like or you can just post it on here. i'm sure anyone that can help will.

[jribz]

OK, thanks for the reply.

Rather than get into a PM exchange, I'll just post the info here since it may help others as well. :nervous:

The site I run is Xbox Solution located at http://www.xboxsolution.com

We have two rss feeds, one that will post a link and another that will write out the story.

First RSS Feed (Link Only): http://www.xboxsolution.com/backend.php

Second RSS Feed (Full Story):http://www.xboxsolution.com/index.rss

I am using the second RSS feed for the bot - and all is well except that when the bot posts the story it posts everything in one block, no line breaks.

If you point an rss reader such as Sharpreader to http://www.xboxsolution.com/index.rss you will see that the output is fine, with line breaks where needed,

Just wondering if there is any way to resolve this.

The forum that my bot posts in is here: http://talk.xboxsolution.com/forumdisplay.php?f=48

The bot's user ID is here: http://talk.xboxsolution.com/member.php?userid=28095

Thanks for the help!

[Slapyo]

here is where i ran into the same problem when coding this. i couldn't find out how to insert a breakline. everytime i put in /r/n it would actually put those characters into it. this was for inserting the description then 2 lines down putting the link. i had to come up with a work around for that.


Open up your replacement variable manager (in admincp -> styles & templates -> replacement variable manager) and add the following entry:
Style: [select whichever style you are using]
Search for text: {br}
Replace with text: <br />

[DS MrSinister]

looks like a kewl hack... i will giive it a try!!

[jribz]

here is where i ran into the same problem when coding this. i couldn't find out how to insert a breakline. everytime i put in /r/n it would actually put those characters into it. this was for inserting the description then 2 lines down putting the link. i had to come up with a work around for that.


Open up your replacement variable manager (in admincp -> styles & templates -> replacement variable manager) and add the following entry:
Style: [select whichever style you are using]
Search for text: {br}
Replace with text: <br />
I already have that replacement variable active, but it seems it only serves the purpose of putting the two lines before the linked url.

[Slapyo]

yup ... that is because i couldn't figure out how to make a line break without doing it that way. if you do /r/n/ or just /n it will create a newline, but when i did that it never created a new line it just put /n in the text. what i did was a work around.

i'm sure there is a way to simulate newlines in a text box within a script, i just don't know how to do it.

[Slapyo]

<a href="https://vborg.vbsupport.ru/showthread.php?t=67078" target="_blank">https://vborg.vbsupport.ru/showthread.php?t=67078</a>

you guys might want to check out that thread. it is called Article Bot and it does the same thing as this script, however there is more control built into it. it also allows for different kinds of feeds, RSS .91, RSS 2.0, ATOM.

i don't know how posts are inserted into the database but you guys may wanna check that out for other kinds of feeds.

[bradsears]

Is it possible to combine the greatness of both of these systems? I'm all for competition but in this case it may be wasteful.

[Slapyo]

nothing to combine. it would be one or the other. the Article Bot contains everything my script does, PLUS some things that have been requested for me to add in.

Article Bot is mine +more

[Sanjiyan]

Can this hack support RDF:Syndicate feeds? I have tried this hack and other vb3 hacks for rss feeds and it appears not or I am doing something wrong.

Any ideas on when this option will be added as I would really like to use it, but if it does not support RDF feeds (.asp files) eg http://myeve.eve-online.com/feed/RDFnewsFromDevBlog.asp
then I will not beable to install it sorry.

[Slapyo]

have you tried this one sanjiyan??

https://vborg.vbsupport.ru/showthread.php?t=67078

if you haven't tried that one, i suggest you try it. it offers support for more than one kind of feed.

[Sanjiyan]

have you tried this one sanjiyan??

https://vborg.vbsupport.ru/showthread.php?t=67078

if you haven't tried that one, i suggest you try it. it offers support for more than one kind of feed.

Yep I have tried it and thrown it back out, it does not work with the system I am after, but as you stated in your posts, your version (hopefully) plans to support it when you have the time to get it working, so I hope to wait till your version of the hack is ready to work with the syndicate I need to use.

[Sanjiyan]

Any ideas on when a newer version that might support the rdf feeds?

[turbidblue]

i got the hack installd and everything is working great, but i have a question about my bot that makes the news threads...

how do i get it to not show rep,age,postcount, and everything else???

i just want it to be the avatar and the name of the bot

any help??

i was told i needed to edit my postbit to identify that userid and to make it simple...

but how??

[RS25com]

I have this installed. It worked great the first day. Now it does not work.

I have three bots, using three different user ID's. The ID's are NUMBERS not names, right? So why is this not working?

it's over at www.sell2.us/forums in Off Topic and RSS Classifieds. :(

[Slapyo]

i got the hack installd and everything is working great, but i have a question about my bot that makes the news threads...

how do i get it to not show rep,age,postcount, and everything else???

i just want it to be the avatar and the name of the bot

any help??


i was told i needed to edit my postbit to identify that userid and to make it simple...

but how??
you should ask over on http://www.vbulletintemplates.com/ ... they can help you with that.

[Slapyo]

I have this installed. It worked great the first day. Now it does not work.

I have three bots, using three different user ID's. The ID's are NUMBERS not names, right? So why is this not working?

it's over at www.sell2.us/forums in Off Topic and RSS Classifieds. :(
post the URLs to the news feeds please.

[Slapyo]

Any ideas on when a newer version that might support the rdf feeds?
not sure when, i'm pretty busy with work ... i work on it in bits here and there.

[Sanjiyan]

not sure when, i'm pretty busy with work ... i work on it in bits here and there.

I do hope it is soon, this is the one hack I am really really waiting for, and I rush to this thread everytime theres an update to the script, because I am in desperate need for this hack ASAP.

[Rob Eh]

Is there a way to specify the time zone used in the control panel for this?

It uses the server time and not the time set by vB in the control panel.

Thanks,

[Mickey19781978]

I have installed this hack and got no errors running this or installing it

I installed version 1.26

My php version is 4.3.6.6

Forum ID 24
User ID 1 (which should be me once i get this to work iw ill create a user for it)
Source = http://www.sportnetwork.net/rss.php?sport=football

Nothing happens... install went smooth and all installed successfully
Scheduled task manager is set to 1 hour on Newsfeeds with filename ./rss.php

But when i hit run now
I still get nothing

Maybe i am using wrong sources?
If so could someone include a few addresses with rugby,football and UK news?

Cheers,

Mick

[Mickey19781978]

turned out i had a small failure in the functions_newpost.php

fixed it and is working now =)

awesome job on this m8ey

[Slapyo]

Is there a way to specify the time zone used in the control panel for this?

It uses the server time and not the time set by vB in the control panel.

Thanks,
hrm, i am not sure on how to do that. i will look into it, but it has to do with the insertion of the post ... i think it just uses now() to grab the date which would be server time. I will have to jump into vB code to see how they do it.

[CharlieBrownDE]

nice hack. installed and no problems 3.0.3 :) thx

[ntventures]

I installed the hack and get the following error:

Fatal error: Call to undefined function: log_cron_action() in /home/atech/www.discusssite.com/docs/rss.php on line 7

I'm running 3.0.3. It installed okay and added new threads when it installed, but hasn't added any more since. How do I fix it?

Thanks

[Slapyo]

hrm that's odd because it has all the files included that it needs above that for that function. you can remove the log_cron_action() line if you want. it's not really all that important.

[turbidblue]

edit: nevermind :)

problem solved :)

[Natch]

OK Slapyo - I need a hand - can you tell me why includex/functions_newpost.php just started giving me this error?## Invalid SQL:

UPDATE forum
SET replycount = replycount + 1,
threadcount = threadcount + 1,
lastpost = 1091567991,
lastposter = 'News',
lastthread = 'After the Accident (Forbes.com)',
lastthreadid = 3901,
lasticonid = 0
WHERE forumid =

## mysql error: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 9Whenever the RSS job is run ? it's somehow lost access to the $foruminfo[forumid] value ...

[cogimo]

Hi All

I found a great to get RSS feeds

http://www.prweb.com/rss.php

[pgowder]

I have 3 feeds setup. When I run the cron task it goes through 2 of them and then I get this error:

Warning: file(): php_network_getaddresses: getaddrinfo failed: Name or service not known in /home/virtual/site2/fst/var/www/html/gathering/includes/RSS/rss_update.php on line 39

Warning: file(http://www.nativenationsnet.net/backend.php): failed to open stream: Resource temporarily unavailable in /home/virtual/site2/fst/var/www/html/gathering/includes/RSS/rss_update.php on line 39

Warning: implode(): Bad arguments. in /home/virtual/site2/fst/var/www/html/gathering/includes/RSS/rss_update.php on line 39

[M1th]

Err... is this hack secure?

RSS.php file contains nothing but


<?
require_once('./global.php');
require_once('./includes/functions_bigthree.php');
require_once('./includes/functions_forumlist.php');
require_once('./includes/functions_newpost.php');
require_once('./includes/RSS/rss_update.php');
log_cron_action('RSS News Feed Posted', $nextitem);
?>

am i right in assuming that no permission check is necassary?

[croft]

Ok. My feeds wont update automaticly at all.
They did it the first time now nothing.
Im using rss version 0,91.
I cant even update the feeds by running the cron manual.

[M1th]

Ok. My feeds wont update automaticly at all.
They did it the first time now nothing.
Im using rss version 0,91.
I cant even update the feeds by running the cron manual.

same here.

[Slapyo]

i have been really busy with my new job and haven't been able to work on this at all.

i do suggest that you check out article bot (https://vborg.vbsupport.ru/showthread.php?t=67078). it is the same as this but offers alot more features for the news.

[cogimo]

Ok. My feeds wont update automaticly at all.
They did it the first time now nothing.
Im using rss version 0,91.
I cant even update the feeds by running the cron manual.

Has anyone been able to get this to work, I'm having the same problem

[croft]

Has anyone been able to get this to work, I'm having the same problem

I couldnt get this to work so i installed another hack instead.
https://vborg.vbsupport.ru/showthread.php?t=67078

That hack i could get to work. =) Try tha one if you cant get this to work.

[Slapyo]

yup yup. that hack can parse alot more than this one can. it also includes many features that were suggested to me to add in. i would suggest people use that article bot.

[Holidazed]

Is there a news feed out there that is just "Weather" news?

Ideas?

[Slapyo]

check http://www.syndic8.com

[Holidazed]

I am looking at it but I fail to see any list of streams that might fit my needs.

Ideas?

[Holidazed]

NM, I found it. Thanks

[Holidazed]

I installed this the other day and it pulled in 7 messages from the feed which is very cool. However, I have got nothing since then. I though I messed up the cron. I ran the script manually and it comes back with the message "RSS News Feed".

Is that all it is supposed to say?

I added 3 more feeds going to the same forum and I get no posts from any of them.

What am I doing wrong?

BTW, I have attached a picture of my cron screen. Did I do it right?

[PTO]

Any way we can minipulate the incoming RSS feeds? I'm not a big fan of the advertisements splattered all throughout my RSS feed even though I love the content they are giving me.

[Holidazed]

Help

[Slapyo]

If you were to go through and pull out advertisements you would have to edit the code in which case it would be for that specific feed only and would work for only that feed.

bitg, i'm not exactly sure. what rss feeds are you using. i would suggest that you goto the article bot which provides many more options but does the same thing.

https://vborg.vbsupport.ru/showthread.php?t=67078

[mobilephonegame]

Hi All,
ive read through this thread, but am no nearer finding a solution

my php is 4.3+ ive installed without any errors showing

ive added a bot, and scheduled for every 59 mins, allocated forum id & News user id

all goes fine until i manually select "run now" in sshedule tasks

all im shown is:

Warning: main(class.RSS.php): failed to open stream: No such file or directory in /home/public_html/gamesarcade/includes/RSS/rss_update.php on line 17

and

Fatal error: main(): Failed opening required 'class.RSS.php' (include_path='') in /home/public_html/gamesarcade/includes/RSS/rss_update.php on line 17

no other errors appear in the admin area, ive even done the fix for users using a lower spec of php and still doesnt work...

can anyone throw some light>?

the RSS im wanting to use, and have set up is:
http://news.bbc.co.uk/rss/newsonline_uk_edition/world/rss091.xml

any assist with this to get it working would be gr8 :-)

[Beermonster]

Works perfect :)

* Beermonster clicks install

[mobilephonegame]

Works perfect :)

* Beermonster clicks install

<< couldnt get it to work, so used another, and that worked first time :tired: :-)

[Beermonster]

I installed this the other day and it pulled in 7 messages from the feed which is very cool. However, I have got nothing since then. I though I messed up the cron. I ran the script manually and it comes back with the message "RSS News Feed".

Is that all it is supposed to say?

I added 3 more feeds going to the same forum and I get no posts from any of them.

What am I doing wrong?

BTW, I have attached a picture of my cron screen. Did I do it right?

First of all you have set your cron to check every 0 minutes try setting it to 59

Also I'm the same as you some feeds work and some don't, I'm just getting no posts from them, anyone else have a problem with it?

[martinh4]

Installed and it's working great. :)

[venomx]

Ok I run VB3.0.3 and tried Article Bot and it works fine except on the forum home it doesnt updat the Last Post field. Does this hack have the same problem?

[Slapyo]

it might be how article bot inserts the post. article bot may insert the post manually...but this bot will use vB's function to make a new post. however i would still go with article bot because it offers many more features than this one. make the suggestion to the article bot dev.

[Blam Forumz]

My scheduled task is called rss.php it has everything that it should have,the filename is ./rss.php

and when i run it, all i get is a message saying:

rss.php

and no post in the forum i setup the bots to post in... help, i use 3.0.2 and i dotn know how to check my php version

[DanGarion]

If I try to run the scheduled task I all I get is the name of my Scheduled task and nothing that says "Done".

If I try to bring up the rss.php file it's just a blank page.

I don't see any errors generated anywhere.

http://uo.stratics.com/php-bin/rss/raw.php is the feed.

[Slapyo]

dan, that is a rdf feed not a rss feed. try article bot, it is located in the integration forum. it does rdf i think.

[blueuniverse]

Is there any way, to make it accept 287 character, RSS addresses in the source box. Currently it is limiting me to 148. Don't ask about the long feed, it is one of those super configurable ones.

How would I do this?

[Slapyo]

i'm not sure. i only made changes to the script and still don't know all the ins and outs about it. i do know that article bot is alot more advanced in what it does as compared to this bot.

[blueuniverse]

Is there any way, to make it accept 287 character, RSS addresses in the source box. Currently it is limiting me to 148. Don't ask about the long feed, it is one of those super configurable ones.

How would I do this?

Ignore this now, I sorted it by editing the field in phpMyAdmin

[Slapyo]

oh ok cool ... ;) glad you got it working the way you want.

[scoobycray]

Iv'e installed but do not get anything happening....

If I run the scheduled task manually its just says rss.php and nohing else.

running php 4.3.4 and setup the forum ID and user all OK trying to use RSS Feed URL: http://news.bbc.co.uk/rss/newsonline_uk_edition/world/rss091.xml

Any ideas? Would love to get this running....

[scoobycray]

DOn't worry fixed it - for anyone else just check you are putting the user ID NUMBER and not the Users name when you add a new bot :)

[scoobycray]

I had the same issue with BBC where it grabs the first set of headlines when you first run it but will bot update, no idea why though.

[ToddW]

MoreOver Feeds aren't working for me... Any idea why?

TheRegister feed wroked then I moved to a new sub-forum and they didn't update in here when I ran the cron..?

[58sniper]

One minor suggestion....

adv_portal_news_archivebits ends up spitting out a line like this:

<b><a href="http://dev.defensivetactics.com/forum/showthread.php?t=51" title="<br /><br />View the Entire Article<br />">Development: AL QAEDA SEEN PLANNING FOR 'SPECTACULAR ATTACK'</a></b>

I'm concerned about the breaks in the title parameter. That's causing some validation problems.

In rss_update.php, I found this:
$rss_description .= "{br}{br}View the Entire Article ( . $rss_allItems[$j][LINK] . ){br}";

If I pull out the three breaks, is that going to cause problems elsewhere?

[ToddW]

I`m still waiting for help with MoreOver's feedt rhat uses spaces.. %20 .. anyone ?

[msimonds]

This is a great mod, installed and working great, I do have a question though.

If I add over 30 rss feeds (individual team forums), i run the cron job and a few times I have received Time out errors. Is there a max number of feeds we can use!! ALSO

WHat is the proper setting to uptdate this once every 24 hours or once ever 12 hours

Thanks
Mike

[msimonds]

when I run the cron job, this is the error that I am getting

Fatal error: Maximum execution time of 30 seconds exceeded in /home/rantman/public_html/forums/includes/RSS/rss_update.php on line 62

Is there a way to increase the time!!

[58sniper]

Okay, I seem to be having an issue with this. I got it installed without a problem. I added a feed, added it as a cron job, and then manually ran the cron job. It promptly added 10 posts without a problem. And then it never added anything again, even when cron ran the script. If I manually run the script - same problem.

Now - if I delete the news and manually run it again, I get 10 new posts without a problem. But nothing after that.

Anyone else run into that?

[Slapyo]

is their a date field? if there isn't then it can't check to see if the date is the same ... so it won't add any new posts. post your feed and i can look at it.

[58sniper]

That looks to be the problem.
http://www.defensereview.com/backend.php is my feed.
I'll test with some other feeds and let ya know. Thanks for the clue.

[ToddW]

Did anyone figure out how to get moreover to work with the spaces it uses? None of my moreover feeds work..

[Slapyo]

post the feed toddw ... that way i can look at it.

[ToddW]

Here's just one of the hundreds they offer.
All use the %20 - and none work with this RSS News Feed Hack :disappointed:


http://p.moreover.com/cgi-local/page?c=Human%20rights%20news&o=rss

[nelto]

Okay, I seem to be having an issue with this. I got it installed without a problem. I added a feed, added it as a cron job, and then manually ran the cron job. It promptly added 10 posts without a problem. And then it never added anything again, even when cron ran the script. If I manually run the script - same problem.

Now - if I delete the news and manually run it again, I get 10 new posts without a problem. But nothing after that.

Anyone else run into that?
Yes me, Im having the same problem.
Any help here would be appreciated.

[58sniper]

Yes me, Im having the same problem.
Any help here would be appreciated.

The suggestion by Slapyo appears to have resolved my problem. Your best bet is to view the actual feed in your browser and see if there is a pubDate field.

Examples:

Broken: http://www.defensereview.com/backend.php
Correct: http://news.com.com/2547-1_3-0-5.xml

View each, and you'll notice that the correct feed has a pubDate field for each item. The first one was the one I originally used for testing, and it didn't work. I changed to the other, and it's been working fine since.

[Scutter]

I'm thinking about using this hack but would like it to do 2 different things, first is what was intended as a rss news feed, secondly there is an XML feed from an online game which I play, the tags in this XML file are different than a news feed (names). Would it be quite simple to alter the sript to allow this? For example I was thinking that I could insert a simple if statement in the template that makes the posts to say something like (in simple terms so you get what I'm thinking)
If bot = this bot then
post title = some tag
post description = some other tag

I presume it is possible, but is it just a couple of template changes or will I need to alter the files of this hack to do it?

[Slapyo]

scutter, no template changes ... it would all be within the script.

[Slapyo]

Here's just one of the hundreds they offer.
All use the %20 - and none work with this RSS News Feed Hack :disappointed:


http://p.moreover.com/cgi-local/page?c=Human%20rights%20news&o=rss
toddw, the %20 corresponds to a space. try removing the %20 and replacing it with a space. however you will have a problem since there is no pubDate field in the news feed. the pubDate field is required because of the need to check for new news.

http://p.moreover.com/cgi-local/page?c=Human rights news&o=rss

[ToddW]

How come MoreOver doesn't use a PUB Date Field? Is there a way around this for MoreOver Feeds?

edit- Looks like since they 'refresh' every 15 minutes they don't find the need for a pubdate field ??? :ermm:

[58sniper]

How come MoreOver doesn't use a PUB Date Field? Is there a way around this for MoreOver Feeds?

edit- Looks like since they 'refresh' every 15 minutes they don't find the need for a pubdate field ??? :ermm:

Because their feeds are RSS 0.91 feeds, and you typically don't see pubDate in 0.91 feeds.

Nothing you can do about it other than complain to MoreOver.

[rjerina]

i installed this, it did the first feed read, but it seems to not be finding new entries, or just not adding new entries?

using the feed http://rss.pcworld.com/rss/latestnews.rss

the most recent article should be "New Cool, Quiet Laptop Hard Drives", but the most recent is "Is XP's Fix Safe?"

not sure what is going on.

[58sniper]

i installed this, it did the first feed read, but it seems to not be finding new entries, or just not adding new entries?

using the feed http://rss.pcworld.com/rss/latestnews.rss

the most recent article should be "New Cool, Quiet Laptop Hard Drives", but the most recent is "Is XP's Fix Safe?"

not sure what is going on.

They don't have a pubDate field in their feeds. See https://vborg.vbsupport.ru/showpost.php?p=557388&postcount=431

[rjerina]

They don't have a pubDate field in their feeds. See https://vborg.vbsupport.ru/showpost.php?p=557388&postcount=431

so i cant use this feed? thats a bummer :(

this sucks. i cant use the best pc computing news feed i have found thus far, because this thing needs pub dates? why doesnt it just check against titles or links. seems kind of lame and limited.

[58sniper]

so i cant use this feed? thats a bummer :(

this sucks. i cant use the best pc computing news feed i have found thus far, because this thing needs pub dates? why doesnt it just check against titles or links. seems kind of lame and limited.

If you read this entire thread, you'd know it's because without a pubDate field, the script doesn't know when it was released, and, thus, if it's new or not. Perhaps if PC Computing got off their @ss and came out with an RSS 2.0 feed, instead of using that 0.91 feed, the world would be a better place.

[rjerina]

<rss version="2.0">
-
<channel>
<title>PCWorld.com - Latest News Stories</title>
<link>http://www.pcworld.com</link>
<description>Latest news stories from PCWorld.com</description>

it is rss 2.0 they just decided not to do pubdate. maybe if this script didnt rely on times so much.

the script shouldnt be doing this by times anyways. what if two rss feeds come out at the exact same time? it ignores one or both? it should just compare the titles or urls.

[rjerina]

i may just scrap this thing and do this all in perl and just insert it right into the database, this is rediculous.

[58sniper]

It needs a timestamp so that it can tell if it was released since the last time it checked. Sure, the script could query the database and see if it's already there, but why introduce more overhead? It really frosts me when RSS feeds DON'T contain that data. It doesn't really cost more to have a more complete feed. And RSS clients can do so much more with the extra data.

[Slapyo]

it relies on times and titles. because yahoo will release the same article 40 minutes later and obviously you don't want 2 of the same post. but it needs the time because a compliant rss feed would have pubDate in it. try the article bot in the vBulletin Integration forum. i am not sure exactly how that bot works.

[rex_b]

when I run the task nothing happens.. just says

RSS Bot

[Slapyo]

that's all it should say. there is no output in the scheduled task manager besides that. threads should be created in your forum.

[Sooner95]

ah ha,, I am guessing that the other RRS news feed thread is dead and this one has the updates? lol

got me confused, I installed the other one..i should be ok tho, correct? can i reinstall this one over it and continue? thx

[Sooner95]

Fatal error: Call to undefined function: build_new_post() in /home/bulldog/public_html/forum/includes/RSS/rss_update.php on line 110

Ok, i get this... what am I missing?

[Sooner95]

nevermind, I used the OLD rss_update.php file and it works LOL

[Slapyo]

this rss news feed is a continuation of the other one in this forum.

[Sooner95]

figured that out,, hehe Great hack BTw!

had a quick question..tho might have been asked in the 30 pages of this thread lol

Anyways to keep the rss posts from appearing in the new posts search/count ?

[Sooner95]

nevermind, found it

https://vborg.vbsupport.ru/showthread.php?t=63798&page=16&pp=15&highlight=post+count

[kctalker]

I am getting duplicate posts, only one has the article, the other is blank. I am using the current version just installed tonight.

Is there any fix for this or what other information do you need from me? Thanks!

[kctalker]

Actually, it is happening to all my posts since installing this hack. Please help!

My site is http://www.kctalk.com and everything is totally jacked up now. How do I remove the changes from the db?

[Slapyo]

just follow the instructions and do them in reverse and the opposite of what they say.

[abaum]

So that rss 2.0 functions, I took over the code of rsuplido -> https://vborg.vbsupport.ru/showpost.php?p=532303&postcount=354. However thereby <description> is not taken over. Mine rss looks in such a way:

<item>
<title>
<![CDATA[ Souleage Site]]>
</title>
<link>http://www.domain.com/index.php?id=34</link>
<description>
<img src="http://cgnetworks.com/newsimages/2004_10/2004_10_26_soul70.jpg" align="left" class="left" hspace="2" vspace="0" width="70" height="70" />
<![CDATA[ Text... ]]>
</description>
<pubDate>Tue, 26 Oct 2004 07:38:00 +0000</pubDate>
</item>


Can help their me?

Excuses please my bad English!

Thank you Andreas

[Slapyo]

just looking at the code briefly it looks like rsuplido's code only affects the title.

so, take a look at the section of code for the description and it might be a copy/paste replacement then you just change the <title> to <description> and it might work. i'm not sure, but it might work.

[abaum]

if(preg_match_all("|<description><img (.+) /><\!\[CDATA\[(.+)\]\]></description>|iUs",$itemData,$match,PREG_SET_ORDER))
{
$desc = $match[0][1];
$this->CHANNELS[$channelID]['ITEMS'][$itemID]['DESCRIPTION'] = "$desc";
} else {
if(preg_match_all("|<description>(.+)</description>|iUs",$itemData,$match,PREG_SET_ORDER))
{
$desc = $match[0][1];
$this->CHANNELS[$channelID]['ITEMS'][$itemID]['DESCRIPTION'] = "$desc";
} else {
$this->CHANNELS[$channelID]['ITEMS'][$itemID]['DESCRIPTION'] = "";
}
}


Functioned unfortunately also not. Only the title is picked out. The variable $rss_description remains empty.

Andreas

[WizardX]

Is this all compatible with new 5.0.1 release? Cause I get the following error:

Warning: array_merge() [function.array-merge]: Argument #2 is not an array in /usr/sites/isptalk.nl/web/includes/init.php on line 517

Warning: Invalid argument supplied for foreach() in /usr/sites/isptalk.nl/web/includes/sessions.php on line 270




Unable to add cookies, header already sent.
File: /usr/sites/isptalk.nl/web/includes/init.php
Line: 517





http://www.isptalk.nl/rss_install.php

Thanks in advance.

[deepdark]

PHP Version 4.3.9
Scheduled Task Manager :DONE
RSS News Feed Manager:BOT DONE

but no posts do i need to make any other change i mean where i need to define the RSS adresess?


any help please???

[wattieuk]

ive installed the mod and the first scheduled rss post works fine, but it doesn't post subsequent messages, despite the feeds being updated. Anyone else got this problem?

[wattieuk]

ive installed the mod and the first scheduled rss post works fine, but it doesn't post subsequent messages, despite the feeds being updated. Anyone else got this problem?
Weird. I found this post (https://vborg.vbsupport.ru/showpost.php?p=500356&postcount=59) and uploaded the attached file. Works fine now!

[Floris]

Any way I can limit this to just x entries - so I don't get 25 new threads each time to spam up the getdaily or getnew search?

[dorkus]

With this script, if it doesn't have the pubdate field, it won't work.

I'm trying to get the slashdot feed to work, and it worked the initial time, but it won't post anymore due to their feeds not having pubdate fields.

[dorkus]

For those interested in the slashdot feed, and are using this hack, I found an address that converts the slashdot feed to rss 2.0 and uses the pubdate field.

http://my.rsscache.com/express.xstreamsoft.com/dyn/rss/dcc98699-350a-43b6-b33b-ddd4c11cebb2-en.aspx

[bishop93]

Here I thought this one would work when the other RSS mod didn't since this one seems actually promising with all the glowing reviews. But, I get this return from the cron job:

Warning: main(./global.php): failed to open stream: No such file or directory in /home/xxx/public_html/xxx/rss.php on line 2

Fatal error: main(): Failed opening required './global.php' (include_path='.:/usr/lib/php:/usr/local/lib/php') in /home/xxx/public_html/xxx/rss.php on line 2

Thoughts?

Thanks in advance.

[bishop93]

Nevermind. One of our devs got this to work. I have no idea how. But it seemed to be something about the cron setup wrong. It is, as I said, a most excellent mod.

[bishop93]

Okay. I was wrong. Sorta. It's read and inserted once. I know that the feed has been updated once since then, but it's not updating the forum.

Why?

[bishop93]

Does anyone have answers to these questions? Such functionality is brilliant option to my operations, at least, but I can see why these would be necessary to others.

[mikehawk]

Hey guys, I'm a hardcore noob:

rss.php

Warning: main(./includes/cron/rss.php): failed to open stream: No such file or directory in /home/.gambia/realopin/realopinion.com/realboards/admincp/cronadmin.php on line 56

Fatal error: main(): Failed opening required './includes/cron/rss.php' (include_path='.:/usr/local/lib/php') in /home/.gambia/realopin/realopinion.com/realboards/admincp/cronadmin.php on line 56

What do I do? This happened when I ran the task.

[mikehawk]

Ok I changed the path in the task manager to rss.php in my main directory.. it wasn't in cron.

But now I get this error:

rss.php

Parse error: parse error, unexpected ':' in /home/.gambia/realopin/realopinion.com/realboards/includes/functions_newpost.php on line 190

What could be wrong? Please help. :nervous:

[mikehawk]

Well its posts the updates now.. i hope it doesn't stop posting like some of the members on this board have experienced... :ermm: Its awesome though, great hack!

[mikehawk]

Well its posts the updates now.. i hope it doesn't stop posting like some of the members on this board have experienced... :ermm: Its awesome though, great hack!


Ok well my problem is this:

It updates when I manually run the task with one user, but it will only update again if I change the bot id to another user. It doesn't update again when the bot id is the same. What could be the problem? I have to use another users id everytime I want to update the feed.

[CtrlAltDel]

yeah im having the same problem

thing ran the initial time, and made the threads, now every subsiquent run of the script doesnt generate squat

quality...

[CtrlAltDel]

ah pubdate...well this sucks
time to try and figure out a way around this one

[wattieuk]

I had the 'not updating' problem but if you look back one page to this message:
https://vborg.vbsupport.ru/showpost.php?p=573641&postcount=462
I say how it was sorted out - works fine now.

[jshafer817]

I had this hack installed but I uninstalled it hoping that another hack could pull off what I wanted to do.
I am trying to read an rss feed thats been encoded in utf-8
Is this possible for me to do???

Thanks for your time! I could have never gotten this far without all of yaulls help.

[H2k]

You'll have to ask your host to upgrade to a more recent version of PHP.

Or if you follow the link to the php function I posted before, it shows this code:

// For users prior to PHP 4.3.0 you may do this:
function unhtmlentities($string)
{
$trans_tbl = get_html_translation_table(HTML_ENTITIES);
$trans_tbl = array_flip($trans_tbl);
return strtr($string, $trans_tbl);
}

$c = unhtmlentities($a);

echo $c; // I'll "walk" the <b>dog</b> now

?>

If upgrading PHP is not possible for you then you could try to replace the html_entity_decode function using this example.


By the way, has anyone managed to fix the double post issues? I have these issues with both yahoo and bbc feeds.
Why when i run it some time i get this error

Fatal error: Maximum execution time of 30 seconds exceeded in /home/myuser/public_html/includes/functions_databuild.php on line 480

[Neutral Singh]

Gr8 Hack !! Just installed version 1.26 ... I have some issues...

1. When I first ran the cron job manually to test and refreshed the forumpage. it automatically logged me in as that bot member whose id i had used to add the new bot.

2. Then it did not show up the forums at all... i mean on going to index page it only showed headings --> Forums - Last Post - Threads - posts, but no forum was showing. I had to delete the bot all together to show the forums on index page.

3. Strangly, in all this hick ups, the rss feed was succesfully entered as new threads but with one more issue... the following is what is showing up... The thing in bold below is spoiling the show.

Daily Times,&nbsp;Pakistan&nbsp;- 8 hours ago... sculptures were prominent in reminding the people of their colonial past; the sculpture of Alfred Woolner, the vice chancellor of the Punjab University (1928 ...

I wonder what is the problem...

Please guide

Regards

[fpouk]

I was trying to install this mod for the RSS Feed for my forums..
I was running the install script that came with it... then it got to where it was adding data into my thread table and it just hung up...
I was like okay no problem... i will just go to phpmyadmin and do a repair on my thread table.. Well when i opened up phpmyadmin and went into my forums DB i notice that my thread table say's 0 Rows now.. when i click on the table it doesnt show anything.. it's just blank?

Well i tried to repair the table anyways but it just times out or something. It will get to about the 6th bar on my browser and just quit.


any help will be appreciated...


oh yeah... i cant access my forums or admin cp now because i get a internal server error

it's been giving me this internal server error ever since that install script crashed on me.

http://www.secfanatics.com

[egthareal]

Database error in vBulletin 3.0.6:

Invalid SQL: ALTER TABLE rss_settings DROP forumid, DROP userid, DROP source
mysql error: Can't DROP 'forumid'. Check that column/key exists

mysql error number: 1091

Date: Tuesday 08th of March 2005 03:58:37 AM
Script: http://www.tips4poker.com/forums/rss_upgrade.php
Referer:
Username: egthareal
IP Address: xxx


can someone please assist me in this upgrade? thanks

[inverted]

can i see anyone's live sites using this hack? just wanna get a better feel for how things get displayed and whatnot. thanks.

- inverted

[RackChat]

I installed it on my boards(www.rackchat.com) about an hour ago. I tried to limit the number of articles to be posted by adding variables in the URL but when I run the task it doesn't add anything to the forum, yet I don't get any errors. When I remove those variables everything works. I haven't read through all 33 pages so it might have been suggested before. Anyway adding another option to limit the quantity of threads it starts or a way to limit the articles it fetches would be a great feature.

Great hack though, well done!

[subnet_rx]

I installed the hack and get the following error:

Fatal error: Call to undefined function: log_cron_action() in /home/atech/www.discusssite.com/docs/rss.php on line 7

I'm running 3.0.3. It installed okay and added new threads when it installed, but hasn't added any more since. How do I fix it?

Thanks

I'm getting this same error on 1.26, might be something to look into.

[venomx]

I tested the default one that comes with VB. The validater gave one error. I was going to install this one but from what I read it also has problems?

[wacodep]

Okay, why does THIS feed work fine using the v1.26 hack:
http://rss.news.yahoo.com/rss/topstories

But this one does NOT?
http://news.search.yahoo.com/news/r...TF-8&fl=0&x=wrt

[cashpath]

Is this all compatible with new 5.0.1 release? Cause I get the following error:

Warning: array_merge() [function.array-merge]: Argument #2 is not an array in /usr/sites/isptalk.nl/web/includes/init.php on line 517

Warning: Invalid argument supplied for foreach() in /usr/sites/isptalk.nl/web/includes/sessions.php on line 270




Unable to add cookies, header already sent.
File: /usr/sites/isptalk.nl/web/includes/init.php
Line: 517





http://www.isptalk.nl/rss_install.php

Thanks in advance.

Does anyone know how to fix this to work with php 5?

[Blootix]

No screenshots? Anybody has a live demo?

[cashpath]

To answer my own stupid question if you have PHP 5 add this to both RSS files...

$phrasegroups = array();

// get special data templates from the datastore
$specialtemplates = array();

// pre-cache templates used by all actions
$globaltemplates = array();

// pre-cache templates used by specific actions
$actiontemplates = array();

Of course I did that and the feeds still don't produce anything.. but at least I don't get an error

[cashpath]

Well unfortunately I couldn't get this to work, so... guess I won't use it..

[kevinrstruck]

Works like a charm for me. No issues at all. Installed in about 5-10 minutes.


So far so good.

http://www.thephotostation.net/forums/forumdisplay.php?f=91
http://www.thephotostation.net/forums/forumdisplay.php?f=90
http://www.thephotostation.net/forums/forumdisplay.php?f=89

[Zachariah]

thanks Slapyo, ill install soon.

[kevinrstruck]

Works like a charm for me. No issues at all. Installed in about 5-10 minutes.


So far so good.

http://www.thephotostation.net/forums/forumdisplay.php?f=91
http://www.thephotostation.net/forums/forumdisplay.php?f=90
http://www.thephotostation.net/forums/forumdisplay.php?f=89


It is working perfectly now.

[opsman]

I have found my problem with this error

Warning: file(): URL file-access is disabled in the server configuration in /includes/RSS/rss_update.php on line 39

Warning: file(http://news.yahoo.com/rss/): failed to open stream: no suitable wrapper could be found in /includes/RSS/rss_update.php on line 39

Warning: implode(): Bad arguments. in /includes/RSS/rss_update.php on line 39


How can I get by this any ideas
I tried all the official working news feeds but same answer everytime

[Zachariah]

my problems: 5 feeds no mater the order in the DB only the 1st feed posts.
Any one of the feeds work, but it will not post more than 1 in the bot list.

Thoughts ?

[RichieBoy67]

I have done this install a few times and never had a problem... this time I am getting the following


Warning: main(class.RSS.php): failed to open stream: No such file or directory in /home/www/ureside/forums/includes/RSS/rss_update.php on line 28

Fatal error: main(): Failed opening required 'class.RSS.php' (include_path='') in /home/www/ureside/forums/includes/RSS/rss_update.php on line 28


I have gone over my edits and do not see a problem... Is there something I have to edit in the update file???


I fixed it.... I had to go into the RSS/rss_updae.php and edit the path to the class.RSS.php file.... If somebody else has this problem I hope this helps :)

[Benj]

how would i set up the cron as a vbulletin sceduled task my host doesnt support cron job coz there crap :D

[libertate]

Excellent job. This fixed one of my problems. Thank you!

Some suggestions for version 1.27... :squareeyed:

* The management tasks[Add a Bot], [Modify Bots], [Delete Bots], [Delete All News] shouldn't it be a submenu item of RSS NewsFeed on the navigation frame, instead on a single page?

* When [Delete Bots] is selected, the name is not displayed only the ID. Can the name be added?

* A title for the feed would be nice. So instead of some obscure URL, a standard source would descibes it. i.e. "Medical News" instead of "http://blahblah.blah/~blah.rss"

* Would it be possible to display a list instead of individually managing the RSS feeds?
i.e.
1. [ Source(text)] [Title(text)] [User (text)] [Enabled (Radio)] [Delete (Radio)]
2. [ Source(text)] [Title(text)] [User (text)] [Enabled (Radio)] [Delete (Radio)]
It is very unlikely that someone whould have a 25+ feed... and if they do there is always the scroll bar... :nervous:

* Although I understand the convenience on deleting all fetched news, I don't think you need (should ) to have this option. The pruning of posts by user is already available.

* Can you add additional functionality such that are available in "fps_external (https://vborg.vbsupport.ru/showthread.php?t=69834) " are available? That would be a PERFECT combination!

Again, thank you for a job well done.

[RichieBoy67]

I do not want the bot listing in the stats.... I cannot remember how to do this at this hour... can anyone help please???

Is there a way to do this???? I do not want the bots showing up in post count or anywhere else.....

Thanks

[Slapyo]

sorry, i don't have time to support this hack anymore. however, i don't want to leave you guys hanging.

https://vborg.vbsupport.ru/showthread.php?t=67078

check out the Article Bot. it's like my script but with alot more options and as far as i know it is currently being supported.