Anyone having the problem of the last entry getting added again and again?

Yep, I'm having that problem but only with some RSS feeds that I pull into certain forums. I think it might be something to do with the website itself and the way they are presenting the information. For instance, if they update the date of a particular news heading, it's classed as new by the RSS hack script.

Slapyo - I know you built some form of checking into your script. Is this based on thread title, date or both. I would recommend it's changed to search for the title, so a search is done beforehand on existing forum threads. That way, dupilcation could be avoided. Maybe when you've finished your studies :).

well, I tried changing that line to just check for the title and still I'm getting duplicates whenever the script runs. I was going to put this rss feature on live at my site but this bug just hinders me from doing so. :(

did you make sure that you modified the files like noted in the installation file? i haven't noticed this problem with the script myself.

the rss script which goes through and checks for old posts checks for the title and the date. i haven't touched this from the original script. but i do know that it checks the title and the date.

Yep, I went through it all step by step. :(

Ok hmm, I removed all the database entries, reuploaded all the files and tried again
When I enable the RSS Feed, I get this error

Fatal error: Call to undefined function: build_new_post() in /home/deimos/public_html/forums/includes/RSS/rss_update.php on line 103

Any ideas? heh

hrm ... i'm not sure. i'm sure that you did edit the files and upload everything to where it needs to go. but the only thing i could say is to double and triple check it.

also, do you have any other hacks installed? a while back someone installed this on 2 instances of vB. one was clean and one was hacked. he had a problem with the hacked one that we were never able to solve, and it was installed properly.

I decided to modify the program instead. I added a 'rss_url' in the threads table so I can use that as the key for comparing. I then changed the loop code in rss_update.php to compare each record via an SQL call, comparing the rss_url field. I also wanted a url column so I can easily do a news listing on my main page and have the link ready.

It's all working now. Thanks for this hack.

I am getting duplicates again... very annoying.. it's adding and adding... is there a fix for this? If so, can somebody post the instructions?

-Andy

me 2 ^

what news feeds are you using? the requirement from the news feed is there has to be a field <pubDate> in it ... otherwise it won't grab the date and it won't be able to check for duplicates since it checks for the title & the date.

post your news feeds that you are using.

Slapyo: awesome hack many thanks :)

I have a couple of problems:

with the feed from:

http://www.masshibsteria.com/home/in..._rss&no_html=1

I get duplicates. Viz your comment above is it because the feed lacks a pub date?

Also from this feed:

http://sport.scotsman.com/football_h...cfm?format=rss

some of the stories have an apostrophe "Hibees' strike stars will be battling it out for Riordan's crown"

and these stories are repeating and the ' is being replaced by

"Hibees&#x27; strike stars will be battling it out for Riordan&#x27;s crown"

Any ideas? :)

I should add, I've added 3 other newsfeeds without any problem :)

I am using: http://rss.news.yahoo.com/rss/celebrity

And it does have a <pubDate> in it. It's grabbing the same top story over and over again.

-Andy

ok bobby, here is what i can see.

your first feed has no pubDate field so it is not able to check if it repeats because it lacks that field.

your second feed having the ' and " in the title may be causing a problem because as you noted &#x27; does not equal ' when comparing strings.

i will look into the 2nd problem some more when i get a chance. i have 2 more finals today that are back to back and then i graduate on sunday. will have lots more time after that to work on it.

andy, your problem may be the same problem as bobby ... having the ' or " in the title may mess up when it checks for duplicates. because in the database they are not stored as " or ' but html encoded. while they appear the same, the string is different. i'm gonna look more into this.

Can anyone else get this one to work? http://news.search.yahoo.com/usns/yn...?p=half-life+2

It doesn't appear to be working at all.

-Andy

what does it not do? does anything show up? it appears to have all the proper fields, but it isn't rss, it is rdf which is slightly different, notice the CDATA tags around the title.

Yeah, it doesn't show up at all.. doesn't do anything. It would be cool if this would work, because you can search for specific news. Unless, you know of one?

-Andy

how can i moderate what news gets shown, Moderate Usergroup dosent seem to be working....

http://www.syndic8.com

you can look around on there for news feeds.

what exactly do you mean?

i want to select exactly what appears on my site before the bot posts it

Slapyo - I've just had a thought about why it might be duplicating posts for certain forums. If you've included the slight code modification highlighted by yourself in Post 222 of this thread (which strips HTML codes from the thread title), then could it be that because tags are being removed, the titles are different. Therefore, when a comparison is made, they fundamentally look different and therefore get posted as a new thread?

The reason this happens for some forums and not others, is because some RSS feeds don't use HTML tags and some do?

What do you reckon?

oh you want to moderate which posts come in ... hrm, unless there is a built in option to moderate posts by forum, it's not built into the rss feed. sorry.

true, that could also be it. maybe a comparison of the title with html tags stripped to a comparison of the title from the feed with html tags stripped.

Fatal error: Call to undefined function: html_entity_decode() in /home/street44/public_html/forum/includes/RSS/rss_update.php on line 77

I've had a think about my problem using feeds without a publication date.

Would it be possible to modify the code, so that it only checks for similarly titled threads? Without any date checking? I think that would solve my problems at least for the meantime :)

I think that's a good idea and probably would solve it. You're not likely to get the same headline again - so that would work for me also.

Slapyo - any ideas?

I'm going to try manually editing the topic titles as they come in: that would hopefully solve that problem in the meantime. I'll let you know if that works. :)

here you go, in rss_update.php:

FIND:
REPLACE WITH:

Anyone seen this? I get this as a cron error every hour:

I suspect one of the news feed is timing out. Is there a way to prevent this error being emailed to me every hour if a news feed times out?

Just wondering, and thanks for the hack.

hm, i think you would have to add in a script to watch the time and if the execution time hit 29 seconds to stop. not sure exactly how to put that in though.

Stange... :( I have the newest version of the hack installed but I am still getting duplicate stories. Anyone know why this might happen? I went to bed and now I have like 7 stories saying "Intel's 3D divorce rate" and another 7 duplicating another story and so on. The duplicates are not in a row.

Please help :) My bot is nuts.

what is the link to your RSS feed?

I found this mod for RSS feeds. The code may help you discover how others tackle the post duplication problem.

http://www.w3reports.com/forumfeeds.php

looks like it's the same script... copyright issues?

that was one of the original rss feed scripts that doesn't insert the news with the function built into vB. no search works, no post counts go up, nada.

I get 30 second max execution errors on my forum at www.smsmasters.co.uk My members got so fed with 17 pages worth of news in Today's Posts. Is there a way to stop these posts appearing in Today's Posts or New Posts???