Usenet gateway

THIS THREAD IS CLOSED!
Posts in here will NOT be answered

This hack has undergone many major changes since this thread was started. Consequently, most of the posts have become dated and of little use. To coincide with the latest major release (20010712), a new one has been started.

See this thread for the latest version and discussion

Updated version - New features & bugfixes

This release adds a few options and fixes some of the bugs discovered over the last few days.

• Added option in control panel for eliminating usenet users and posts from the stats.
• Added option in control panel for eliminating usenet posts from the 'Get New Posts' search.
• Added option in control panel to allow multiple iterations of replies load per batch run to avoid staggling posts left in the article table when pulling history or large batches.
• Removed hardcoded regexes that were removing things like 'fred wrote in message <123213@news.news.com>'. This should now be entered in your 'Replacements' in the 'Spam Control' section of the control panel.
• Corrected usenet user post count when importing usenet users
• Corrected miscounting of replies in the forum table. (this was confusing vB and causing the next thread link to display when it shouldn't)
• Disabled the refresh in the control panel after updates to prevent the 'cant send header' errors when gzip or buffered output is disabled.
• Prevented orhan replies with the subject of '(no subject)' being threaded incorrectly alongside non-related articles with no subject. They are now not loaded at all unless there is a reference in the header.

Download version 2.4 for vB 2.0b3

There are upgrade instructions for anyone who has installed v2.3. There are only two places in which to change code and a few SQL statements to run. I'll be around for a few hours if anyone installs it tonight and runs into problems. But it's straightforward enough... trust me

Both the newnews.pl and the usenet.php need replacing with this new version

The latest version of this hack will always be in this first post of the thread.

Fastworward, help!

A user asked me about some crap messages shown in a non-usenet forum, lots usenet posts that arenet from the one I pull with things like:

"This message has been automatically sent by Micromuse's Netcool/NNTP
Internet Service Monitor."

and some SPAM etc. dated Feb 12, but added today.

The forum in question is

http://www.lowcarber.org/forum/forum...p?s=&forumid=2

I'm still using 2.3. Is this a new bug, or a known one fixed in 2.4?

I only use one usenet group as forum #30 and it is populated correcly, while this stuff shows in forum #2.

Don't panic! The only way this can happen is if you have a destination forum for one of your usenet groups set to a real forum.

Just look in your usenet groups option in the control panel and find the offending group. If you're getting automated messages it sounds like it's alt.test or another test group.

If you're absolutely sure it's not showing on your groups options page then look in the usenet_group table for it. But there is no way this can happen by itself. The script has to be told which forum to place messages in.

Hmm,

I've deleted alt.test (was it set by default to forum #2)? I might've had run it by mistake before I set up the only usenet group I carry. My usenet server has a huge retention rate, so feb12 messages soesn't surprise me.

Originally posted by tamarian
Hmm,

I've deleted alt.test (was it set by default to forum #2)? I might've had run it by mistake before I set up the only usenet group I carry. My usenet server has a huge retention rate, so feb12 messages soesn't surprise me. ]

Nope. It was set to forumid 9999. This was done to avoid the very problem you've just ran into. Maybe I'll just leave the test group out altogether next time.

But if the messages have just appeared today, the script must still be pulling them from usenet. Was it in the usenet_group table?

No, my database shows one group only, alt.support.diet.low-carb, and the same in my CP.

I might've just noticed it today when a user reported it, since the retention is hight on y NNTP server, I pulled feb 12 messages in late March, so they appear at the bottom and don't get noticed. That's what comes to mind.

Hmmm, sorry about that. I think I will leave the alt.test group out of the next release... just to be safe. But there really is no way for the script to place messages in any forum other than that specified in the usenet_group table.

Can you clean up the forum easily? It's easy to identify a post as usenet, but it's not easy to determine which forum it's in because there's no forumid in the post table. If there's not too many it's probably safer to do it by hand rather than a query.

Let me know if you need any help.

Oh it could've been a screw up on my part. Remember my earlier post when I was using the 2.1 script on top the 2.3 version of the hack.

Thanks for the offer of help. There are quite a few of them to do by hand.

How can I identify usenet messages. I might be able to do something like

"delete all usenet posts where forum id=2"

If that what you ment by the difficulty due to not having forum id in post table, I can try this:

"delete all usenet posts prior to feb 13"?

There is a column in both the post table and the thread table called 'isusenetpost'. The threads are easy to delete as there is also a 'foumid' column there. But there is no way of knowing which usenet articles from the post table are from the forum that needs cleaning as there is no forumid column.

The cleanest way may be to purge all usenet posts and re-pull the news. You can opt to keep articles that are usenet, but were posted by your users in the control panel if that is a requirement.

I have no problem purging all usenet posts and polling them back again. How can I purge them? Just to make sure I got it right:

Delete all threads where is usenetpost=1
Delete all posts where isusentpost=1

Any other queries I need to do?