[RELEASE] Usenet Gateway for vB 2.01

Description:

This hack provides a gateway to selected usenet groups. The entire usenet group is mirrored locally. Posts submitted locally to mirrored newsgroups will also be sent out to usenet. A single perl script is executed via cron that collects news and posts any outgoing articles. Incoming messages are parsed against customizable spam filters and threaded correctly before being placed in your forums. By default, no personal information is sent to usenet other than the posters username and whatever you have globally configured as a footer and organization. The option does exist for a user to include a custom email to use for usenet posts. After initial installation (which involves creating some new tables, adding a few columns to existing vB tables, and entering the database connection information to the news script) configuration is completed via the vB control panel.

• See it in action at dBforums.com (usenet forums are at the bottom)
• Download latest version for vB 2.2.4

Current version is 20020323 for vB2.xx

Current feature list:

• correct threading of outgoing posts on usenet
• full control panel integration
• email notification to usenet replies
• emoticon translation into vb icons
• vb code removal or conversion prior to posting to usenet
• hyperlinked urls in messages
• color coded, italicized & indented quotes
• vB style quote to usenet style quote conversions for outgoing posts
• logging of outgoing posts
• support for multiple news servers
• support for seperate footers per forum in outgoing posts
• multi-language (selectable) handling of quoted MIME printable headers (for all those funny foreign characters)
• handling of mailing archive groups
• flexible spam control and replacement variable options for incoming and outgoing messages
• configurable auto-expire option
• option for users to show email address in their outgoing usenet posts (may be different than normal one in profile
• option to enable/disable user signatures on outgoing posts.
• fully compatible with vB moderation functions

Still to be done:

• canceling of messages after they have been sent to usenet via control message in accordance with RFC1036.
• binary attachment support
• email to PM gateway

Screeshots:
Main Options
More Options
Configuration of Newsgroups
Add New Newsgroups
Spam Control
What an imported usenet article looks like in vB

The latest version of this hack will always be in this first post of the thread.

Before asking questions in this thread, please read all the posts in this thread. If your question goes unanswered, it will probably be because the question has already been answered countless times in this thread.

This hack was created for use on my forums and will only be supported as time permits.


Links to required Perl Modules

• News::NNTPClient
• Net::SMTP
• MIME::WordDecoder
• Text::AutoFormat
• Mail::Address
• URI::Find
• DBI
• DBD::MySQL
• Date::Parse
• HTML::Entities

[Shaman]

This is/was a great hack. I hope that FF won't let it go by the wayside or that Jelsoft will take it (or the PHP version) on.

[bitbender]

Well, I got my perl module issues sorted out, and my version of Fast Forwards usenet interface appears to be working as designed on vb 2.2.2!!!!

I have only done medium level testing so far. I have placed a one reply in each usenet group, and the cron (W2k Sched task in my case) newnews.pl sorted it right out, send the replies out, and I verified them via M$oft's OutCrook Express's News Viewer!

I changed nothing, just reworked the mods into my 2.2.2 base. Ohh, I DID get the latest newnews.PL from this thread!

All appears well, I'll keep reporting back my results!!

[Shaman]

This is a better hack than the PHP version still. It's more intelligent, mainly. I have it working with 2.2.2 and am loathe to change. :/

[fastforward]

I made a start with a PHP version of this, but as Gilby has already released a version, there's no point continuing it.

I feel more comfortable developing a Perl version anyway as I know a bit more about it that PHP.

So, if there is still interest, I will go back to developing the Perl version.

A few things I intend to do:

• Reduce the number of modules required
• Support for attachments (probably only incoming ones)
• Clean up of multipart posts from Outlook and Netscape readers
• More options to handle messages regardless of whether it's parent is present
• Better performance with less server load. This is especially a problem with mailing list archives that have to use the title for threading.
• Extra options to handle x-noarcharve posts such as deleting after a specified time period.
• Modularize the script to ease the transfer of moving over to vb 3 when it arrives. The threaded model of version 3 should make things a lot easier.

A few things I'd like to do if possible:

• Clean up the code in the control panel ( volunteers are welcome )
• Option to recieve posts via a mail gateway such as news2mail.com. This will eliminate the need for a usenet account and will mean no more worrying about message number tracking. The problem will be finding a way to post back to usenet.
• Option to retrieve headers only and automagically connect to usenet to retrieve the body when viewed. This will save a boat load of space, but obviously won't allow searching of bodies. This will require hacking the vB code substantially and may not be practical, so don't hold your breath.

If people are still interested in a Perl version and anyone has any requests, let me know here.

PS to bitbender: I have been running it on 2.2 on for several weeks with no problems so far.

[Shaman]

I am still interested in it, for sure. I am using it with 2.2.4 now, it was a bit laborious hacking it in by hand, but I did it in under 20 minutes. It all seems to be working perfectly.

I'm not sure about the news2mail gateway but if you can make it work worth our while, that sure makes sense.

Performance isn't bad in my application but I have a local news server and I only track about five groups. I appreciate the work you did!

[bitbender]

Originally posted by fastforward
If people are still interested in a Perl version and anyone has any requests, let me know here.

PS to bitbender: I have been running it on 2.2 on for several weeks with no problems so far.

FF, please count me in.

Yes, I too, have had no issues with it, working just fine, at 2.2.3!

[AntiOnline]

Greetings:

FF, count me in too! This has become a major feature on our site, and you did an incredible job.

We have 75,000 users and growing, 27 hosted newsgroups, and everything's stood up to the test of time!

[jOOPeON]

Hi,

I just started to work again with usenet hack. I used it in
one forum a year ago, but then there was few problems.
Now everything works almost too smoothly
One humble question came in my mind...
Is it possible to attach file link to ougoing usenet post, if there is a attachment in post ?

-jOOP

Sample:

Test Test...


File in forum:
http://localhost/forums/attachment.php?postid=2878

--
Signature

[fastforward]

Originally posted by jOOPeON
Is it possible to attach file link to ougoing usenet post, if there is a attachment in post ?

Originally I had no intention of adding support for binary attachments. My reasoning was that you're probably mirroring text groups so have no business posting binary attachments to these groups.

Having said that, in the groups I mirror, people occasioanlly post log files and code snippets which people seem to find acceptable. I do intend to add incoming attachment support, so I guess there's no reason not to add outgoing attachments as well.

[jOOPeON]

Originally posted by fastforward

My reasoning was that you're probably mirroring text groups so have no business posting binary attachments to these groups.

True,

But I like to add just link to file, so the usenet readers could visit the site to see the file.
Actually it could look just as the attachment link in vb posts.

Here in scandinavia, binary groups are going to be removed from the servers, so this trick could be great help especially to buy/sell group users. Now group moderators just tells people to find some free internet space for their file uploads

-jOOP