NNTP Gateway for Usenet ( Newsgroups ), Mailing Lists

The latest version: V2.3.2 (31 Oct 2004)

What is it?

This hack provides a gateway from vBulletin forums to the USENET. It allows forum members to read newsgroup messages as well as to send posts to the newsgroups. In a nutshell, it turns vBulletin forums into a basic function news reader.

For an added bonus, this package also supports mailing list archive and forum to mailing list gateway. The software retrieves messages from a pop3 account and import them into the forums threaded. It also allows a forums user to send messages to the mailing list by the forum posting interface.

Why?

The gateway will be appreciated by forum members who do not know how to set up a news reader, or people who do not have port 119 open in their network. A forum interface to read and post news are often considered to be better than a news reader.

The gateway adds more contents to your forums and allows the members to interact with wider audience. It is common to see new users signing up to use this feature.

History

This hack is basically a hacked vB2 NNTP gateway written by Gilby to work with vB3. The original hack can be found here:

https://vborg.vbsupport.ru/showthrea...7&page=1&pp=15


Features

Some of the features and improvement include:

• Handles both newsgroup posts and mailing lists.
• Handles MIME encoded messages and attachment.
• Handles UUEncoded messages.
• Two Admin Control Panels for all settings.
• Automated installation.
• Correct thread implementation. All imported messages and posts sent to newsgroups will be threaded correctly according to References and Message-ID headers.
• Optional setting to thread by subject for mailing lists that strip references header.
• Supports TABLE_PREFIX.
• Imported messages will be searchable just like normal forum posts.
• Similar threads are built when the messages are imported if the system setting is on.
• Multiple attachments importing.
• Save attachment to database or file according to vB setting.
• Thumbnail created while importing image files.
• Multiple attachment post to newsgroup is also supported.
• Selectable X-No-Archive header honouring system.
• Light on system resources.
• Global Killfile to filter out unwanted messages. Killfiles can have OR and AND effectiveness.
• Strip footer by setting for incoming messages. Strips out repeated message footers.
• Built options to disable signatures and footers for out going messages.

Future development

• AdminCP interfaces. (completed)
• Installation/upgrade script. (completed, coded into AdminCP)
• Traditional "> " lead posting interface.

What do you need and do not need?

You do not need to edit vB php files if you use two bookmarks
You need to edit one vB php file to insert two lines of code if you want direct click links within AdminCP.
No need to run SQL.

Installation

Basically the AdminCP interfaces take care of installation as well as upgrade in the background. Back up your database first. The scripts can modify your database without alerting you!

Please follow the instructions in the readme.txt file for more details.

Support

Please post your support questions to this thread. Please check the FAQ in the third post within this thread first.

================== UPDATES ======================
Jul 31 2004 (release version 2.2.0)

• Provided option of not importing attachments
• Provided option of pause between messages
• Provided option of not sending messages to USENET
• Provided mechanism to skip importing a message if it was crashed on it
• Provided "Default Footer" option
• Provided strip_footer function for mailing lists
  (written originally by KevinM)
• Provided Killfile functions with Header/Body/AND/OR settings
• Provided ability to thread messages by subject for mailing lists
• Provided ability to use a single POP3 account for different mailing lists
  (use prefix)
• Merged KevinM's attachment routine, save thumbnail to hard drive
  as vB3.0.3
• Official support to have the script run by Scheduled Tasks
• Many bug fixes

Please note, when you manually run the script, you must turn on the debug mode. Otherwise nothing will be displayed on the screen.

To run as debug mode: http://forums.yourdomain.com/gateway.php?debug=1

25 Oct 2004

• Provided the option of sending posts using the real email address
• Provided the option of sending posts using a fixed email address for
  each group.
• Posts are now held back if they are in the moderating queue
• Scheduled task log is now working
• Provided the option of sending the debug messages to the vB3 cron log
• Bug fixes

31 Oct 2004

Fixed bugs running the gateway by the vB3 Scheduled Tasks

1 March 2004

Uploaded the new zip file. It contains one or two bug fixes.

[KevinM]

Here are two more mini-hacks which some may find useful. Thanks to BustaCap for testing the threading-by-subject script and to trackpads for testing the killfile filter.

Threading by Subject Title
The threading by subject script works on email lists only, whereby sometimes there are no message references included in the headers, so the only way to match the thread is by the subject title. Now with bugs removed

Killfile Filter to block posts
The killfile is similar to the equivilent you would find on a newsreader. It allows the admin to block certain messages from downloading if certain words or phrases are met by checking the posters name, subject title, email address or server where they originated. It is a fairly lightweight filter suitable for blocking annoying trolls and needs to used carefully to avoid blocking legitimate posts. Some may find this against the principles of usenet, but it's here if you want it.

You can use either or both of these together and are compatible with the stripping of email footers.

[scottct1]

Does the threading for Mailing lists work correctly now? I installed it the other day and it worked a few times, and then I started getting mysql errors (posted above in this thread)

Also please explain how to set it up to strip document footers. On yahoogroups I would like to keep the ads (I would feel guilty stipping those out) however I would like to remove the instructions on how to unsubscribe to the Yahoogroup that is on the bottom of each message.

Thanks

[KevinM]

Originally Posted by scottct1

Does the threading for Mailing lists work correctly now? I installed it the other day and it worked a few times, and then I started getting mysql errors (posted above in this thread)

Yes, problem solved

Originally Posted by scottct1

Also please explain how to set it up to strip document footers. On yahoogroups I would like to keep the ads (I would feel guilty stipping those out) however I would like to remove the instructions on how to unsubscribe to the Yahoogroup that is on the bottom of each message.

The strip-footers script can be found here You enter the characters or words that start the area you want stripped - just enough for it to be unique, and from that point on the rest of the message will be stripped.

[Marv]

Originally Posted by lierduh

should work although not tested. I must say it is quite strange that you need to have these to be able to post to the ng.

Thanks lierduh,
that worked for me.

We?re connecting to a non-public newsgroup on a buiseness-server, so we?re forced to do this modifications in the header. In german newsgroups you?ld find thousands of people yelling at you if you would participate a newsgroup with Outlook or OutlookExpress. They don?t even call those products "newsreader", they call it "data garbage centrifuge" because of the amount of bugs they are producing while trying to convert header-lines in a proper manner).
Most ng-members are using special news-readers like forte agent and others. These software pieces are producing correct headers but can?t read the one?s Outlook is inserting there.

lierduh, one problem remains.
In Germany they use letters like "?" "?" and "?" (and in capitols also of course). Is there anybody out there who could write some code which would convert these special german letters and other signs into ASCII on the server and in the moment gateway.php is sending the mesages out to the newsgroups ? (for outgoing posts to the newsserver & for incoming)

German letters:
? to &#196
? to &#228
? to &#214
? to &#246
? to &#220
? to &#252
? to &#223

Signs:
< to &#60
> to &#62
? to &#167
? to &#169
? to &#174
& to &#38
? to &#183
" to &#43
&#160 (Space)


We would really appreciate if someone could do this.. Any chance..? :ermm:

[Marv]

Originally Posted by KevinM

Killfile Filter to block posts

Great, Kevin - that?s another thing I was looking for. Thanks ! You read my mind

This filter checks 4 elements of each incoming message, and if ANY of them
match it will ignore that message.

The four elements are:

Username
Subject title
Email address
Server address

Could you add the body of a message as fifth element ? Some (including me) could find that very useful.

[Marv]

lierduh,

i noticed that the hack?s only fetching posts (sending them to a newsserver) which are answers. postings which are opening up a new thread aren?t included and they never show up in a newsgroup.

Can anyone confirm that for his/her site ?

[BustaCap]

Originally Posted by Marv

lierduh,

i noticed that the hack?s only fetching posts (sending them to a newsserver) which are answers. postings which are opening up a new thread aren?t included and they never show up in a newsgroup.

Can anyone confirm that for his/her site ?

I just tested this and my new thread showed up in the newsgroup.

When I first installed this hack I was using the vB cron for running the gateway, with the gateway.php supposed to be run every hour. However, I found that it was only running several times a day for some reason. When I used this method posts would often not show up. To trouble shoot I manually ran gateway.php immediately after posting and I found that all of those posts would be sent. I now use my site cpanel to run the cron job every 10 minutes and all posts show up. Try posting a new thread and then manually run gateway.php and see if that works.

[Marv]

Originally Posted by BustaCap

Try posting a new thread and then manually run gateway.php and see if that works.

Hello BustaCap, thanks for your suggestion - but that?s exactly what I already did before I?d posted my question here. I have no glue why starting posts aren?t send to the newsservers. (It?s definitifly the hack itself - the postings just don?t seem to be in my forum when I run gateway.php..)

And even worse - I can?t run cron jobs in my hosting package :ermm:
Any ideas? (beside upgrading the hosting-package..)

[lierduh]

Originally Posted by Marv

Hello BustaCap, thanks for your suggestion - but that?s exactly what I already did before I?d posted my question here. I have no glue why starting posts aren?t send to the newsservers. (It?s definitifly the hack itself - the postings just don?t seem to be in my forum when I run gateway.php..)

A bit confused there. As soon as you post a new thread, the thread should be in your forums. That is nothing to do with the gateway. Or unless you meant: you posted a new thread, but the gateway script did not send it to the server, or the server did not accept your new thread post. In any case, you might like to run it manually, capture the screen print out and post here for fault finding purpose.

[trackpads]

Originally Posted by KevinM

Here are two more mini-hacks which some may find useful. Thanks to BustaCap for testing the threading-by-subject script and to trackpads for testing the killfile filter.

Threading by Subject Title
The threading by subject script works on email lists only, whereby sometimes there are no message references included in the headers, so the only way to match the thread is by the subject title. Now with bugs removed

Killfile Filter to block posts
The killfile is similar to the equivilent you would find on a newsreader. It allows the admin to block certain messages from downloading if certain words or phrases are met by checking the posters name, subject title, email address or server where they originated. It is a fairly lightweight filter suitable for blocking annoying trolls and needs to used carefully to avoid blocking legitimate posts. Some may find this against the principles of usenet, but it's here if you want it.

You can use either or both of these together and are compatible with the stripping of email footers.

GREAT work!!!!!!!