NNTP Gateway for Usenet ( Newsgroups ), Mailing Lists

The latest version: V2.3.2 (31 Oct 2004)

What is it?

This hack provides a gateway from vBulletin forums to the USENET. It allows forum members to read newsgroup messages as well as to send posts to the newsgroups. In a nutshell, it turns vBulletin forums into a basic function news reader.

For an added bonus, this package also supports mailing list archive and forum to mailing list gateway. The software retrieves messages from a pop3 account and import them into the forums threaded. It also allows a forums user to send messages to the mailing list by the forum posting interface.

Why?

The gateway will be appreciated by forum members who do not know how to set up a news reader, or people who do not have port 119 open in their network. A forum interface to read and post news are often considered to be better than a news reader.

The gateway adds more contents to your forums and allows the members to interact with wider audience. It is common to see new users signing up to use this feature.

History

This hack is basically a hacked vB2 NNTP gateway written by Gilby to work with vB3. The original hack can be found here:

https://vborg.vbsupport.ru/showthrea...7&page=1&pp=15


Features

Some of the features and improvement include:

• Handles both newsgroup posts and mailing lists.
• Handles MIME encoded messages and attachment.
• Handles UUEncoded messages.
• Two Admin Control Panels for all settings.
• Automated installation.
• Correct thread implementation. All imported messages and posts sent to newsgroups will be threaded correctly according to References and Message-ID headers.
• Optional setting to thread by subject for mailing lists that strip references header.
• Supports TABLE_PREFIX.
• Imported messages will be searchable just like normal forum posts.
• Similar threads are built when the messages are imported if the system setting is on.
• Multiple attachments importing.
• Save attachment to database or file according to vB setting.
• Thumbnail created while importing image files.
• Multiple attachment post to newsgroup is also supported.
• Selectable X-No-Archive header honouring system.
• Light on system resources.
• Global Killfile to filter out unwanted messages. Killfiles can have OR and AND effectiveness.
• Strip footer by setting for incoming messages. Strips out repeated message footers.
• Built options to disable signatures and footers for out going messages.

Future development

• AdminCP interfaces. (completed)
• Installation/upgrade script. (completed, coded into AdminCP)
• Traditional "> " lead posting interface.

What do you need and do not need?

You do not need to edit vB php files if you use two bookmarks
You need to edit one vB php file to insert two lines of code if you want direct click links within AdminCP.
No need to run SQL.

Installation

Basically the AdminCP interfaces take care of installation as well as upgrade in the background. Back up your database first. The scripts can modify your database without alerting you!

Please follow the instructions in the readme.txt file for more details.

Support

Please post your support questions to this thread. Please check the FAQ in the third post within this thread first.

================== UPDATES ======================
Jul 31 2004 (release version 2.2.0)

• Provided option of not importing attachments
• Provided option of pause between messages
• Provided option of not sending messages to USENET
• Provided mechanism to skip importing a message if it was crashed on it
• Provided "Default Footer" option
• Provided strip_footer function for mailing lists
  (written originally by KevinM)
• Provided Killfile functions with Header/Body/AND/OR settings
• Provided ability to thread messages by subject for mailing lists
• Provided ability to use a single POP3 account for different mailing lists
  (use prefix)
• Merged KevinM's attachment routine, save thumbnail to hard drive
  as vB3.0.3
• Official support to have the script run by Scheduled Tasks
• Many bug fixes

Please note, when you manually run the script, you must turn on the debug mode. Otherwise nothing will be displayed on the screen.

To run as debug mode: http://forums.yourdomain.com/gateway.php?debug=1

25 Oct 2004

• Provided the option of sending posts using the real email address
• Provided the option of sending posts using a fixed email address for
  each group.
• Posts are now held back if they are in the moderating queue
• Scheduled task log is now working
• Provided the option of sending the debug messages to the vB3 cron log
• Bug fixes

31 Oct 2004

Fixed bugs running the gateway by the vB3 Scheduled Tasks

1 March 2004

Uploaded the new zip file. It contains one or two bug fixes.

[KevinM]

Originally Posted by john_rsd

Kevin
So in order for the threading function to be initiated it checks the first chars of the subject feild for matches on 'RE' ? Then parses an subject index for a match?

The primary method of matching threads is by a reference number which is inthe headers of newsgroup posts and emails. This was written by Lierduh and is highly effective. All newsgroups thread pretty much perfectly just by this method.

Some mail groups (my personal experience is with Yahoo groups) do not reliably put the reference number in the header of the email, therefore I wrote a secondary check (case insensitive!) which checks again before starting a new thread. The "Re: " can appear anywhere in the subject title.

If you really, really have problems, you can find in the script where it searches for "Re: " and change it to "", and it will search every post that comes through for a subject match. This WILL slow things down, and will also match a thread from (say) 3 years ago if it has the same title. This is completely untested (and not recommended) as the basic threading will pick up most of them.

Originally Posted by john_rsd

BTW, thanks for this really cool mod

Lierduh cooked the Sunday roast, I just made some gravy :cheeky:

[john_rsd]

Thanks Kevin

I undestand a little better now, so ill dig deeper.

the source of the list is a private server using Lyris, its not yahoo or anything like that.

I might try using"" as this list server only has 50 or so posts a day so is never busy enough to slow the server down.

Also if using "" it should only d/l message that are in the pop3 box so it might not cause a flood.

The roast and the gravy make a fine combo mate, sweet

[john_rsd]

Oh nuts

Just seen the glitch again

This time i ran the script directly

Gateway version 2.1.0 1 group(s) gatewayed.
Successfully connected to the 'mail.proteluser.com' pop3 server
Retrieving 4 E-mails from the 'mail.proteluser.com' pop3 server
Getting message number 1:
'Re: [dxp] OT: Differential Stripline Impedance' from Ian Wilson. New thread.

But this was a reply to an earlier one and has Re in it

Header from first messsage/thread

Return-Path: <bounce-dxp-3338@forums.altium.com>
Delivered-To: rsdtvcom-john@rsdtv.com
Received: (qmail 97607 invoked from network); 24 Jun 2004 17:25:39 -0000
Received: from unknown (HELO 203.10.94.132) (203.213.65.162)
by host109.ipowerweb.com with SMTP; 24 Jun 2004 17:25:39 -0000
Content-Class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0
Subject: [dxp] OT: Differential Stripline Impedance
Date: Thu, 24 Jun 2004 18:18:34 +0100
Message-ID: <LYRIS-3338-45721-2004.06.25-03.13.39--john#rsdtv.com@forums.altium.com>
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Thread-Topic: OT: Differential Stripline Impedance
Thread-Index: AcRaEGUknFNjWsP/Qper0ZkkCMrRLA==
From: "Jason Morgan" <jason.morgan@citel.com>
To: "DXP Technical Forum" <dxp@forums.altium.com>
List-Unsubscribe: <mailto:leave-dxp-3338F@forums.altium.com>
Reply-To: "DXP Technical Forum" <dxp@forums.altium.com>

Header from reply messsage

Return-Path: <bounce-dxp-3338@forums.altium.com>
Delivered-To: rsdtvcom-john@rsdtv.com
Received: (qmail 69724 invoked from network); 24 Jun 2004 23:30:20 -0000
Received: from unknown (HELO 203.10.94.132) (203.213.65.162)
by host109.ipowerweb.com with SMTP; 24 Jun 2004 23:30:20 -0000
Message-Id: <LYRIS-3338-45752-2004.06.25-09.17.27--john#rsdtv.com@forums.altium.com>
X-Sender: i.wilson@considered.com.au@mail.considered.com.au@127.0.0.1
X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9
Date: Fri, 25 Jun 2004 09:25:28 +1000
To: "DXP Technical Forum" <dxp@forums.altium.com>
From: Ian Wilson <i.wilson@considered.com.au>
Subject: Re: [dxp] OT: Differential Stripline Impedance
In-Reply-To: <LYRIS-395-45721-2004.06.25-03.13.39--i.wilson#considered.c
om.au@forums.altium.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
List-Unsubscribe: <mailto:leave-dxp-3338F@forums.altium.com>
Reply-To: "DXP Technical Forum" <dxp@forums.altium.com>

Is this because the Message ID is different? Or is the filter being confused between ID's in the reply and only seeing the first one (different) and not the second one (match)

John

[vprp]

When I run gateway.php, I get the message:

Gateway version 2.1.0 1 group(s) gatewayed.
Could not connect to NNTP-serverNot connectedNot connected

What's the problem?

I am trying to do this for the newsgroup rec.sport.golf

[Princeton]

Originally Posted by vprp

When I run gateway.php, I get the message:

Gateway version 2.1.0 1 group(s) gatewayed.
Could not connect to NNTP-serverNot connectedNot connected

What's the problem?

I am trying to do this for the newsgroup rec.sport.golf

you are not allowed to access newsgroup via script or you have settings wrong (username/password if required).

I remommend you do a test first.
Use-------
- msnews.microsoft.com as SERVER
- microsoft.public.security as NEWSGROUP

NOTE: add 58700 as LAST MESSAGE or check the XREF # in header of last message found in usergroup.

[vprp]

I tried that microsoft newsgroup and i got the same message. should i try to reinstall?

[Princeton]

I still get the message.
And ... yes, I am using the code posted on that POST.

Would you care to see the file?
I've made some "visual/html" modifcations to the file ... however, everything is clearly marked where the changes have been made (your mini-hacks , etc,)

[Princeton]

Originally Posted by vprp

I tried that microsoft newsgroup and i got the same message. should i try to reinstall?

yes, start from scratch ... make sure you read the instructions carefully

[john_rsd]

Anyone else using email delivery method and found problems with the script crashing?

I have been sending most things I could think of at the script and it has crashed twice, big time.

The first time was when a message was forwarded to the lists, the forward contained multiple forwarded messages and no subject. In this case the first message was part entered into the database (with no subject filled in) but nothing on the rest.

The second time was when a message was sent to the lists with no subject, perhaps it was still recovering from the first crash.

I might go back and try the nntp feed instead of email.

[Imhotep]

:ermm: My forums stopped grabbing new Usenet messages on 6/21. I'm trying to finger out exactly where the log jam has occured. My cron job is still going off like clockwork.

This is the entire content of my cron results minus the header garbage.

Gateway version 2.1.0  4 group(s) gatewayed.

Connecting to server, server says: 200 News.GigaNews.Com

Server responded after user name: 381 more authentication required

Server responded after password: 281 News.GigaNews.Com

Info for comp.graphics.apps.lightwave at news-40.giganews.com:
211 12589 237200 249788 comp.graphics.apps.lightwave

Getting message number 249690:

Any ideas as to what is happening? It looks like it only hits one group... then stops. ? :ermm: