NNTP Gateway for Usenet ( Newsgroups ), Mailing Lists

The latest version: V2.3.2 (31 Oct 2004)

What is it?

This hack provides a gateway from vBulletin forums to the USENET. It allows forum members to read newsgroup messages as well as to send posts to the newsgroups. In a nutshell, it turns vBulletin forums into a basic function news reader.

For an added bonus, this package also supports mailing list archive and forum to mailing list gateway. The software retrieves messages from a pop3 account and import them into the forums threaded. It also allows a forums user to send messages to the mailing list by the forum posting interface.

Why?

The gateway will be appreciated by forum members who do not know how to set up a news reader, or people who do not have port 119 open in their network. A forum interface to read and post news are often considered to be better than a news reader.

The gateway adds more contents to your forums and allows the members to interact with wider audience. It is common to see new users signing up to use this feature.

History

This hack is basically a hacked vB2 NNTP gateway written by Gilby to work with vB3. The original hack can be found here:

https://vborg.vbsupport.ru/showthrea...7&page=1&pp=15


Features

Some of the features and improvement include:

• Handles both newsgroup posts and mailing lists.
• Handles MIME encoded messages and attachment.
• Handles UUEncoded messages.
• Two Admin Control Panels for all settings.
• Automated installation.
• Correct thread implementation. All imported messages and posts sent to newsgroups will be threaded correctly according to References and Message-ID headers.
• Optional setting to thread by subject for mailing lists that strip references header.
• Supports TABLE_PREFIX.
• Imported messages will be searchable just like normal forum posts.
• Similar threads are built when the messages are imported if the system setting is on.
• Multiple attachments importing.
• Save attachment to database or file according to vB setting.
• Thumbnail created while importing image files.
• Multiple attachment post to newsgroup is also supported.
• Selectable X-No-Archive header honouring system.
• Light on system resources.
• Global Killfile to filter out unwanted messages. Killfiles can have OR and AND effectiveness.
• Strip footer by setting for incoming messages. Strips out repeated message footers.
• Built options to disable signatures and footers for out going messages.

Future development

• AdminCP interfaces. (completed)
• Installation/upgrade script. (completed, coded into AdminCP)
• Traditional "> " lead posting interface.

What do you need and do not need?

You do not need to edit vB php files if you use two bookmarks
You need to edit one vB php file to insert two lines of code if you want direct click links within AdminCP.
No need to run SQL.

Installation

Basically the AdminCP interfaces take care of installation as well as upgrade in the background. Back up your database first. The scripts can modify your database without alerting you!

Please follow the instructions in the readme.txt file for more details.

Support

Please post your support questions to this thread. Please check the FAQ in the third post within this thread first.

================== UPDATES ======================
Jul 31 2004 (release version 2.2.0)

• Provided option of not importing attachments
• Provided option of pause between messages
• Provided option of not sending messages to USENET
• Provided mechanism to skip importing a message if it was crashed on it
• Provided "Default Footer" option
• Provided strip_footer function for mailing lists
  (written originally by KevinM)
• Provided Killfile functions with Header/Body/AND/OR settings
• Provided ability to thread messages by subject for mailing lists
• Provided ability to use a single POP3 account for different mailing lists
  (use prefix)
• Merged KevinM's attachment routine, save thumbnail to hard drive
  as vB3.0.3
• Official support to have the script run by Scheduled Tasks
• Many bug fixes

Please note, when you manually run the script, you must turn on the debug mode. Otherwise nothing will be displayed on the screen.

To run as debug mode: http://forums.yourdomain.com/gateway.php?debug=1

25 Oct 2004

• Provided the option of sending posts using the real email address
• Provided the option of sending posts using a fixed email address for
  each group.
• Posts are now held back if they are in the moderating queue
• Scheduled task log is now working
• Provided the option of sending the debug messages to the vB3 cron log
• Bug fixes

31 Oct 2004

Fixed bugs running the gateway by the vB3 Scheduled Tasks

1 March 2004

Uploaded the new zip file. It contains one or two bug fixes.

[AWS]

Originally Posted by fustanella

It's great, that's what is is.

I am delighted with this; thank you! It was painless to install, worked both inbound and out the first time without fiddling, and adds a handy new service for my members.

I would like to find a way to automate the gateway.php calls under WinNT 4, though. I understand the concept of 'cron' but don't see how this maps directly over, other than through "Scheduled Tasks" - specifically, how I can run the PHP script either in a browser window that closes when done, or without the need for a browser at all.

(Anyone who has insights on my related question of importing text files as vB3 messages is welcome to chime in as well, over at https://vborg.vbsupport.ru/showthread.php?t=66250 - thanks.)

On Windows create a scheduled task.
Create a new task. When the box pops up browse to php.exe. The full path should show for the exe.
You should see something like
C:\Inetpub\php\php.exe
or whatever the path to php.exe is. At the end of that add the path to gateway.php. When done you should have something like
C:\Inetpub\php\php.exe C:\Inetpub\wwwroot\gateway.php
Now go to advanced and set it to run every 15 minutes for 23 hours and 59 minutes. Click OK and when the password box comes up put in your administartor password.
You now should have a scheduled task called php.exe. Open scheduled task folder and right click the task and run it. A DOS box should popup as it runs. If the box pops up and disapears right away then try to run the task command from a dos prompt. By doing this you'll see what the problem is. More than likely bad path.

[fustanella]

Originally Posted by AWS

C:\Inetpub\php\php.exe C:\Inetpub\wwwroot\gateway.php

Thank you. I'm familiar with scheduled tasks, and prefer to try in a command window before creating a task so errors can be seen. Good thing, as I got 'em.

Turck MMCache - can not create shared memory area
in unknown on line 0

- this occurs both in a graphical warning requiring clicking on the pretty button, and in the text in the command window. I'm running the same script that works through my browser.

I searched my machine and checked my vB install to make sure I didn't have various copies of PHP floating about, and I have only the one in my c:\php4\php dir. It's PHP 4.3.3 if that matters.

[Princeton]

the more I use this hack the more I like it!

... missing htmlspecialchars($subject)

[Cold Steel]

After getting thousands of messages from usenet, I'm starting to run into errors.

This is the latest:

Getting message number 300606: 
'Re: Bush's war -- against Richard Clarke' from Congenital Kano. Thread found by References.
Getting message number 300607: 
'Reversing Pledge, Bush Lets OPEC Bilk Americans' from Paul Kekai Manansala. New thread.
Getting message number 300608: 
<html><head><title>asiansINC.com Database Error</title><style type="text/css"><!--.error { font: 11px tahoma, verdana, arial, sans-serif; }--></style></head>
<body></table></td></tr></table></form>
<blockquote><p class="error">&nbsp;</p><p class="error"><b>There seems to have been a slight problem with the asiansINC.com database.</b><br />
Please try again by pressing the <a href="javascript:window.location=window.location;">refresh</a> button in your browser.</p><p class="error">An E-Mail has been dispatched to our <a href="mailto:">Technical Staff</a>, who you can also contact if the problem persists.</p><p class="error">We apologise for any inconvenience.</p><form><textarea class="error" rows="15" cols="100" wrap="off">Database error in vBulletin 3.0.1:

Invalid SQL: SELECT post.postid FROM thread AS thread, post AS post
                                    WHERE thread.threadid = post.threadid AND
                                    (post.msgid = '&lt;For-Tom_Cerco's-benefit-A772E2.23264904042004@phswest.com&gt;') AND
                                    thread.forumid = 72    
                                    ORDER BY post.dateline DESC LIMIT 1
mysql error: You have an error in your SQL syntax.  Check the manual that corresponds to your MySQL server version for the right syntax to use near 's-benefit-A772E2.23264904042004@phswest.com&gt;') AND
            

mysql error number: 1064

Date: Sunday 20th of June 2004 01:44:04 AM
Script: http://forums.asiansinc.com/gateway.php
Referer: 
Username: Webmaster
IP Address: **.**.**.***
</textarea></form></blockquote>

</body></html>

Any ideas?

[Cold Steel]

Also, I signed up for an account at Giganews for $7.99/month for 2GB.

Then I found out about Teranews.com - 50MB per day for only a $3.95 setup fee. This is slightly less than 2GB a month, but their paid acccount for $7.95 gives me 250MB a day (~7.3 GB month).

Is there really such a huge difference in the quality of the connection? I don't think I'll need that much bandwidth. I'm considering dropping giganews and signing up with a free account at teranews. Thoughts?

[Cold Steel]

Originally Posted by allan grossman

If it's a UNIX or Linux box use top to determine the process ID then kill the process with kill -9 process_id - if it's a Windows machine you can use Task Mangler to kill the task.

How do I determine which one is gateway.php?

[lierduh]

Originally Posted by Cold Steel

After getting thousands of messages from usenet, I'm starting to run into errors.

This is the latest:

                                    (post.msgid = '<For-Tom_Cerco's-benefit-A772E2.23264904042004@phswest.com>') AND

Any ideas?

The single quote in the message ID is the cause of the problem. Message ID should not contain quotes charactor. You can either bypass this message by editing the lastmsg in the control panel, or do the following:

Around line 338 find:

(post.msgid = '$par_msgid') AND

Change it to:

(post.msgid = '" . addslashes($par_msgid) . "') AND

[KevinM]

Originally Posted by trackpads

Also, is there a way to disable the killfile for mlists only? The killfile is an AWEOME addition and has really, really helped me on my site.

Yes you can. Find this:

PHP Code:

else if ($killfileresult = checkkillfile($message)) 


and change it to to:

PHP Code:

else if ($killfileresult = checkkillfile($message) AND $grouptype == 'news') 


This will work on newsgroups only.

To work on mailing lists only:

PHP Code:

else if ($killfileresult = checkkillfile($message) AND $grouptype != 'news') 


I haven't tried it, but it should work.

[KevinM]

Originally Posted by Marv

Could you add the body of a message as fifth element ? Some (including me) could find that very useful.

If you find this:

PHP Code:

if     ((stristr(trim($message['subject']), $killphrase)) OR
                        (stristr(trim($message['from']), $killphrase)) OR
                        (stristr(trim($message['path']), $killphrase))) 


and change it to this:

PHP Code:

if     ((stristr(trim($message['subject']), $killphrase)) OR
                        (stristr(trim($message['from']), $killphrase)) OR
                        (stristr(trim($message['path']), $killphrase)) OR
                        (stristr(trim($message['text']), $killphrase))) 


This will now search the main text of the message as well.

[KevinM]

This is a quick one to get the thread preview to work with this hack.

Find:

PHP Code:

logging("'$subject' from ". $from_name .
                    ". New thread.\r\n"); 


And below it put:

PHP Code:

// enable thread preview
                    $DB_site->query("UPDATE " . TABLE_PREFIX . " thread
                    SET firstpostid = $postid
                    WHERE threadid = $threadid"); 


If you have thread preview enabled in the adminCP then all new threads from now on will show a preview when you hover over them.