NNTP Gateway for Usenet ( Newsgroups ), Mailing Lists

The latest version: V3.5.0Alpha 1 (21 Jul 2005)

What is it?

This hack provides a gateway from vBulletin forums to the USENET. It allows forum members to read newsgroup messages as well as to send posts to the newsgroups. In a nutshell, it turns vBulletin forums into a basic function news reader.

For an added bonus, this package also supports mailing list archive and forum to mailing list gateway. The software retrieves messages from a pop3 account and import them into the forums threaded. It also allows a forums user to send messages to the mailing list by the forum posting interface.

Why?

The gateway will be appreciated by forum members who do not know how to set up a news reader, or people who do not have port 119 open in their network. A forum interface to read and post news are often considered to be better than a news reader.

The gateway adds more contents to your forums and allows the members to interact with wider audience. It is common to see new users signing up to use this feature.

History

This hack was originally coded by Gilby for vB2. I modified the hack to work with vB3 and have been maintaining and improving the hack. I have now done the modification for this to work with vB3.5.

The very original hack by Gilby can be found here:

https://vborg.vbsupport.ru/showthrea...7&page=1&pp=15

The vB3 version can be found here:

https://vborg.vbsupport.ru/showthread.php?t=65152

Features

Some of the features and improvement include:

• Handles both newsgroup posts and mailing lists.
• Handles MIME encoded messages and attachment.
• Handles UUEncoded messages.
• Two Admin Control Panels for all settings.
• Automated installation.
• Correct thread implementation. All imported messages and posts sent to newsgroups will be threaded correctly according to References and Message-ID headers.
• Optional setting to thread by subject for mailing lists that strip references header.
• Imported messages will be searchable just like normal forum posts.
• Similar threads are built when the messages are imported if the system setting is on.
• Multiple attachments importing.
• Save attachment to database or file according to vB setting.
• Thumbnail created while importing image files.
• Multiple attachment post to newsgroup is also supported.
• Selectable X-No-Archive header honouring system.
• Light on system resources.
• Global Killfile to filter out unwanted messages. Killfiles can have OR and AND effectiveness.
• Strip footer by setting for incoming messages. Strips out repeated message footers.
• Built options to disable signatures and footers for out going messages.

Future development

• Traditional "> " lead posting interface.

What do you need and do not need?

You do not need to edit vB php files.
No need to manually run SQL.

Installation

Basically the AdminCP interfaces take care of installation as well as upgrade in the background. Back up your database first. The scripts can modify your database without alerting you!

Please follow the instructions in the readme.txt file for more details.

It is not a must to click "INSTALL" for this hack if you use it, I do try better answering the questions from people who do.

Support

Please post your support questions to this thread. Please check the FAQ in the second post within this thread first. It may be surprising, but it is possible your question is answered in the adminCP NNTP settings.

Please always provide more details about your problem. Examples and debug messages will certainly help. Well, if you don't even try to have some effort to compose your question, the answer you get may also not include much effort.

================== UPDATES ======================
Jul 21 2005 (release version 3.5.0 Alpha 1)

AdminCP xml file is written by Kadence.

• Ported to work with vB3.5.
• Existing bug fixes.
• Debug message can be sent by email or logged in vB Tast Scheduler.
• Known problem: image attachment thumb nail creation still needs work.
• Needs to be done: Documentation.

Quick installation instruction:
1) Upload files to their correct directory
2) Enter AdminCP, click on NNTP Settings once to install/upgrade. Change settings to your like.
3) Click on NNTP Groups once to install/upgrade.
3a) For new install. Run once
http://www.yourdomain.com/setlastmsg.php

Delete this file afterwards.
If you are upgrading from an older version. There is no need to upload this file. Actually this file should not be there except brand new install.

4)
Manually run in debug mode: http://forums.yourdomain.com/gateway.php?debug=1

[dotcomguy]

Originally Posted by itdevil

I am having similar issues, the mod put too much load on the server, i am 2x DualCore Xeon 2.8 with 2GB ram 100mbit.... Cron set to run every 24hrs or midnight ... once the cron run the load shoots up to 6 - 20 till apache crashes....

even now the server is dead slow due the gateway.php cron

xyz 18394 11.2  3.6  86660 74972 ?        Ss   00:00  25:39 /usr/bin/php /home/xyz/public_html/forum/gateway.php

i have also noticed, sometimes gateway file (but stop leeching from news server) never get killed automatically, on next cron another process is created...

This is a great hack, but someone need to update and optimize the code.

I haven't had the same problem, but I would recommend running it more than once a day. I run it every 10 minutes. Otherwise you'll have to download a significantly more amount of data

[krohnathlonman]

the largest bottlekneck in large scale archiving is the hard drive speed. If you don't have high speed SCSI drives don't ever try to archive millions of posts! Run things manually and NOT through cron if you're weeks behind in the posts.

[arkadia]

i build a new nttp gateway from zero using php, C++ and Delphi. it's works very more efficiently than the actual nntp gateway (i test it on a 100.000 articles newsgroup without any problem) and also i inprouve the charset encoding/decoding and correct some litle bug i found in gateway.php

exemple : after inserting an article
$db->query("
UPDATE " . TABLE_PREFIX . "thread
SET firstpostid = $postid
WHERE threadid = $threadid
");

that look false and must be

$db->query("
UPDATE " . TABLE_PREFIX . "thread
SET lastpostid = $postid
WHERE threadid = $threadid
");



i need now some volonters to test it and to gave me they comments !

[TMM-TT]

Originally Posted by arkadia

i build a new nttp gateway from zero using php, C++ and Delphi. it's works very more efficiently than the actual nntp gateway

By the way - don't forget the handshake part between server<->client. The "MODE READER" (that should be placed before the AUTHINFO) is quite important to use, to identify the client as a client.

[arkadia]

i not understand exactly what you mean? off course i handle the AUTHINFO and all the neccessary nntp protocol !

[TMM-TT]

Originally Posted by arkadia

i not understand exactly what you mean? off course i handle the AUTHINFO and all the neccessary nntp protocol !

In the current version of the gateway the client "identification" is missing. Since I run a usenet-server that communicates internally with another news-server, the gateway can't handshake correctly without the MODE-string.

A "normal" usenet-client sends "MODE READER" to a server before the authorization, and a usenet-server sends "MODE SERVER" to another server, because of the different article-transfering modes. This usually is based on IP, so if a server is configured to handle feed-traffic, it also treats a local client as a server - which is wrong.

It doesn't happen often, but it happens and may be important for some.

[arkadia]

ok, i do it ! i send a mode reader just after the connection

[scoutz]

wow this mod seem to work fine in 3.6.8 as well as 3.6.7 just needs a few very small query fixes. thanks for this mod.

[mccollin]

Would you share the fixes?

[solokron]

This is what I am receiving when I run gateway.php

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
	<title>Domain - Forum Database Error</title>
	<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" />
	<style type="text/css">
	<!--	
	body { background-color: white; color: black; }
	#container { width: 400px; }
	#message   { width: 400px; color: black; background-color: #FFFFCC; }
	#bodytitle { font: 13pt/15pt verdana, arial, sans-serif; height: 35px; vertical-align: top; }
	.bodytext  { font: 8pt/11pt verdana, arial, sans-serif; }
	a:link     { font: 8pt/11pt verdana, arial, sans-serif; color: red; }
	a:visited  { font: 8pt/11pt verdana, arial, sans-serif; color: #4e4e4e; }
	-->
	</style>
</head>
<body>
<table cellpadding="3" cellspacing="5" id="container">
<tr>
	<td><img src="http://www.mydomain.com/forum/image.php?type=dberror" alt="Database Error" width="48" height="48" /></td>
	<td id="bodytitle" width="100%">Database error</td>
</tr>
<tr>
	<td class="bodytext" colspan="2">The Domain - Forum database has encountered a problem.</td>
</tr>
<tr>
	<td colspan="2"><hr /></td>
</tr>
<tr>
	<td class="bodytext" colspan="2">
		Please try the following:
		<ul>
			<li>Load the page again by clicking the <a href="#" onclick="window.location = window.location;">Refresh</a> button in your web browser.</li>
			<li>Open the <a href="/">www.mydomain.com</a> home page, then try to open another page.</li>
			<li>Click the <a href="javascript:history.back(1)">Back</a> button to try another link.</li>
		</ul>
	</td>
</tr>
<tr>
	<td class="bodytext" colspan="2">The www.mydomain.com forum technical staff have been notified of the error, though you may <a href="mailto:admin@mydomain.com">contact them</a> if the problem persists.</td>
</tr>
<tr>
	<td class="bodytext" colspan="2">&nbsp;<br />We apologise for any inconvenience.</td>
</tr>
</table>

<form><textarea rows="15" cols="70" wrap="off" id="message">Database error in vBulletin 3.6.8:

Invalid SQL:

			SELECT post.*, thread.*,
			post.dateline AS postdateline, post.msgid AS postmsgid,
			thread.title AS threadtitle, post.visible AS postvisible,
			thread.visible AS threadvisible
			FROM post as post LEFT JOIN thread as thread
			ON (thread.threadid = post.threadid
			AND post.userid = thread.postuserid
			AND post.postid = thread.firstpostid)
			WHERE post.isusenetpost = 0
			AND post.postid &gt; 
			AND thread.forumid = 27;

MySQL Error  : You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'AND thread.forumid = 27' at line 11
Error Number : 1064
Date         : Monday, December 10th 2007 @ 07:00:51 PM
Script       : http://www.mydomain.com/forum/gateway.php?debug=1
Referrer     : 
IP Address   : 000.000.000.000
Username     : admin
Classname    : vB_Database</textarea></form>

</body>
</html>