NNTP Gateway for Usenet ( Newsgroups ), Mailing Lists

The latest version: V3.5.0Alpha 1 (21 Jul 2005)

What is it?

This hack provides a gateway from vBulletin forums to the USENET. It allows forum members to read newsgroup messages as well as to send posts to the newsgroups. In a nutshell, it turns vBulletin forums into a basic function news reader.

For an added bonus, this package also supports mailing list archive and forum to mailing list gateway. The software retrieves messages from a pop3 account and import them into the forums threaded. It also allows a forums user to send messages to the mailing list by the forum posting interface.

Why?

The gateway will be appreciated by forum members who do not know how to set up a news reader, or people who do not have port 119 open in their network. A forum interface to read and post news are often considered to be better than a news reader.

The gateway adds more contents to your forums and allows the members to interact with wider audience. It is common to see new users signing up to use this feature.

History

This hack was originally coded by Gilby for vB2. I modified the hack to work with vB3 and have been maintaining and improving the hack. I have now done the modification for this to work with vB3.5.

The very original hack by Gilby can be found here:

https://vborg.vbsupport.ru/showthrea...7&page=1&pp=15

The vB3 version can be found here:

https://vborg.vbsupport.ru/showthread.php?t=65152

Features

Some of the features and improvement include:

• Handles both newsgroup posts and mailing lists.
• Handles MIME encoded messages and attachment.
• Handles UUEncoded messages.
• Two Admin Control Panels for all settings.
• Automated installation.
• Correct thread implementation. All imported messages and posts sent to newsgroups will be threaded correctly according to References and Message-ID headers.
• Optional setting to thread by subject for mailing lists that strip references header.
• Imported messages will be searchable just like normal forum posts.
• Similar threads are built when the messages are imported if the system setting is on.
• Multiple attachments importing.
• Save attachment to database or file according to vB setting.
• Thumbnail created while importing image files.
• Multiple attachment post to newsgroup is also supported.
• Selectable X-No-Archive header honouring system.
• Light on system resources.
• Global Killfile to filter out unwanted messages. Killfiles can have OR and AND effectiveness.
• Strip footer by setting for incoming messages. Strips out repeated message footers.
• Built options to disable signatures and footers for out going messages.

Future development

• Traditional "> " lead posting interface.

What do you need and do not need?

You do not need to edit vB php files.
No need to manually run SQL.

Installation

Basically the AdminCP interfaces take care of installation as well as upgrade in the background. Back up your database first. The scripts can modify your database without alerting you!

Please follow the instructions in the readme.txt file for more details.

It is not a must to click "INSTALL" for this hack if you use it, I do try better answering the questions from people who do.

Support

Please post your support questions to this thread. Please check the FAQ in the second post within this thread first. It may be surprising, but it is possible your question is answered in the adminCP NNTP settings.

Please always provide more details about your problem. Examples and debug messages will certainly help. Well, if you don't even try to have some effort to compose your question, the answer you get may also not include much effort.

================== UPDATES ======================
Jul 21 2005 (release version 3.5.0 Alpha 1)

AdminCP xml file is written by Kadence.

• Ported to work with vB3.5.
• Existing bug fixes.
• Debug message can be sent by email or logged in vB Tast Scheduler.
• Known problem: image attachment thumb nail creation still needs work.
• Needs to be done: Documentation.

Quick installation instruction:
1) Upload files to their correct directory
2) Enter AdminCP, click on NNTP Settings once to install/upgrade. Change settings to your like.
3) Click on NNTP Groups once to install/upgrade.
3a) For new install. Run once
http://www.yourdomain.com/setlastmsg.php

Delete this file afterwards.
If you are upgrading from an older version. There is no need to upload this file. Actually this file should not be there except brand new install.

4)
Manually run in debug mode: http://forums.yourdomain.com/gateway.php?debug=1

[rmd708]

Has anyone gotten the footer striper to work. It doesn't work as far as I can tell.

[rmd708]

By the way, the thing about attachments not working. It may be because guests don't have permission to post attachments, and these posts are being put through as guests.

[dkendall]

I implemented the e-mail address masking feature mentioned earlier in this thread. But we use vBulletin for tech suport, and sometimes we post messages telling users to e-mail us directly.

So I've tweaked the masking feature so that it does not strip out addresses from certain domains.

I've also made it configurable in the NNTP Gateway Settings by adding these variables:

Title: Email Address Mask
Varname: email_address_mask
Value: \1 (AT) \2 (DOT) \3
Description: Mask for obfusicating email addresses in imported messages.

Title: Mask Email Exclude
Varname: mask_email_exclude
Value: onedomain.com;domaintwo.net;third.org
Description: Semicolon-delimited lowercase list of domains for which e-mail addresses should NOT be masked.

In gateway.php locate the line logging("Gateway version "... and insert this beneath it:

// Get email mask settings
$email_address_mask = $nntp_settings['email_address_mask'];
$mask_email_exclude = explode(';', $nntp_settings['mask_email_exclude']);

Locate the following block of code:

	//Hide real email address for mailing lists
	if ($nntp['grouptype'] == 'mail')
	{
		$message['text'] = preg_replace('/([-_.\\w]+)@([\\w]+[-\\w]+)\\./', '\\1 (AT) \\2 (DOT) ', $message['text']);
	}

Replace it with this:

	//Mask real email addresses in message text
	if (isset($email_address_mask))
		$message['text'] = preg_replace_callback('/(.+)@([a-zA-Z0-9\.]+)/', "mask_email_callback", $message['text']);

A few lines later, after the call to from_name, add the following code:

	//clean up name
	if (isset($email_address_mask))
		$from_name = preg_replace('/(.+)@(.+)/', '\\1', $from_name);

NOTE: This handles the case where the display name in the nntp posting is an e-mail address. It simply changes my.name@domain.com to my.name, rather than using the email_address_mask value and without regard to the mask_email_exclude setting.

Finally, add this code to the end of functions_nntp.php:

function mask_email_callback($parts)
{
	global $email_address_mask;
	global $mask_email_exclude;

	if (in_array(strtolower($parts[2]), $mask_email_exclude))
		return $parts[0];

	return preg_replace('/(.+)@(.+)\\.([^\\.]+)/', $email_address_mask, $parts[0]);
}

This will replace "my.name@my.domain.com" with "my.name (AT) my.domain (DOT) com". Of course, you can change the email_address_mask value to anything (e.g., "\1@..."). Leaving it blank will disable this feature.

David

[Deepdog009]

Oh, been away 4 awhile. Your ? about links is pertaining to the settings in admincp. I deleted the links coming back to forum. I deleted most info linking back and setup to only show post info without sig and footer.

dkendall when U export your posts do U experience sometimes some posts dont up to usenet? Why is that? Is there a fullproof way to verify that all posts were exported to usenet without problems.

I think it might have something to do with mod posts. Ive noticed that non mod posts work ok, but mod posts sometimes dont export.

Updated files and now getting this message on some posts:
Warning: Invalid argument supplied for foreach() in /includes/functions_threadedmode.php on line 202
Email mask setting changes page 18

Used updates from page 17, working beauts!

[tiffin100]

Hi

I am having a massive problem with duplicated threads being imported. I have even tried setting the time delay to 2 seconds between posts incase it was a slow server issue.

I have set the cron to run once an hour, which works fine.

I get no errors, I just get duplicate threads and lots of them, any ideas or help would be great..

[dkendall]

Hmmm. I answered this yesterday, but my reply hasn't shown up!

Originally Posted by Deepdog009

dkendall when U export your posts do U experience sometimes some posts dont up to usenet?

No, I'm not seeing that.

Originally Posted by Deepdog009

Updated files and now getting this message on some posts:
Warning: Invalid argument supplied for foreach() in /includes/functions_threadedmode.php on line 202
Email mask setting changes page 18

I looked at this code, and don't see how it can be related to the Email mask setting changes. It's probably caused by posts who's parents are missing from vBulletin.

When you start using the NNTP gateway, you have to make sure the "Last Message Index" in NNTP Gateway Newsgroups is set to zero.

Also, it won't import posts from NNTP that originated in vBulletin unless you use the full_import_from_usenet mod I posted earlier.

David

[Deepdog009]

Ok, testing your updates, working ok so far. I guess U be right about some posts showing up with error message if begining thread not found. That sucks.

Im still having a minor problem with Moderation. When posts arent moderated they successfully export to NNTP, but when posts are moderated sometimes they dont feed to NNTP. Does NEbody know why that is???:cross-eyed:

Grinler got one 4 ya!
U B O ta

[lierduh]

Originally Posted by tiffin100

Hi

I am having a massive problem with duplicated threads being imported. I have even tried setting the time delay to 2 seconds between posts incase it was a slow server issue.

I have set the cron to run once an hour, which works fine.

I get no errors, I just get duplicate threads and lots of them, any ideas or help would be great..

This is due to having two instances of the scripts running at the same time. The script has ability to avoid such situation, but only check for scripts running within 30 minutes. So if you have an importation that is lasting for 1/2hour (eg. the very first import), don't set up/enble cron!

[dkendall]

Originally Posted by Deepdog009

When posts arent moderated they successfully export to NNTP, but when posts are moderated sometimes they dont feed to NNTP. Does NEbody know why that is???:cross-eyed:

I'm guessing here...

When the gateway copies posts to NNTP, it keeps track of the last message copied.

If you have a moderated post that is older than an unmoderated post, and the unmoderated post is copied over to NNTP, the gateway may never look at the older post.

David

[tiffin100]

Originally Posted by lierduh

This is due to having two instances of the scripts running at the same time. The script has ability to avoid such situation, but only check for scripts running within 30 minutes. So if you have an importation that is lasting for 1/2hour (eg. the very first import), don't set up/enble cron!

thanks lierduh for the quick response, this solved the problem.