NNTP Gateway for Usenet ( Newsgroups ), Mailing Lists

The latest version: V2.3.2 (31 Oct 2004)

What is it?

This hack provides a gateway from vBulletin forums to the USENET. It allows forum members to read newsgroup messages as well as to send posts to the newsgroups. In a nutshell, it turns vBulletin forums into a basic function news reader.

For an added bonus, this package also supports mailing list archive and forum to mailing list gateway. The software retrieves messages from a pop3 account and import them into the forums threaded. It also allows a forums user to send messages to the mailing list by the forum posting interface.

Why?

The gateway will be appreciated by forum members who do not know how to set up a news reader, or people who do not have port 119 open in their network. A forum interface to read and post news are often considered to be better than a news reader.

The gateway adds more contents to your forums and allows the members to interact with wider audience. It is common to see new users signing up to use this feature.

History

This hack is basically a hacked vB2 NNTP gateway written by Gilby to work with vB3. The original hack can be found here:

https://vborg.vbsupport.ru/showthrea...7&page=1&pp=15


Features

Some of the features and improvement include:

• Handles both newsgroup posts and mailing lists.
• Handles MIME encoded messages and attachment.
• Handles UUEncoded messages.
• Two Admin Control Panels for all settings.
• Automated installation.
• Correct thread implementation. All imported messages and posts sent to newsgroups will be threaded correctly according to References and Message-ID headers.
• Optional setting to thread by subject for mailing lists that strip references header.
• Supports TABLE_PREFIX.
• Imported messages will be searchable just like normal forum posts.
• Similar threads are built when the messages are imported if the system setting is on.
• Multiple attachments importing.
• Save attachment to database or file according to vB setting.
• Thumbnail created while importing image files.
• Multiple attachment post to newsgroup is also supported.
• Selectable X-No-Archive header honouring system.
• Light on system resources.
• Global Killfile to filter out unwanted messages. Killfiles can have OR and AND effectiveness.
• Strip footer by setting for incoming messages. Strips out repeated message footers.
• Built options to disable signatures and footers for out going messages.

Future development

• AdminCP interfaces. (completed)
• Installation/upgrade script. (completed, coded into AdminCP)
• Traditional "> " lead posting interface.

What do you need and do not need?

You do not need to edit vB php files if you use two bookmarks
You need to edit one vB php file to insert two lines of code if you want direct click links within AdminCP.
No need to run SQL.

Installation

Basically the AdminCP interfaces take care of installation as well as upgrade in the background. Back up your database first. The scripts can modify your database without alerting you!

Please follow the instructions in the readme.txt file for more details.

Support

Please post your support questions to this thread. Please check the FAQ in the third post within this thread first.

================== UPDATES ======================
Jul 31 2004 (release version 2.2.0)

• Provided option of not importing attachments
• Provided option of pause between messages
• Provided option of not sending messages to USENET
• Provided mechanism to skip importing a message if it was crashed on it
• Provided "Default Footer" option
• Provided strip_footer function for mailing lists
  (written originally by KevinM)
• Provided Killfile functions with Header/Body/AND/OR settings
• Provided ability to thread messages by subject for mailing lists
• Provided ability to use a single POP3 account for different mailing lists
  (use prefix)
• Merged KevinM's attachment routine, save thumbnail to hard drive
  as vB3.0.3
• Official support to have the script run by Scheduled Tasks
• Many bug fixes

Please note, when you manually run the script, you must turn on the debug mode. Otherwise nothing will be displayed on the screen.

To run as debug mode: http://forums.yourdomain.com/gateway.php?debug=1

25 Oct 2004

• Provided the option of sending posts using the real email address
• Provided the option of sending posts using a fixed email address for
  each group.
• Posts are now held back if they are in the moderating queue
• Scheduled task log is now working
• Provided the option of sending the debug messages to the vB3 cron log
• Bug fixes

31 Oct 2004

Fixed bugs running the gateway by the vB3 Scheduled Tasks

1 March 2004

Uploaded the new zip file. It contains one or two bug fixes.

[lierduh]

Originally Posted by Odoin

I have two newsgroup feeds setup... seems to be one of the forum pulls the posts made to it back into the topic so there is double posts.. one from the local user and the same post over acting as if it was from a outside user... Any idea why it would do this on just one forum?

Have you changed "useragent" or organization? The latest version checks both useragent and organization in the header. The message only get imported if they do not match at the same time.

Otherwise do you have special charactors in the organization setting?

[lierduh]

Originally Posted by KevinM

I am getting this error whenever it hits a message with an attachment (with the latest script)



Is this related to creating the thumbnail?

I will come up with a solution very soon.

[lierduh]

Originally Posted by Imhotep

IMHO, it would be cool if we could add variables for incoming and outgoing ports. My host allows incoming on port 119 but not outgoing. So, with my new NSP, I am experimenting with getting posts to get out to Usenet.

Giganews allows connections to ports 119,80, and 23

Am I to assume that I can search the PHP files for "119" and change them to whatever I want? This is what I have done for my first test.... changing all instances of 119 to 80. So far, no luck. I hope my host did not block ALL outgoing NNTP data. :ermm: I still have 23 left to test.

The best way is to use the telnet method I described in the first page to find out exactly what is the problem.

As I mentioned before, if you can recieve the message, but not post. It mean the 119 port is OPEN! It means your web server can connect to the news server on port 119 using a high port on the web server.

What the host support might try to say to you if the guy really knows what he is talking about is. The 119 port is not open on your web server to let other people connect to it as a news server. That is, other people using a high port to connect to port 119 on your web server. This more likely is disabled!

For a TCP/IP to connect.

server port:119 <========> client port : high port (above 1024)

Also post some screen dump or test result always helps others to help you.

[Imhotep]

As I mentioned before, if you can recieve the message, but not post. It mean the 119 port is OPEN!

Agreed, but open one way only! Have you ever heard the term "firewall"? Like I said before, my host has blocked outgoing on port 119. They could care less about incoming. They block outgoing and cite "ongoing usenet abuses" as their excuse. Port blocking is a two lane road. You have incoming and outgoing. Just because I can receive posts does not mean it's open both ways. I hope I don't have to explain this again. :ermm:

And, since I did not get an aswer to my questions, but rather a misguided and thinly-veild scolding, I will have to email my host instead and see if they have a global block on all outgoing NNTP.

SERVER <====NNTP INCOMING====< NSP
SERVER >====NNTP BLOCKED?====> NSP

[lierduh]

Originally Posted by Imhotep

Agreed, but open one way only! Have you ever heard the term "firewall"? Like I said before, my host has blocked outgoing on port 119. They could care less about incoming. They block outgoing and cite "ongoing usenet abuses" as their excuse. Port blocking is a two lane road. You have incoming and outgoing. Just because I can receive posts does not mean it's open both ways. I hope I don't have to explain this again. :ermm:

And, since I did not get an aswer to my questions, but rather a misguided and thinly-veild scolding, I will have to email my host instead and see if they have a global block on all outgoing NNTP.

SERVER <====NNTP INCOMING====< NSP
SERVER >====NNTP BLOCKED?====> NSP

It seems you understand about firewall and TCP/IP, please explain what port is blocked and what is open, be it UDP or TCP protocol. How about an iptable/ipchain rule table for the firewall?

You receive and post news on the same port! I like to see the result of the telnet session from your web server to the nntp server's port 119. Do you know you can post news in the same session.

Now if they filter your tcp contents, then it is a different story, nothing to do with port blocking.

I am here trying to help you, you do not seems to understand my intention, rather, you think I gave you misguided information just for the fun of it. I shall not bother.

[Odoin]

No nothing has changed since the forum was first setup. It only double posts in one of the forums... the other one has no problem. I went as far as to remove the forum and newsgroup but it still did it... I am going to try a few other groups and see what comes up.

Originally Posted by lierduh

Have you changed "useragent" or organization? The latest version checks both useragent and organization in the header. The message only get imported if they do not match at the same time.

Otherwise do you have special charactors in the organization setting?

[lierduh]

Originally Posted by Odoin

No nothing has changed since the forum was first setup. It only double posts in one of the forums... the other one has no problem. I went as far as to remove the forum and newsgroup but it still did it... I am going to try a few other groups and see what comes up.

Can you get the full header for one of the messages in that group?

[recumbent615]

Originally Posted by Sanjiyan

Not trying to be big headed here, but the reason this option was put into the Gilby hack, was mainly because I asked it to be, I think to date no one else uses that function in the hack bar me, but it is a very nice function to have.

Anyway, can not wait for you to get this option working again, THEN I will upgrade my boards.

I would have to disagree with you on this, I have been using this hack for quite some time under Vb2.x and use it only for the import of mailing lists. I've also used to to grab the archrive of the mailing lists from news groups that have done the archival. The one thing that is keeping me at VB2.x has been this ability...

Thanks for bringing this up to rev ...


Kevin

[Imhotep]

Originally Posted by lierduh

It seems you understand about firewall and TCP/IP, please explain what port is blocked and what is open, be it UDP or TCP protocol. How about an iptable/ipchain rule table for the firewall?

You receive and post news on the same port! I like to see the result of the telnet session from your web server to the nntp server's port 119. Do you know you can post news in the same session.

Now if they filter your tcp contents, then it is a different story, nothing to do with port blocking.

I am here trying to help you, you do not seems to understand my intention, rather, you think I gave you misguided information just for the fun of it. I shall not bother.

From Lunarpages FAQ...

Do you support ssh / telnet access?

NO

Although the c-panel has an ssh/telnet section, it is disabled on all of our servers for security reasons.

please explain what port is blocked and what is open, be it UDP or TCP protocol.

My host has informed me that all outgoing traffic, be it UDP or TCP, on port 119 is blocked.

How about an iptable/ipchain rule table for the firewall?

What about it? Do I look like I am wearing the sysop hat? Do you really think they're going to setup special rules for me?

You receive and post news on the same port!

I am acutely aware of this fact.

I like to see the result of the telnet session from your web server to the nntp server's port 119.

see above.

Do you know you can post news in the same session.

Again, I am aware of this fact.

Now if they filter your tcp contents, then it is a different story, nothing to do with port blocking.

Which is why I am now wondering if they have a global filter on all outgoing NNTP packets.

you do not seems to understand my intention, rather, you think I gave you misguided information just for the fun of it. I shall not bother.

I understand your intention completely. I also understand that you might be as stubborn as a mule. My host writes to me and tells me all outgoing on port 119 has been blocked by their firewall. You seem bent on telling me it's wide open (both ways)... simply on the premise that I am receiving Usenet posts in my forum. IMHO, this is inherently wrong.

[Odoin]

I have not changed useragent... but I have changed organization but that was when I first set it up...

I added another newsgroup and still getting double posting... however the first newsgroup does not do this...

Any ideas?

Originally Posted by lierduh

Have you changed "useragent" or organization? The latest version checks both useragent and organization in the header. The message only get imported if they do not match at the same time.

Otherwise do you have special charactors in the organization setting?