NNTP Gateway (Usenet/Newsgroups)

This hack provides a gateway to any NNTP newsgroup such as those from Usenet and other newsgroup servers. Selected newsgroups are imported from your news server and post on the forum are sent out to the newsgroup. This hack was greatly inspired by fastforward's usenet gateway hack. This one is a complete rewrite of that hack written in PHP and works with vB 2.2.2.

Installation:
Installation is pretty simple. No modifications to the vB php files are needed and it simply consists of dropping in the files to your forums directory, running the SQL modifications, and modifying the newsgroup settings for your desired newsgroups. For the complete installation details, read the readme.txt file.

Here are some of the features:

• Real cool vbcode to text conversion!
• All posts are inserted, even if parent thread cannot be found for replies
• Guests posts also get sent to the newsgroup.
• Importing of attachments from the newsgroup
• Poll info sent to newsgroup
• Attachment link sent to newsgroup
• Supports unlimited newsgroups and servers.
• email notification of new replies, just like regular forum posts
• Can import HTML messages and convert to limited vb code.

For control panel integration, please check out GameCrash's Graphical Interface addition.

What's not in it that was in fastforward's usenet gateway hack:

• No control panel integration at the moment. (Available separately)
• Does not support separate footers
• Does not allow custom email address to be used via the users option
• No spam control or string replacement support.
• No purging or autoexpire.

There is still lots that I'd like to add to it, but this is competely functional for basic newsgroup gateway support. It is most likely not free of bugs. If you have questions, suggestions or annoyances, please post in this thread.

vb3 support
lierduh modified this to provide support for the beta versions of vb3. When the release candidates of vb3 come out, I plan to add support for vb3 and add other features to this hack. If you have suggestions for new features, let me know.

E-mail gateway support
New in this version, you can now gateway with an email list. This will fetch emails from a POP email account and send via php's mail function to the mailing list's email address. To set up the list, in the newsgroup field, enter in the email address to send outgoing posts to. In the server field, enter in the pop3 server, and in the username and password fields enter in, you guessed it, the username and password to your pop account. Note: the email gateway capabilities has not been fully tested, so it might not work with your POP server. Also, it does not work in combo with a news<->forum gateway in the same forum (you can use either kind though in differnet forums within your forums).

[hubba]

>p?NG postings

Uff...

On the forumhome "new posts" and "new themes" counting includes the Newsgroupposts, that I would like to exclude, so that only "real" forumposts are added...

[allan grossman]

Originally Posted by lierduh

I made a stupid mistake at the last minute. Please download the gateway.php again. I have attached a "a" to the end of the file name. The only thing changed was two lines in the gateway.php script. There is no need to replace the other files.

So all the joy I had trying to get this to work last night has been addressed?

Thanks, lierduh - I really appreciate all your hard work.

[Imhotep]

I think I am still having issues with VB posts going out to Usenet. I'm giving it a bit of time... for the post to propagate... but it still looks like they're not showing up. This is for new posts and replies.

hmmmm...

I bascially dropped my NNTP SQL tables and installed them fresh from the new ZIP file. I also replaced the PHP files.

[hubba]

installed the updated script. Now MySQL error:

UPDATE post AS post, thread AS thread SET isusenetpost = 1 WHERE
post.threadid = thread.threadid AND
isusenetpost = 0 AND
thread.forumid NOT IN (200,201,202)

mysql error: Column: 'isusenetpost' in field list is ambiguous

[Imhotep]

oh buggar! now I have an even bigger problem. This is from my host...

Hello,

I have disabled your crontab which ran the "gateway.php" script. This crontab
was causing load issues on the server every 5 minutes, which is when the
script was being run. This is far too resource intensive to be run on a shared
server at such a short interval. Please do not re-enable this script as it
will continue to cause issues on the server if left to run.

:ermm:

Is there any way to make this script less system intensive?

[allan grossman]

Originally Posted by Imhotep

oh buggar! now I have an even bigger problem. This is from my host...



:ermm:

Is there any way to make this script less system intensive?

Run it less often

I run mine every 15 minutes. Once the newsgroups are populated the script doesn't beat hell out of the server.

[hubba]

Got it. Just renamed all instances of "isusenetpost" in gateway.php to "isusenetp" and the sql field to "isusenetp".

Edit...

Now all seems to be OK, the script shows "240 posting OK" but no new postings are made in the forum...

[hubba]

Again got it just deleted all posts in the forum, run the gateway and it works

[Imhotep]

Originally Posted by allan grossman

Run it less often

I run mine every 15 minutes. Once the newsgroups are populated the script doesn't beat hell out of the server.

Well bollocks! Lunarpages just told me to not run the script ever again. :ermm:

I would suggest that you not enable the script again, as advised by the Tech.

I can't imagine it putting that much load on the server... especially if it only collects new messages.

this sucks...

[lierduh]

Originally Posted by hubba

installed the updated script. Now MySQL error:

UPDATE post AS post, thread AS thread SET isusenetpost = 1 WHERE
post.threadid = thread.threadid AND
isusenetpost = 0 AND
thread.forumid NOT IN (200,201,202)

mysql error: Column: 'isusenetpost' in field list is ambiguous

Check if your thread table has an 'isusenetpost' field. There shouldn't be one in the thread table. Only in the post table.

[hubba]

Yes that was the error, a double field. I thought about that but couldnt find it. Thanks!

[lierduh]

Originally Posted by Imhotep

oh buggar! now I have an even bigger problem. This is from my host...



:ermm:

Is there any way to make this script less system intensive?

I run each 10 minutes. It takes a few seconds to run each time. If you were using the old script and run every 5 minutes. Then if the script is called before the first one is finished. You can push the server over the edge! The new script prevents this. The new script is less system intensive.

Depending on how many messages stored on the server, the first time the script runs, it can take a long time to import all the messages. You should be able to set the last message number to a high one. That way you ignore all the older messages, only import the new ones. To find out the current message number, open the news post using a normal browser, look for Xref in the headers.

Xref: freenews.iinet.net.au aus.tv.pay:132041

The 132041 is the message number.

Check your current message setting in the nntp_groups table against what is on the server, and set the lastmsg for that group to the latest message number. Run the script manually, keep an eye on the server load. If you have shell access, type "top" should show. After you are sure that your script is not a resource hog, email the support nicely and explain to them you have fixed the problem.

(The next version will be even less server intensive.)

[lierduh]

Originally Posted by hubba

>p?NG postings

Uff...

On the forumhome "new posts" and "new themes" counting includes the Newsgroupposts, that I would like to exclude, so that only "real" forumposts are added...

Still in the dark.

I searched my forum's home page for the word "theme", not there. I then searched for "post", nothing is associated with a number.

[memobug]

Originally Posted by fsfl2k

I used the one posted earlier in the tread and it worked fine. Here it is again.

newsgroup: webdiscuss.test
server: webdiscuss.com
forum: a blank test forum id

Anthony

Greetings,

Can anyone confirm whether the test newsgroup list described above (here is still running? The original parameters were these:

newsgroup= webdiscuss.test
server= free.teranews.com

I have the c-imap library installed and my web services provider opened TCP port 119, but I get a notepad window popping up which says

1 group(s) gatewayed.
Logging in to free.teranews.com, group webdiscuss.test

and that's it - nothing gets imported into my designated empty forum. It seems a little further along than before they opened the port, when it used to say
"1 group(s) gatewayed.
Could not connect to NNTP-serverLogging
NotConnected Not Connected."

So, further along, but no joy!

Regards,

Matt

EDIT: To try uploading I put a test message in the forum and got this result in a popup

Posting Message from Username: 501 Unknown command: From: Username <Username.1666kd@timelimit.yourserver.com>
Posting Message from Username: 501 Unknown command: Newsgroups: webdiscuss.test

[allan grossman]

Originally Posted by memobug

Greetings,

Can anyone confirm whether the test newsgroup list described above (here is still running? The original parameters were these:

newsgroup= webdiscuss.test
server= free.teranews.com

I have the c-imap library installed and my web services provider opened TCP port 119, but I get a notepad window popping up which says

1 group(s) gatewayed.
Logging in to free.teranews.com, group webdiscuss.test

and that's it - nothing gets imported into my designated empty forum. It seems a little further along than before they opened the port, when it used to say
"1 group(s) gatewayed.
Could not connect to NNTP-serverLogging
NotConnected Not Connected."

So, further along, but no joy!

Regards,

Matt

memobug, maybe you can try using the nntp.php that's on page 24 of this thread? It'll do extended logging and maybe we can figure out what's what.