NNTP Gateway (Usenet/Newsgroups)

This hack provides a gateway to any NNTP newsgroup such as those from Usenet and other newsgroup servers. Selected newsgroups are imported from your news server and post on the forum are sent out to the newsgroup. This hack was greatly inspired by fastforward's usenet gateway hack. This one is a complete rewrite of that hack written in PHP and works with vB 2.2.2.

Installation:
Installation is pretty simple. No modifications to the vB php files are needed and it simply consists of dropping in the files to your forums directory, running the SQL modifications, and modifying the newsgroup settings for your desired newsgroups. For the complete installation details, read the readme.txt file.

Here are some of the features:

• Real cool vbcode to text conversion!
• All posts are inserted, even if parent thread cannot be found for replies
• Guests posts also get sent to the newsgroup.
• Importing of attachments from the newsgroup
• Poll info sent to newsgroup
• Attachment link sent to newsgroup
• Supports unlimited newsgroups and servers.
• email notification of new replies, just like regular forum posts
• Can import HTML messages and convert to limited vb code.

For control panel integration, please check out GameCrash's Graphical Interface addition.

What's not in it that was in fastforward's usenet gateway hack:

• No control panel integration at the moment. (Available separately)
• Does not support separate footers
• Does not allow custom email address to be used via the users option
• No spam control or string replacement support.
• No purging or autoexpire.

There is still lots that I'd like to add to it, but this is competely functional for basic newsgroup gateway support. It is most likely not free of bugs. If you have questions, suggestions or annoyances, please post in this thread.

vb3 support
lierduh modified this to provide support for the beta versions of vb3. When the release candidates of vb3 come out, I plan to add support for vb3 and add other features to this hack. If you have suggestions for new features, let me know.

E-mail gateway support
New in this version, you can now gateway with an email list. This will fetch emails from a POP email account and send via php's mail function to the mailing list's email address. To set up the list, in the newsgroup field, enter in the email address to send outgoing posts to. In the server field, enter in the pop3 server, and in the username and password fields enter in, you guessed it, the username and password to your pop account. Note: the email gateway capabilities has not been fully tested, so it might not work with your POP server. Also, it does not work in combo with a news<->forum gateway in the same forum (you can use either kind though in differnet forums within your forums).

[Gardener]

This is the first hack I have tried to install for VB, and I love it. The install went well with very few problems, except for one of the news groups I added. I added alt.binaries.pictures.gardens as one of the groups, and most of the pictures just show up as a page full of characters instead of the actual picture. I don't think this is a script problem, but rather a problem of not having something installed in either Apache or PHP to decode these pictures. Can someone give me a hand.

Thanks

[Gilby]

Originally posted by Gardener
I don't think this is a script problem, but rather a problem of not having something installed in either Apache or PHP to decode these pictures. Can someone give me a hand.

This is an incompatibility with script where it does not support attachments that are not mime attachments and are uuencoded, which is what OE does for attachments.

[Gardener]

Thanks. I noticed that some of the pictures do show up as attachments, but most don't. I'll just have to remove that group. Thanks for your help.

[Gardener]

Is there any way to attach a footer to the end of postings made from my site and posted to usenet? Such as "Posted via my website".

[Gardener]

Originally posted by Gardener
Is there any way to attach a footer to the end of postings made from my site and posted to usenet? Such as "Posted via my website".

Anyone?

[Gilby]

Originally posted by Gardener
Is there any way to attach a footer to the end of postings made from my site and posted to usenet? Such as "Posted via my website".

Find the lines (around line 550) that start with "$signature .= " and change those to how you want the signature to be.

[Gardener]

Thanks for the help. I had almost forgot about asking this.

[MaXxed]

I have two issues with this and would welcome some help

1. Using the graphical interface, I cant seem to move a newsgroup (rec.travel.australia+nz) - I think this may be due to the '+' in the name. When clicking on the groups settings, I just get blank fields.

2. How does this hack handle outgoing attachments? I have them disabled by default as I am using text groups, but I am curious what would happen.

Thanks for any/all help.
Max.

[SRozhon]

I am getting this after manual sync. what should I do?

1 group(s) gatewayed.
Could not connect to NNTP-serverLogging in to netnews.insightbb.com, group comp.graphics.apps.photoshop
Not connectedNot connected

[Gardener]

Gilby,

I posted before aobut trying to use this hack to download some biniary groups, but I was having trouble with the uuencoded attachments. I've been looking for a way to solve this, and someone sent me here: http://www.php.net/manual/en/function.base64-decode.php

Would this work? I'm willing to pay for this if you can get it working. PM me with the details.

Thanks

[qnx]

I am testing the NNTP gateway to the public news server at inn.qnx.com and trying to pull all articles from the group "qdn.cafe" (less than 600 posts). it always fails after message number 562:

...
Getting message number 561: Thread found; 'Re: This is great....' from Bill Caroselli \(Q-TPS\)
Getting message number 562: Thread found; 'Re: This is great....' from Mario Charest

Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to allocate 10 bytes) in /var/www/html/forums/admin/functions.php on line 1318

Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to allocate 11 bytes) in Unknown on line 0
...

the php.ini has the default setup of 8M memroy. and I have installed the gui interface from GameCrash.

the posts in that newsgroup are all less than 1M. does the gateway code NOT release the memory after pulling each article? this means we will run out of 8M php allowed memory if we have many posts?
Is there a way to fix it without increasing the 8M limit?

Thanks!
Frank

[qnx]

Now it's getting worse!
I doubled memory_limit to 16M in php.ini and try to hit gateway.php again, this time it gets a few more posts:
...
Getting message number 568: Thread found; 'Re: This is great....' from Eric Norton
Getting message number 569: Thread found; 'Re: This is great....' from Mario Charest
<br>
<b>Fatal error</b>: Allowed memory size of 16777216 bytes exhausted (tried to allocate 644127 bytes) in <b>/var/www/html/forums/admin/functions.php</b> on line <b>1330</b><br>
<br>
<b>Fatal error</b>: Allowed memory size of 16777216 bytes exhausted (tried to allocate 0 bytes) in <b>Unknown</b> on line <b>0</b><br>

The worst part is now if I go to the forums main page, I see all the duplicates!! it seems gateway.php doesn't know that it has already pulled some of the posts. It gets ALL posts again! Doesn't gateway.php updates the "last_post" count after successfully pulled ONE post? it seems gateway.php only updates the "last_post" count when it finishes polling all posts. If it fails (in my cases), it lost the opportunity to update the "last_post count.

Frank

[qnx]

In addition to the "last_post" count, I suggest we use the message-id from the post as a unique id to avoid duplicates. before we insert a new post, we do a select on msgid to make sure it is NOT a duplicate. if we index the msgid, this overhead should be trivial but the benefit is huge. You can import multiple times without worrying about duplicates. btw, we should probably raise the msgid to varchar(255) which is the largest that we can index I believe. we can have better unique-ness with 255 chars I hope.

Frank

[MaXxed]

Is there a problem with subscribing to a usenet thread and getting email notifications of replies? It doesnt seem to be working for me?

[ryancooper]

Hey Guys; great hack!
I got it to work with a test server but when I try to use my ISP's I get this and nothing else:

1 group(s) gatewayed.
Logging in to news.epix.net, group rec.arts.disney.parks

can this mean I am not getting into my server? I have my user name and password set up.