NNTP Gateway (Usenet/Newsgroups)

This hack provides a gateway to any NNTP newsgroup such as those from Usenet and other newsgroup servers. Selected newsgroups are imported from your news server and post on the forum are sent out to the newsgroup. This hack was greatly inspired by fastforward's usenet gateway hack. This one is a complete rewrite of that hack written in PHP and works with vB 2.2.2.

Installation:
Installation is pretty simple. No modifications to the vB php files are needed and it simply consists of dropping in the files to your forums directory, running the SQL modifications, and modifying the newsgroup settings for your desired newsgroups. For the complete installation details, read the readme.txt file.

Here are some of the features:

• Real cool vbcode to text conversion!
• All posts are inserted, even if parent thread cannot be found for replies
• Guests posts also get sent to the newsgroup.
• Importing of attachments from the newsgroup
• Poll info sent to newsgroup
• Attachment link sent to newsgroup
• Supports unlimited newsgroups and servers.
• email notification of new replies, just like regular forum posts
• Can import HTML messages and convert to limited vb code.

For control panel integration, please check out GameCrash's Graphical Interface addition.

What's not in it that was in fastforward's usenet gateway hack:

• No control panel integration at the moment. (Available separately)
• Does not support separate footers
• Does not allow custom email address to be used via the users option
• No spam control or string replacement support.
• No purging or autoexpire.

There is still lots that I'd like to add to it, but this is competely functional for basic newsgroup gateway support. It is most likely not free of bugs. If you have questions, suggestions or annoyances, please post in this thread.

vb3 support
lierduh modified this to provide support for the beta versions of vb3. When the release candidates of vb3 come out, I plan to add support for vb3 and add other features to this hack. If you have suggestions for new features, let me know.

E-mail gateway support
New in this version, you can now gateway with an email list. This will fetch emails from a POP email account and send via php's mail function to the mailing list's email address. To set up the list, in the newsgroup field, enter in the email address to send outgoing posts to. In the server field, enter in the pop3 server, and in the username and password fields enter in, you guessed it, the username and password to your pop account. Note: the email gateway capabilities has not been fully tested, so it might not work with your POP server. Also, it does not work in combo with a news<->forum gateway in the same forum (you can use either kind though in differnet forums within your forums).

[Gilby]

Originally posted by Sanjiyan
Any news on the local user replies, and why they are locking up the gateway.php ie downloads from the usenet server works, yet uploads lockup.

Did you run the SQL update? Also try running it with the showqueries=1 in the URL.

[Gilby]

Originally posted by Erwin Loh
May I congratulate you Gilby by telling you that it works - you just need to use a public newserver that does not require a username or password.

Cool - I am hunting down one with the forums I want right now... but it works!

My two test servers that I used when developing it are "InterNetNews NNRP server INN" and one does require the username and password and it works fine. I think that it still might not work with some other nntp servers. If you have a specific server that it's not working on and you can give me access to play around with that server, I may be able to figure out what's causing the problem.

[Gilby]

Originally posted by GameCrash
1 group(s) gatewayed.
Logging in to news.microsoft.com, group microsoft.public.german.entwickler.dotnet.csharp:
Max: newsgroup
Min: such

Apparently that group does not exist, but the nntp class did not catch it as an error.

[Gilby]

Originally posted by GameCrash
One thing, after import every post is set to the import date, not to the date it has really been posted to the NG. Is it possible to do that or is it a problem (timezones etc.)?

I have the script set to import it as the date of the message if the lastmsg is set to 0. When this is set to 0, you're getting all the posts from the newsgroup. Otherwise, it'll import it and set it as the date of import, as this will make it be ordered correctly and not show up before other posts in the thread.

If you want to change how this works, you can go to line 77 and change the if-else statement that does this. So, you could comment out stuff something like:

			// if ($group[lastmsg] == 0){
				$date = strtotime($message[date]);
			//} else {
				//$date = time();
			//}

[Sanjiyan]

Originally posted by Gilby


Did you run the SQL update? Also try running it with the showqueries=1 in the URL.

I have ran the update yes, and heres the text the server gives me back, it apears to be locking up on the rec.arts.anime.misc group, which is the group where I have put my local post to on my vb server.

ALL SORRY ABOUT THE LENGTH OF THIS POST IT IS VERY LARGE:

The script stops/locksup after it checks the newsgroup just before the the newsgroup to which i have posted to, as you can see the last group checked is: rec.arts.anime.marketplace, which is the newsgroup just before the rec.arts.anime.misc group to which I have posted my reply.

---

Query: SELECT template FROM template WHERE title='options'
Time before: 0.056210041046143
Time after: 0.058135986328125

--------------------------------------------------------------------------------

Query: SELECT user.*,userfield.* FROM user LEFT JOIN userfield ON userfield.userid=user.userid WHERE user.userid='1'
Time before: 0.068742990493774
Time after: 0.070780992507935

--------------------------------------------------------------------------------

Query: INSERT INTO session (sessionhash,userid,host,useragent,lastactivity,lo cation) VALUES ('d1c7dfccf2638fdec0d342fc87b444b0','1','194.117.1 33.180','Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)','1014304767','/gateway.php?showqueries=1')
Time before: 0.08442497253418
Time after: 0.085327982902527

--------------------------------------------------------------------------------


Warning: Cannot add header information - headers already sent by (output started at /usr/home/wocsnet/public_html/global.php:120) in /usr/home/wocsnet/public_html/admin/functions.php on line 1530
Query: SELECT templatesetid,replacementsetid,userselect FROM style WHERE styleid='1'
Time before: 0.086961030960083
Time after: 0.088186979293823

--------------------------------------------------------------------------------

Query: SELECT template,title
FROM template
WHERE (title IN ('gobutton','timezone','username_loggedout','usern ame_loggedin','phpinclude','headinclude','header', 'footer','forumjumpbit','forumjump','nav_linkoff', 'nav_linkon','navbar','nav_joiner','pagenav','page nav_curpage','pagenav_firstlink','pagenav_lastlink ','pagenav_nextlink','pagenav_pagelink','pagenav_p revlink')
AND (templatesetid=-1 OR templatesetid='16'))


ORDER BY templatesetid
Time before: 0.088878035545349
Time after: 0.096379995346069

--------------------------------------------------------------------------------

Query: SELECT * FROM usergroup WHERE usergroupid=6
Time before: 0.10236001014709
Time after: 0.10398197174072

--------------------------------------------------------------------------------

Query: SELECT varname,value FROM nntp_settings
Time before: 0.11269104480743
Time after: 0.11516797542572

--------------------------------------------------------------------------------

Query: SELECT * FROM nntp_groups WHERE enabled=1
Time before: 0.11601901054382
Time after: 0.11823499202728

--------------------------------------------------------------------------------


Warning: Cannot add header information - headers already sent by (output started at /usr/home/wocsnet/public_html/global.php:120) in /usr/home/wocsnet/public_html/gateway.php on line 433
14 group(s) gatewayed.
200 Powered by Typhoon 2.0 -- http://www.highwinds-software.com/ (Typhoon v2.0.0)
381 More Authentication Required
502 Authentication Failed
211 99 3216690 3216800 rec.arts.anime
Logging in to news-text.blueyonder.co.uk, group rec.arts.anime:
Query: SELECT post.postid,post.username,post.userid,post.datelin e,post.attachmentid,post.pagetext,post.showsignatu re,post.ipaddress,thread.pollid,thread.title,post. threadid,thread.forumid FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND post.dateline=thread.dateline AND post.userid=thread.postuserid AND (thread.forumid=59)
Time before: 0.16228294372559
Time after: 0.17449402809143

--------------------------------------------------------------------------------

Query: SELECT post.postid,post.username,post.userid,post.datelin e,post.attachmentid,post.pagetext,post.showsignatu re,post.ipaddress,thread.title,post.threadid,threa d.forumid,thread.msgid,thread.prefix FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND (thread.forumid=59)
Time before: 0.17489802837372
Time after: 0.18513000011444

--------------------------------------------------------------------------------

200 Powered by Typhoon 2.0 -- http://www.highwinds-software.com/ (Typhoon v2.0.0)
381 More Authentication Required
502 Authentication Failed
211 117 3216740 3216874 rec.arts.anime.creative
Logging in to news-text.blueyonder.co.uk, group rec.arts.anime.creative:
Query: SELECT post.postid,post.username,post.userid,post.datelin e,post.attachmentid,post.pagetext,post.showsignatu re,post.ipaddress,thread.pollid,thread.title,post. threadid,thread.forumid FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND post.dateline=thread.dateline AND post.userid=thread.postuserid AND (thread.forumid=60)
Time before: 0.22879302501678
Time after: 0.29553496837616

--------------------------------------------------------------------------------

Query: SELECT post.postid,post.username,post.userid,post.datelin e,post.attachmentid,post.pagetext,post.showsignatu re,post.ipaddress,thread.title,post.threadid,threa d.forumid,thread.msgid,thread.prefix FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND (thread.forumid=60)
Time before: 0.29594302177429
Time after: 0.35159695148468

--------------------------------------------------------------------------------

200 Powered by Typhoon 2.0 -- http://www.highwinds-software.com/ (Typhoon v2.0.0)
381 More Authentication Required
502 Authentication Failed
211 293 3235896 3236281 rec.arts.anime.fandom
Logging in to news-text.blueyonder.co.uk, group rec.arts.anime.fandom:
Query: SELECT post.postid,post.username,post.userid,post.datelin e,post.attachmentid,post.pagetext,post.showsignatu re,post.ipaddress,thread.pollid,thread.title,post. threadid,thread.forumid FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND post.dateline=thread.dateline AND post.userid=thread.postuserid AND (thread.forumid=67)
Time before: 0.40430796146393
Time after: 0.43287003040314

--------------------------------------------------------------------------------

Query: SELECT post.postid,post.username,post.userid,post.datelin e,post.attachmentid,post.pagetext,post.showsignatu re,post.ipaddress,thread.title,post.threadid,threa d.forumid,thread.msgid,thread.prefix FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND (thread.forumid=67)
Time before: 0.43329298496246
Time after: 0.45920097827911

--------------------------------------------------------------------------------

200 Powered by Typhoon 2.0 -- http://www.highwinds-software.com/ (Typhoon v2.0.0)
381 More Authentication Required
502 Authentication Failed
211 15 3215061 3215129 rec.arts.anime.games
Logging in to news-text.blueyonder.co.uk, group rec.arts.anime.games:
Query: SELECT post.postid,post.username,post.userid,post.datelin e,post.attachmentid,post.pagetext,post.showsignatu re,post.ipaddress,thread.pollid,thread.title,post. threadid,thread.forumid FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND post.dateline=thread.dateline AND post.userid=thread.postuserid AND (thread.forumid=68)
Time before: 0.48813498020172
Time after: 0.49104905128479

--------------------------------------------------------------------------------

Query: SELECT post.postid,post.username,post.userid,post.datelin e,post.attachmentid,post.pagetext,post.showsignatu re,post.ipaddress,thread.title,post.threadid,threa d.forumid,thread.msgid,thread.prefix FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND (thread.forumid=68)
Time before: 0.4914870262146
Time after: 0.49426400661469

--------------------------------------------------------------------------------

200 Powered by Typhoon 2.0 -- http://www.highwinds-software.com/ (Typhoon v2.0.0)
381 More Authentication Required
502 Authentication Failed
211 0 2 1 rec.arts.anime.info
Logging in to news-text.blueyonder.co.uk, group rec.arts.anime.info:
Query: SELECT post.postid,post.username,post.userid,post.datelin e,post.attachmentid,post.pagetext,post.showsignatu re,post.ipaddress,thread.pollid,thread.title,post. threadid,thread.forumid FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND post.dateline=thread.dateline AND post.userid=thread.postuserid AND (thread.forumid=69)
Time before: 0.52041101455688
Time after: 0.5222350358963

--------------------------------------------------------------------------------

Query: SELECT post.postid,post.username,post.userid,post.datelin e,post.attachmentid,post.pagetext,post.showsignatu re,post.ipaddress,thread.title,post.threadid,threa d.forumid,thread.msgid,thread.prefix FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND (thread.forumid=69)
Time before: 0.52265894412994
Time after: 0.52438294887543

--------------------------------------------------------------------------------

200 Powered by Typhoon 2.0 -- http://www.highwinds-software.com/ (Typhoon v2.0.0)
381 More Authentication Required
502 Authentication Failed
211 456 3258414 3259022 rec.arts.anime.marketplace
Logging in to news-text.blueyonder.co.uk, group rec.arts.anime.marketplace:
Query: SELECT post.postid,post.username,post.userid,post.datelin e,post.attachmentid,post.pagetext,post.showsignatu re,post.ipaddress,thread.pollid,thread.title,post. threadid,thread.forumid FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND post.dateline=thread.dateline AND post.userid=thread.postuserid AND (thread.forumid=70)

[Mark Hewitt]

Just installed this. (In fact upgraded from fastforwards hack). Everything seems to work just fine from what I can tell.

Haven't tried the outgoing messages yet. Is there a way to disable peoples profiles being printed on them etc, as I don't think my members would be too keen on that.

Also does it support proper threading of messages?

One problem I've noticed straight away is that is doesn't truncate usenet names, meaning the thread starter table is often over long.

In fastforwards hack the quoted text appeared in italics, any way to do that here?

I'm sure I'll have more comments when I have more time to mess with it

[lifesourcerec]

Ok. I installed it and ran gateway,php and it seems to work. Since I am new to this, a) what would be a commandline for crontab and b) anyone know of a public server? I'm on cable access and we just use "news" which isn't really a server.

[Gilby]

Originally posted by Mark Hewitt
Just installed this. (In fact upgraded from fastforwards hack). Everything seems to work just fine from what I can tell.

Haven't tried the outgoing messages yet. Is there a way to disable peoples profiles being printed on them etc, as I don't think my members would be too keen on that.

Yeah, you can. For now you'll have to go into the code and change it. To do that, comment out this line:

$signature .= "$newthread[username]'s Profile: $settings[profileurl]$newthread[userid]\n";

Also does it support proper threading of messages?

Of course it does. I think it matches the new incoming posts with the correct thread better than fastforwards hack did. However, I made i
t where if the user modifies the subject line to be completely different, then a new thread will be created.

One problem I've noticed straight away is that is doesn't truncate usenet names, meaning the thread starter table is often over long.

So on the forumdisplay, they are to long? Or within the thread itself? Can you give me an example of one and how you'd like it chopped down? It can be done where the chopping will just be done on the forumdisplay so that the tables columns don't get smushed, yet in the viewthread, it'll remain the full length.

In fastforwards hack the quoted text appeared in italics, any way to do that here?

Yeah, find this line:

$message[text] = preg_replace("/((\n[ ]*>[^\n]*)+)/","$1",$message[text]);

and replace it with this:

$message[text] = preg_replace("/((\n[ ]*>[^\n]*)+)/","$1",$message[text]);

Or put in whatever other vb code you want there.

vB code with in the code tags gets parsed... so the $1 that is shown in blue up there is [ i ][ color=blue ]$1[ /color ][ /i ], of course that's without the spaces.

[Gilby]

Originally posted by lifesourcerec
a) what would be a commandline for crontab

If you are on a unix or linux server and have shell access, you can make a crontab by putting this line:

5,15,25,35,45,55 * * * * /usr/bin/lynx -source "http://www.yourserver.com/vbdir/gateway.php?log=cron"

in a text file, let's say it's name is crontab.txt, and then type in at the command line:
crontab crontab.txt

That'll set you up with a cron job that'll execute every 10 minutes. You can change the 5,15, etc part to be whatever you want. That corresponds to the minutes of the hours.

[dost]

Originally posted by Gilby
I made it where if the user modifies the subject line to be completely different, then a new thread will be created.

Well, I'd really appreciate it if you could tell me how to make it so that it won't start a new thread when the subject is changed in replies