NNTP Gateway (Usenet/Newsgroups)

This hack provides a gateway to any NNTP newsgroup such as those from Usenet and other newsgroup servers. Selected newsgroups are imported from your news server and post on the forum are sent out to the newsgroup. This hack was greatly inspired by fastforward's usenet gateway hack. This one is a complete rewrite of that hack written in PHP and works with vB 2.2.2.

Installation:
Installation is pretty simple. No modifications to the vB php files are needed and it simply consists of dropping in the files to your forums directory, running the SQL modifications, and modifying the newsgroup settings for your desired newsgroups. For the complete installation details, read the readme.txt file.

Here are some of the features:

• Real cool vbcode to text conversion!
• All posts are inserted, even if parent thread cannot be found for replies
• Guests posts also get sent to the newsgroup.
• Importing of attachments from the newsgroup
• Poll info sent to newsgroup
• Attachment link sent to newsgroup
• Supports unlimited newsgroups and servers.
• email notification of new replies, just like regular forum posts
• Can import HTML messages and convert to limited vb code.

For control panel integration, please check out GameCrash's Graphical Interface addition.

What's not in it that was in fastforward's usenet gateway hack:

• No control panel integration at the moment. (Available separately)
• Does not support separate footers
• Does not allow custom email address to be used via the users option
• No spam control or string replacement support.
• No purging or autoexpire.

There is still lots that I'd like to add to it, but this is competely functional for basic newsgroup gateway support. It is most likely not free of bugs. If you have questions, suggestions or annoyances, please post in this thread.

vb3 support
lierduh modified this to provide support for the beta versions of vb3. When the release candidates of vb3 come out, I plan to add support for vb3 and add other features to this hack. If you have suggestions for new features, let me know.

E-mail gateway support
New in this version, you can now gateway with an email list. This will fetch emails from a POP email account and send via php's mail function to the mailing list's email address. To set up the list, in the newsgroup field, enter in the email address to send outgoing posts to. In the server field, enter in the pop3 server, and in the username and password fields enter in, you guessed it, the username and password to your pop account. Note: the email gateway capabilities has not been fully tested, so it might not work with your POP server. Also, it does not work in combo with a news<->forum gateway in the same forum (you can use either kind though in differnet forums within your forums).

[memobug]

Originally Posted by memobug

Thanks allan for the suggestion and lierduh for the test. I really appreciate the help. It took me nearly a month of haggling to get that port opened!

I installed the debug version of nntp that allan suggests and got this


So I guess this is consistent with what lierduh reports in his trial. Is there another mail server to try before I subscribe to a news service or do my results suggest I am getting far enough along for a reasonable chance of success?

Any recommendation on which news service I should go with when I subscribe? I'm looking to bring in rec.arts.bonsai , which is a text-based service. My three main requirements are cheap, cheap and you-guessed-it.

The email mod built into gateway - for posting - seemed to be generating errors on my server, so I commented it out as recommended in another post here.

Regards,

Matt

Hmmm. I applied for one of those $2.95, five-day Newsranger evaluations and tried to pull in posts from rec.arts.bonsai for the past couple of days.

I got this message:

1 group(s) gatewayed.
502 news: Transfer permission denied to (my ISP's machine's numerical IP appeared here)
Logging in to www.newsranger.com, group rec.arts.bonsai

UPDATE: this problem was corrected by using client.newsranger.com instead of www.newsranger.com so things are looking good now!
Regards,

Matt

[memobug]

Things are looking good now, but I have a question pertaining to follow-up gateway.php synchronization. I want to be careful not to import the same records or export them!

A) Do I run setlastmessage.php

1) ...before every gateway import
2) ...after every gateway import

and

B) Do I run it only once or every time I run gateway?

The Readme isn't clear. It says:

If you want to automatically set the newsgroups to only import new messages, upload the setlastmsg.php to your vbulletin folder and run it.

Looking at the code, it appears I need to run it AFTER gateway and possibly EVERY time to keep the lastmsg counter updated.

Thanks,

Matt

[lierduh]

Originally Posted by memobug

A) Do I run setlastmessage.php

Matt

That is only for people who do not want to import all the existing messages on the server. So run it once only at the installation if needed. Don't blame no posts are imported if you do decided to run. :nervous:

[lierduh]

Originally Posted by hubba

>themes

Boy ME and my english sorry. I mean THREADS... I have the "total new threads since last visit" and "total new postings since last visit" link in the forum. Excluded the NG Forums, that works fine, they dont show up if one searches for new postings, but the NG posts still COUNT. So, i.e. no "real" new posts in the forum, but 3 new in the NG forum = "3 new postins since last visit". I just wanna make a "Totalposts" = xxx - NGposts = realposts.

Sorry, we are running different vBulletin. :surprised: I don't have these links.

[hubba]

>different

Maybe I mean the $newposts[count] und $activethreads[count] variables. They count ALL including the NG posts. The exclude statement only shows them not up, but counts it....

[himerus]

Lierduh,

There is a BIG error in your code in function sendnews.

PHP Code:

if (!$isreply){
   $DB_site->query("UPDATE " . TABLE_PREFIX . "thread 
     SET msgid = '<".addslashes($msgid).">', 
     prefix = '".$prefix[$newthread['forumid']]."'");
} 


There is No WHERE clause in the query.

After using this, EVERY thread in my DB has been updated with the new msgid from the latest new thread created in the newsgroup forums.

Any way to fix this? By fixing this, I mean, resetting all the msgid's to the correct ones, or removing values where they don't belong in the thread table?

[himerus]

Corrected code:

PHP Code:

if (!$isreply){
   $DB_site->query("UPDATE " . TABLE_PREFIX . "thread 
     SET msgid = '<".addslashes($msgid).">', 
     prefix = '".$prefix[$newthread['forumid']]."'
     WHERE threadid={$newthread['threadid']}");
} 


If you've got any suggestions on a quick script to repair those corrupted fields, I would be greatly appreciative. I'm not sure what effect this error, and now having a incorrect msgid on EVERY thread in the DB might or might not cause. I'm not even sure what effect that field has in general to future postings. Could I just reset all those values to empty?

[allan grossman]

Got a little trouble here too, lierduh

Database error in vBulletin 3.0.1:

Invalid SQL: UPDATE post AS post, thread AS thread SET isusenetpost = 1 WHERE 
	post.threadid = thread.threadid AND 
	isusenetpost = 0 AND
	thread.forumid NOT IN (25,27,29,34)
mysql error: You have an error in your SQL syntax near 'AS post, thread AS thread SET isusenetpost = 1 WHERE 
	post.threadid = thread.th' at line 1

mysql error number: 1064

Date: Thursday 13th of May 2004 04:24:04 PM

Any thoughts?

[himerus]

Mine's also not assigning posts to the proper thread after this change. I'm going to look over the code again for another hour or so, and revert to my last version if I can't get it working properly.

Messed up part is, the posting via the vB adminCP scheduled tasks worked after only minimal tweaking of the new script, and I have it tested to successfully post properly. (which was my only problem before)

I think the main problem is coming from the references. They don't seem to be assigning at all anymore.

[lierduh]

No need to worry, although thank you very much for pointing out.

the msgid in the thread table is not actually used. So it does not cause any problem at all. You can safely comment this section out. I have got a new version ready to be uploaded including an optional "X-No-Archive" header honouring. Allan will love this.

I am thinking of getting rid of this field altogether. Saving a field or index is always a good thing.

(by the way, the error is inherited from the original script)

Originally Posted by himerus

Lierduh,

PHP Code:

if (!$isreply){
   $DB_site->query("UPDATE " . TABLE_PREFIX . "thread 
     SET msgid = '<".addslashes($msgid).">', 
     prefix = '".$prefix[$newthread['forumid']]."'");
}