NNTP Gateway (Usenet/Newsgroups)

This hack provides a gateway to any NNTP newsgroup such as those from Usenet and other newsgroup servers. Selected newsgroups are imported from your news server and post on the forum are sent out to the newsgroup. This hack was greatly inspired by fastforward's usenet gateway hack. This one is a complete rewrite of that hack written in PHP and works with vB 2.2.2.

Installation:
Installation is pretty simple. No modifications to the vB php files are needed and it simply consists of dropping in the files to your forums directory, running the SQL modifications, and modifying the newsgroup settings for your desired newsgroups. For the complete installation details, read the readme.txt file.

Here are some of the features:

• Real cool vbcode to text conversion!
• All posts are inserted, even if parent thread cannot be found for replies
• Guests posts also get sent to the newsgroup.
• Importing of attachments from the newsgroup
• Poll info sent to newsgroup
• Attachment link sent to newsgroup
• Supports unlimited newsgroups and servers.
• email notification of new replies, just like regular forum posts
• Can import HTML messages and convert to limited vb code.

For control panel integration, please check out GameCrash's Graphical Interface addition.

What's not in it that was in fastforward's usenet gateway hack:

• No control panel integration at the moment. (Available separately)
• Does not support separate footers
• Does not allow custom email address to be used via the users option
• No spam control or string replacement support.
• No purging or autoexpire.

There is still lots that I'd like to add to it, but this is competely functional for basic newsgroup gateway support. It is most likely not free of bugs. If you have questions, suggestions or annoyances, please post in this thread.

vb3 support
lierduh modified this to provide support for the beta versions of vb3. When the release candidates of vb3 come out, I plan to add support for vb3 and add other features to this hack. If you have suggestions for new features, let me know.

E-mail gateway support
New in this version, you can now gateway with an email list. This will fetch emails from a POP email account and send via php's mail function to the mailing list's email address. To set up the list, in the newsgroup field, enter in the email address to send outgoing posts to. In the server field, enter in the pop3 server, and in the username and password fields enter in, you guessed it, the username and password to your pop account. Note: the email gateway capabilities has not been fully tested, so it might not work with your POP server. Also, it does not work in combo with a news<->forum gateway in the same forum (you can use either kind though in differnet forums within your forums).

[PhoenixBB]

Might be a good idea to read the man page for "diff" - it might make things a little clearer.

What? What? I just hotfooted it over to the main page and can't find anything. Not that I think it'll make any difference because it still doesn't make sense.

Have I mentioned how much I hate this new layout and these scrolly code boxes? <g> Thank god I got most of my hacks done, because I'm having a very hard time wrapping my head around this. I like the old ways better, least I could understand the instructions.

As the original code started on line 119, then I'm guessing that the first number mentioned is where the replacement code finishes? In this case, very handily at 125? <g> But...but...that still doesn't indicate where one would insert. And 126 and 132 do have code in the original - just different ones. Brackets in this case. Now the original code has been pushed down to 126 and 132 has still got a bracket!

I have a bewildered look on my face at the moment, because I'm just not getting it. Even from your instructions there is no way I would have gotten that the code would have ended up in the middle without seeing it. I actually replaced completely the 'message [from]' section. Whoops.

The 258c265 is a straight replacement (that's what the 'c' means). Remember you added lines just now? The line number was 258 in the original file - it's now line 265.

Now this bit I get! <runs round in excited circles> Thank God! I was fearing early senility for a moment.

But I have a blank line on 258 - the code doesn't start till 260! After inputting the changes it's still the same. 265 is the end of another similar code. Actually it's the same! No mention of this in the instructions, but 264-265 is the same code 260-262 as above. So does that mean I have to do it twice? Ok. Ain't no way I'm getting this from the 'coded' instructions tho.

Hang about....... I'm going thru this again and I might be seeing the light.....I've just been getting cross at this 214,216c221,223 because I don't have the same code at 214. It is now at 221. A-ha! But...... it doesn't end at 223, that's an empty line! It ends at 221! <clutches head>

If this 608,613c615,617 is supposed to mean the original was on 608, nope it's 613 on mine and ends on 618. I just replaced that code with the new one and it now starts at 619 and ends at 624. Sigh.

Well hell, I appear to have made things worse!! I just ran the gateway and a load of code come up, it didn't even try to connect. Was basically the stuff in the gateway file.

I wonder if it was the duplicating of the 258 code for both the 'insert thread' and 'insert post'? Okay I've put the previous file which seemed to work back, and just changed it so I have put the code you showed above for the message reference. Only thing different is I now have the 'message[from]' code back.

Ah, it's working again!

I'm never gonna get this. If all the instructions will be done like this from now on then my days of adding hacks are at an end. Trying to figure out those instructions was a hundred times harder than putting the original hack in.

The Add Reply button does work Allan, it just 'top posts/replies' which is a big no-no in newsgroups. But of course it must do that and there is no way round it. People will just have to get used to using the Quote button in that forum. Unfortunately that doesn't always strip the VB code, so will mean people doing it manually. They won't of course.

And it's still 'top replying' even using the Quote Button! Seems it does that when one replies to a post that originates from the board and not the newsgroup, and it ends up going to the wrong part of the thread in the NG. <whimpers> This is defeating me.

Thanks for showing me where the code goes Allan, ain't no way I'd have figured it out otherwise. I wouldn't have this gateway hack up at all if the instructions had been in this new 'code'.

I need to lie down!

[Holidazed]

Originally Posted by allan grossman

I dunno - I think you'd be getting authentication errrors, not "WTF is this command" errors.

Gilby wrote one version of nntp.php that does extensive logging - I've attached it here. You'll need to rename it to nntp.php after you back up the one you have now. Maybe it'll help -

good luck!

Allan, I used the file you gave me and got the following message when synchronizing the newsgroups.
__________________
1 group(s) gatewayed.
200 NewsRave v1.01 (posting ok) 381 More authentication needed 480 Authentication required for command Logging in to news-west.newscene.com, group rec.games.frp.dnd
__________________

Now, I have the username in the username field and the database shows the password in the password field (even tho the admin interface for this hack does not show the password), but it does not seem to be sending this information.

Question, in your admin interface for this hack, does the password show there or does it remain an empty field like mine does?

[allan grossman]

Originally Posted by PhoenixBB

What? What? I just hotfooted it over to the main page and can't find anything. Not that I think it'll make any difference because it still doesn't make sense.

man page, not main page

From a terminal window type "man diff" (no quotes) and smack the enter key. Use the spacebar to scroll down and the "q" key to quit. Lotsa good stuff there. "diff --help" will give you the short version.

Read this - maybe it'll help you understand things a little better -

http://linux.oreillynet.com/pub/a/linux/lpt/28_01.html

Originally Posted by PhoenixBB

...Thanks for showing me where the code goes Allan, ain't no way I'd have figured it out otherwise. I wouldn't have this gateway hack up at all if the instructions had been in this new 'code'.

I need to lie down!

You're gonna do fine, PhoenixBB. Don't get caught up quite so much in line numbers - as you add and replace text in the file the line numbers are gonna change. My advice would be to use the search function in your favorite text editor and if you're within a few lines of where you oughtta be (and there are no other instances of that particular string in the file) then apply the change.

Remember the output from diff can be a (append), c (change), or d (delete). If it's okay with lierduh maybe I can come up with a set of instructions that are a little easier to follow.

Hang in there - make sure the changes you're contemplating make sense and don't worry too much about line numbers as long as you're close to the target - as the line numbers change as you edit the file

cheers -

allan

[allan grossman]

Originally Posted by bitg

...Question, in your admin interface for this hack, does the password show there or does it remain an empty field like mine does?

The password shows up in my admin interface. Sounds like you're almost there

[allan grossman]

I worked on the instructions a little - maybe they'll be easier now. If lierduh or anybody else objects to my changing the instructions I'll be happy to delete them. I didn't write this hack, have no input on it and can only report on what worked for me - so here goes:

edit: I had the same problem lierduh did. There are a couple places here where the forum software thows a couple of spaces in "allowsmilie" in the text below. If you just paste this stuff in it'll cause database errors so once you're finished editing newreply.php and gateway.php, replace all instances of "allowsmili e" with "allowsmilie". This appears to be a bug in vB3 -

1) Add two columns to post table. Run these queries against your forum database either from a command line or phpMyAdmin or whatever else you have

ALTER TABLE post ADD COLUMN ref TEXT;

ALTER TABLE post ADD COLUMN pre_postid INT(10);

please note that all line numbers were taken from the original files - since we're adding a couple of lines here and there the numbers will be off just a little



2)open forumhome/newreply.php

find (around line 27):

PHP Code:

if (isset($postid)) { 


add under it:

PHP Code:

 if (isset($pre_postid)) { 
   $pre_postid=(int)$pre_postid; 
 } 


find (around line 305):

PHP Code:

       $DB_site->query("INSERT INTO post (postid,threadid,title,username,userid,dateline,attachmentid,pagetext,allowsmilie,showsignature,ipaddress,iconid,visible) VALUES (NULL,'$threadid','".addslashes(htmlspecialchars($title))."','".addslashes($postusername)."','$bbuserinfo[userid]','".time()."','$attachmentid','".addslashes($message)."','$allowsmilie','$signature','$ipaddress','$iconid','$visible')"); 


replace it with:

PHP Code:

       $DB_site->query("INSERT INTO post (postid,threadid,title,username,userid,dateline,attachmentid,pagetext,allowsmilie,showsignature,ipaddress,iconid,visible,pre_postid) VALUES (NULL,'$threadid','".addslashes(htmlspecialchars($title))."','".addslashes($postusername)."','$bbuserinfo[userid]','".time()."','$attachmentid','".addslashes($message)."','$allowsmilie','$signature','$ipaddress','$iconid','$visible','$pre_postid')"); 


save newreply.php



3) Modify template: newreply

find:

PHP Code:

<input type="hidden" name="threadid" value="$threadid"> 


add under it:

PHP Code:

<input type="hidden" name="pre_postid" value="$postid"> 


4) Modify gateway.php

find (around line 19):

PHP Code:

header("Content-Type: text/plain");
include("global.php");

// Load NNTP classes
include("nntp.php");
// POP3 class
include("class.POP3.php3");
// Load Mime message parser
include("mime.php"); 


replace it with

PHP Code:

header("Content-Type: text/plain");
require_once("global.php");

// Load NNTP classes
require_once("nntp.php");
// POP3 class
//require_once("class.POP3.php3");
// Load Mime message parser
require_once("mime.php"); 


find (around line 66):

PHP Code:

       $get_newthreads=$DB_site->query("SELECT post.postid,post.username,post.userid,post.dateline,post.attachmentid,post.pagetext,post.showsignature,post.ipaddress,thread.pollid,thread.title,post.threadid,thread.forumid FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND post.dateline=thread.dateline AND post.userid=thread.postuserid AND (thread.forumid=". implode(" OR thread.forumid=", $group[forum]) . ")"); 


replace it with:

PHP Code:

       $get_newthreads=$DB_site->query("SELECT post.postid,post.username,post.userid,post.dateline,post.attachmentid,post.pagetext,post.showsignature,post.ipaddress,thread.pollid,thread.title,post.threadid,thread.forumid,post.pre_postid FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND post.dateline=thread.dateline AND post.userid=thread.postuserid AND (thread.forumid=". implode(" OR thread.forumid=", $group[forum]) . ")"); 


find (around line 74):

PHP Code:

       $get_newposts=$DB_site->query("SELECT post.postid,post.username,post.userid,post.dateline,post.attachmentid,post.pagetext,post.showsignature,post.ipaddress,thread.title,post.threadid,thread.forumid,thread.msgid,thread.prefix FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND (thread.forumid=". implode(" OR thread.forumid=", $group[forum]) . ")"); 


replace it with:

PHP Code:

        $get_newposts=$DB_site->query("SELECT post.postid,post.username,post.userid,post.dateline,post.attachmentid,post.pagetext,post.showsignature,post.ipaddress,thread.title,post.threadid,thread.forumid,thread.msgid,thread.prefix,post.pre_postid FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND (thread.forumid=". implode(" OR thread.forumid=", $group[forum]) . ")"); 


find (around line 119):

PHP Code:

                if ($message[from]){
                    $elements=imap_mime_header_decode($message[from]);
                    $message[from] = '';
                    for($i=0;$i<count($elements);$i++) {
                        $message[from] .= $elements[$i]->text;
                    }
                }
            } 


add under it:

PHP Code:

                               if ($message[references]){ 
                                       $elements=imap_mime_header_decode($message[references]); 
                                       $message[references] = ''; 
                                       for($i=0;$i<count($elements);$i++) { 
                                           $message[references] .= $elements[$i]->text; 
                                       } 
                               } 


find (around line 214):

PHP Code:

                                       } 
                                       $DB_site->query("INSERT INTO post (postid,threadid,title,username,userid,dateline,attachmentid,pagetext,allowsmilie,showsignature,ipaddress,iconid,visible,isusenetpost,msgid) VALUES (NULL,'$threadid','".addslashes(htmlspecialchars($message[subject]))."','".addslashes(htmlspecialchars(from_name($message[from])))."','0','".$date."','$attachmentid','".addslashes($message[text])."','1','0','".addslashes(htmlspecialchars(from_email($message[from])))."','0','1','1','".addslashes($message[msgid])."')"); 
                                       $postid=$DB_site->insert_id(); 


replace it with:

PHP Code:

                                       } 
                                         $DB_site->query("INSERT INTO post (postid,threadid,title,username,userid,dateline,attachmentid,pagetext,allowsmilie,showsignature,ipaddress,iconid,visible,isusenetpost,msgid,ref) VALUES (NULL,'$threadid','".addslashes(htmlspecialchars($message[subject]))."','".addslashes(htmlspecialchars(from_name($message[from])))."','0','".$date."','$attachmentid','".addslashes($message[text])."','1','0','".addslashes(htmlspecialchars(from_email($message[from])))."','0','1','1','".addslashes($message[msgid])."','".addslashes($message[references])."')"); 
                                         $postid=$DB_site->insert_id(); 


find (around line 258):

PHP Code:

                                       $DB_site->query("INSERT INTO post (postid,threadid,title,username,userid,dateline,attachmentid,pagetext,allowsmili  e,showsignature,ipaddress,iconid,visible,isusenetpost,msgid) VALUES (NULL,'$threadid','".addslashes(htmlspecialchars($message[subject]))."','".addslashes(from_name($message[from]))."','0','".$date."','$attachmentid','".addslashes($message[text])."','1','0','".addslashes(from_email($message[from]))."','0','1','1','".addslashes($message[msgid])."')"); 


replace it with:

PHP Code:

                                       $DB_site->query("INSERT INTO post (postid,threadid,title,username,userid,dateline,attachmentid,pagetext,allowsmili e,showsignature,ipaddress,iconid,visible,isusenetpost,msgid,ref) VALUES (NULL,'$threadid','".addslashes(htmlspecialchars($message[subject]))."','".addslashes(from_name($message[from]))."','0','".$date."','$attachmentid','".addslashes($message[text])."','1','0','".addslashes(from_email($message[from]))."','0','1','1','".addslashes($message[msgid])."','".addslashes($message[references])."')"); 


find (around line 608):

PHP Code:

                       $get_attachinfo=$DB_site->query("SELECT msgid FROM post WHERE threadid=$newthread[threadid]"); 
                       $ref = ''; 
                       while ($thisref=$DB_site->fetch_array($get_attachinfo)){ 
                               if ($thisref[msgid] and strlen($ref) < 600){ $ref .= "$thisref[msgid] "; } 
                       } 
                       $ref = "\r\nReferences: ".wordwrap($ref, 210, "\r\n        "); 


replace it with:

PHP Code:

                       $get_references=$DB_site->query_first("SELECT msgid,ref FROM post WHERE postid=$newthread[pre_postid]"); 
                       $references = $get_references[ref] . " " . $get_references[msgid]; 
                       $ref = "\r\nReferences: ".wordwrap(stripslashes($references), 210, "\r\n        "); 


find (around line 629):

PHP Code:

                       $DB_site->query("UPDATE post SET isusenetpost = -1, msgid = '<".addslashes($msgid).">' WHERE postid=$newthread[postid]"); 


replace it with:

PHP Code:

                       $DB_site->query("UPDATE post SET isusenetpost =1, msgid ='<".addslashes($msgid).">' , ref = '".$references."' WHERE postid=$newthread[postid]"); 


[Holidazed]

Originally Posted by allan grossman

The password shows up in my admin interface. Sounds like you're almost there

OK, as I stated earlier, my password is not in the password field in my CP, however, it is there in the database. How do I get it to stay in the CP area?

[allan grossman]

Originally Posted by bitg

OK, as I stated earlier, my password is not in the password field in my CP, however, it is there in the database. How do I get it to stay in the CP area?

I guess I'd be looking at the graphical interface hack, bitg - I'm at the office and can't look at mine but I had no problem with the password staying in the CP. My first guess would be a small glitch in the way the CP hack was applied but I could be way off-base here

[Holidazed]

Originally Posted by allan grossman

I guess I'd be looking at the graphical interface hack, bitg - I'm at the office and can't look at mine but I had no problem with the password staying in the CP. My first guess would be a small glitch in the way the CP hack was applied but I could be way off-base here

I just completely uninstalled the graphical interface hack as well as the nntp hack. Then I installed just the nntp hack (no graphical interface). I then configured all the fields in the database.

When I go to http://www.dungeonkeepersdomain.com/mb-test/gateway.php I get the following error:

___________________
1 group(s) gatewayed.
200 NewsRave v1.01 (posting ok)
381 More authentication needed
480 Authentication required for command
Logging in to news-west.newscene.com, group rec.games.frp.dnd
480 Authentication required for command
500 Huh?
Posting Message from DK:
500 Huh?
500 Huh?
500 Huh?
Posting Message from DK:
500 Huh?
___________________

Again, I know that the username, password, server and newsgroup are correct. There is no graphical interface installed. The username and password are showing up just fine in the database.

This happens when I place a message in the newsgroup forum and then run gateway.php. When I have no posts pending in the newsgroup forum, I get no errors, however, I get no incoming posts either. I deliberately did not run setlastmsg.php so that I would get plenty of message.

The wierd part is as follows. If I use a news server that is free and does not require a username and password (often posting rights are removed), I get tons of new messages but cannot post (for obvious reasons). The problem has to be that the username and/or password is not being sent.

ideas?

[allan grossman]

I might have an idea.

I was just thinking that the fact that the password field didn't show up in the graphical interface *and* it isn't being passed to the news server but shows up in phpMyAdmin (I guess) would lead me to believe maybe the password field is either misspelled, the wrong type or in the wrong location.

The table type for nntp_groups should be ISAM, not MyISAM. It won't work if the table type is wrong (I don't know why, but I tried it the other way and the script fails).

If the table type is incorrect you can fix it by running this query against the database -

ALTER TABLE nntp_groups TYPE = ISAM;

Inside the nntp_groups table the password field is spelled "password" and is case-sensitive.

If it's not a spelling error, a typo or a misconfigured table I'm stumped, bitg - but I think the fact that the password isn't being sent and *also* isn't showing up in the graphical interface is significant.

[Holidazed]

Well, I don't know which suggestion is was, but it is working now.

The password field was named PASSWORD, I changed it to password.