NNTP Gateway (Usenet/Newsgroups)

This hack provides a gateway to any NNTP newsgroup such as those from Usenet and other newsgroup servers. Selected newsgroups are imported from your news server and post on the forum are sent out to the newsgroup. This hack was greatly inspired by fastforward's usenet gateway hack. This one is a complete rewrite of that hack written in PHP and works with vB 2.2.2.

Installation:
Installation is pretty simple. No modifications to the vB php files are needed and it simply consists of dropping in the files to your forums directory, running the SQL modifications, and modifying the newsgroup settings for your desired newsgroups. For the complete installation details, read the readme.txt file.

Here are some of the features:

• Real cool vbcode to text conversion!
• All posts are inserted, even if parent thread cannot be found for replies
• Guests posts also get sent to the newsgroup.
• Importing of attachments from the newsgroup
• Poll info sent to newsgroup
• Attachment link sent to newsgroup
• Supports unlimited newsgroups and servers.
• email notification of new replies, just like regular forum posts
• Can import HTML messages and convert to limited vb code.

For control panel integration, please check out GameCrash's Graphical Interface addition.

What's not in it that was in fastforward's usenet gateway hack:

• No control panel integration at the moment. (Available separately)
• Does not support separate footers
• Does not allow custom email address to be used via the users option
• No spam control or string replacement support.
• No purging or autoexpire.

There is still lots that I'd like to add to it, but this is competely functional for basic newsgroup gateway support. It is most likely not free of bugs. If you have questions, suggestions or annoyances, please post in this thread.

vb3 support
lierduh modified this to provide support for the beta versions of vb3. When the release candidates of vb3 come out, I plan to add support for vb3 and add other features to this hack. If you have suggestions for new features, let me know.

E-mail gateway support
New in this version, you can now gateway with an email list. This will fetch emails from a POP email account and send via php's mail function to the mailing list's email address. To set up the list, in the newsgroup field, enter in the email address to send outgoing posts to. In the server field, enter in the pop3 server, and in the username and password fields enter in, you guessed it, the username and password to your pop account. Note: the email gateway capabilities has not been fully tested, so it might not work with your POP server. Also, it does not work in combo with a news<->forum gateway in the same forum (you can use either kind though in differnet forums within your forums).

[Gilby]

Originally posted by patvdv
Gilby, yes I am (also) using 'Mass Prune' from the AdminCP and the behaviour is consistently the same. Deleting all posts from a forum resets the msgid field. Btw, I am using vb 2.2.4

Hmmm... I just tried using the mass prune (the one built into vb), and it didn't change my lastmsg field in the database. I'm confused on why this would even happen though. vB doesn't even know the nntp tables exist when it executes that. unless you are using some other prune script.

Is this mass prune in the nntp graphical interface? If so, then that'll explain it. I don't have that hack installed. In the admin control panel, the 'mass prune' found under the 'Threads & Posts' heading did not change the lastmsg field for me.

[Erwin]

Working smoothly with 2.2.5, if anyone was wondering.

[patvdv]

Originally posted by Gilby


Hmmm... I just tried using the mass prune (the one built into vb), and it didn't change my lastmsg field in the database. I'm confused on why this would even happen though. vB doesn't even know the nntp tables exist when it executes that. unless you are using some other prune script.

Is this mass prune in the nntp graphical interface? If so, then that'll explain it. I don't have that hack installed. In the admin control panel, the 'mass prune' found under the 'Threads & Posts' heading did not change the lastmsg field for me.

Gilby, it seems weird to me too but yes I am using the standard vbulletin 'Mass Prune' option to delete all posts. I don't have the graphical NNTP admin panel installed. But I did some more tests and apparently the problem does not appear when I only prune the full contents on 1 single forum. Previously I had selected the parent forum including subforums. I will run some more tests if I get the chance

[v0nreagan]

Originally posted by rawnet
Hello,
How do we stop Usenet postings from appearing when someone clicks on the "New Threads Since Last Visit" link in the forums?

Using older stuff from Usenet hack - you need a new field in thread table.

ALTER TABLE thread ADD isusenetpost TINYINT(2) DEFAULT 0 not null;

then open your gateway.php and find:

// Create thread
$DB_site->query("INSERT INTO thread (threadid,title,lastpost,forumid,open,replycount,p ostusername,postuserid,lastposter,dateline,iconid, visible,attach,msgid,prefix) VALUES (NULL,'".addslashes(htmlspecialchars($subject))."' ,'".$date."','$forumid','1','0','".addslashes(from _name($message[from]))."','0','".addslashes(from_name($message[from]))."','".$date."','0','1','$attachcount','".addsla shes($message[msgid])."','".addslashes($theprefix)."')");

replace it with (changes enbolded)

// Create thread modded
$DB_site->query("INSERT INTO thread (threadid,title,lastpost,forumid,open,replycount,p ostusername,postuserid,lastposter,dateline,iconid, visible,attach,msgid,isusenetpost,prefix) VALUES (NULL,'".addslashes(htmlspecialchars($subject))."' ,'".$date."','$forumid','1','0','".addslashes(from _name($message[from]))."','0','".addslashes(from_name($message[from]))."','".$date."','0','1','$attachcount','".addsla shes($message[msgid])."','1','".addslashes($theprefix)."')");

From now on each thread created by gateway.php will be flagged as originating from usenet (isusenetpost=1)...

Open search.php and find two lines:

$wheresql.=" AND thread.open<>10";

in my case around line 551 and 601...
Directly below each of them add:

// NNTP Gateway mod
if (!$includeusenetpostsingetnew) { $wheresql.=" AND thread.isusenetpost=0"; };
// End of mod

Having isusenetpost in thread table also allows you to separate threrad counts or create separate links for "View new Usenet posts" ( see example here...

[patvdv]

I had the same idea but implemented it differently, without the added table field:

Open search.php and find:

PHP Code:

// ###################### Start get new #######################
if ($action=="getnew") {
  // generate query
  // do it!
  if ($bbuserinfo[userid]!=0 or $bbuserinfo[lastvisit]!=0) {
    $forumsql=getallforumsql(); 


After that, add:

PHP Code:

// HACK: search for normal or usenet posts
            if ($usenet=="true") {
                  $forums=$DB_site->query("SELECT forum FROM nntp_groups");
                  $forumsql.=" AND forumid IN (0";
                  while ($forum=$DB_site->fetch_array($forums)) {
                      $forumsql.=",$forum[forum]";
              }
            $forumsql.=")";
            } else {
                 $forums=$DB_site->query("SELECT forum FROM nntp_groups");
                  $forumsql.=" AND forumid NOT IN (0";
                  while ($forum=$DB_site->fetch_array($forums)) {
                      $forumsql.=",$forum[forum]";
                  }
                  $forumsql.=")";
        }
// HACK: search for normal or usenet posts 


Then update your forumhome_newposts template to include 2 links rather than one:

To look for normal posts:
<a href="search.php?s=$session[sessionhash]&action=getnew&usenet=false">

To look for usenet posts:
<a href="search.php?s=$session[sessionhash]&action=getnew&usenet=true">

Not being a PHP wizard I am not 100% sure that this would cover all angles but it seems to work for me fine so far

[omniweapon]

I'm not a server admin, and the server is set on safe mode. Which is really pissing me off. Also I have to use phpMyAdmin since they have Telnet disabled. Any help on how to reorganize tables and such would be appreciated.

[akaler]

Hey, Gilby... Thanks for the awesome hack!! =)

I got a question. I see you fixed the mime decode problem in the subject and from fields. I pruned my usenet groups, recompiled php with imap support, and re-imported everything... and sure enough, the froms and subjects were fixed.

There's still some mime in the text, though, that goes away. Especially when some of the header leaks into the text of the message.

Anyway to apply the same decoding to the body of the messages, too?

[akaler]

Oh... I have another question.

Some of the groups I mirror have this one single harassing user who posts something like 20-50 messages a day. I'd like to somehow filter out his messages. I feel that it could make these groups usable again.

I tried pruning by username. Nothing came up... so I then added his various user names as different users on my system. THEN I could prune by username. What then happened was half good. In prune by username, threads he started showed up, but no posts.

Any idea how to filter out the bad seeds? The hacky idea I have above would work if only imported posts, not just threads, could somehow be attached to a username. I know vb.org accomplished this somehow. They grabbed the hacks forums from vbulletin.com, then when people registered here, it would attach their user name to the name they used on their old messages, and whala... full vb user based functions could be applied to old messages.

[Dontom]

Gilby... Thanks for the great hack!!

I would like to exclude Usenetposts which contain a x-archive = no in their header - has anyone already done this? or is there a hint at which part of the script i can sort those posts out?
Thank you!
Tom

[CJi]

I've battled with this hack for the best part of the last 3 hours and I've given up in total frustration. :tired:

No matter *what* I do, all I can get out of gateway.php is this:

1 group(s) gatewayed.
Logging in to news.usenet.com, group alt.sports.airsoft

nothing else.

This is what I have in nntp_groups:

alt.sports.airsoft
<nothing>
98
0
news.usenet.com
1003989135
<password>
1

Help :cry: