NNTP Gateway (Usenet/Newsgroups)

This hack provides a gateway to any NNTP newsgroup such as those from Usenet and other newsgroup servers. Selected newsgroups are imported from your news server and post on the forum are sent out to the newsgroup. This hack was greatly inspired by fastforward's usenet gateway hack. This one is a complete rewrite of that hack written in PHP and works with vB 2.2.2.

Installation:
Installation is pretty simple. No modifications to the vB php files are needed and it simply consists of dropping in the files to your forums directory, running the SQL modifications, and modifying the newsgroup settings for your desired newsgroups. For the complete installation details, read the readme.txt file.

Here are some of the features:

• Real cool vbcode to text conversion!
• All posts are inserted, even if parent thread cannot be found for replies
• Guests posts also get sent to the newsgroup.
• Importing of attachments from the newsgroup
• Poll info sent to newsgroup
• Attachment link sent to newsgroup
• Supports unlimited newsgroups and servers.
• email notification of new replies, just like regular forum posts
• Can import HTML messages and convert to limited vb code.

For control panel integration, please check out GameCrash's Graphical Interface addition.

What's not in it that was in fastforward's usenet gateway hack:

• No control panel integration at the moment. (Available separately)
• Does not support separate footers
• Does not allow custom email address to be used via the users option
• No spam control or string replacement support.
• No purging or autoexpire.

There is still lots that I'd like to add to it, but this is competely functional for basic newsgroup gateway support. It is most likely not free of bugs. If you have questions, suggestions or annoyances, please post in this thread.

vb3 support
lierduh modified this to provide support for the beta versions of vb3. When the release candidates of vb3 come out, I plan to add support for vb3 and add other features to this hack. If you have suggestions for new features, let me know.

E-mail gateway support
New in this version, you can now gateway with an email list. This will fetch emails from a POP email account and send via php's mail function to the mailing list's email address. To set up the list, in the newsgroup field, enter in the email address to send outgoing posts to. In the server field, enter in the pop3 server, and in the username and password fields enter in, you guessed it, the username and password to your pop account. Note: the email gateway capabilities has not been fully tested, so it might not work with your POP server. Also, it does not work in combo with a news<->forum gateway in the same forum (you can use either kind though in differnet forums within your forums).

[Schorsch]

I uninstalled this hack, same reason like gopherhockey posted above.

[Marv]

I told you about the problems in german newsgroups and about the problem with the email-adresses. This is solved now.
But now I?m struggling with something different.

I noticed that postings which have been send via the gateway don?t have proper headers and - you guessed it - on german newsservers this is like a spike in the flesh and it?s raining flames.

Okay - that?s the prolog..now facts:

mime.php should encode the outgoing posts in line 132 - 177 :

/*!
-- MIME Encoding --
send() - create the mime message and send it to the listed recipients.
*/
function send() {

srand((double) microtime()*1000000);
$rnum = rand();

$boundary = "=_".md5($rnum);

$message = "This message is in MIME format.\n\n";

foreach($this->attachments AS $attachment) {

$message.= "--".$boundary."\n";
$message.= "Content-Type: ".$attachment["content-type"];
if ($attachment["content-type"] == "text/plain") {
$message.= "; charset=ISO-8859-1\n";
$message.= "Content-Transfer-Encoding: Quoted-Printable\n\n";
$message.= $this->quoted_printable_encode($attachment["contents"])."\n";
} else {
$message.= "; name=\"".$attachment["filename"].
"\"\nContent-Transfer-Encoding: base64\n";
$message.= "Content-Disposition: attachment; filename=\"".
$attachment["filename"]."\"\n\n";
$message.= chunk_split(base64_encode($attachment["contents"]))."\n";
}

}
$message.= "--".$boundary."--\n";

$headers = "MIME-Version: 1.0\n";
$headers.= "Content-Type: multipart/mixed;".chr(10).chr(9).
"boundary=\"$boundary\"";

if (is_array($this->headers)) {
foreach($this->headers AS $header) {
$headers.= "\n$header";
}
}

foreach($this->recipients AS $email) {
mail($email, $this->subject, $message, $headers);
}

}

So far mime.php should encode the headers with:

Mime-Version: 1.0
Content-Type: multipart/mixed

The body should be encoded with MIME, too :

Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: Quoted-Printable

But as As far as I can see mime.php isn?t doing what it should. It doesn?t encode the headers of outgoing posts and it doesn?t encode the body nor it adds the lines

Mime-Version: 1.0
Content-Type: ...
Content-Transfer-Encoding: ..

to the headers.

In other words: it seems that it?s doing nothing at all.
Could someone please have a look what?s happening there ? It?s now the third night and actual 3.04 AM and I?m about to go crazy about this thing... aranoid:


Note:
In my files I changed the values to

Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit

because that?s the german nationwide standard.

[gopherhockey]

This from logican from another conversation on possibly making posts invisible that are generated from this hack. Anyone want to try this with this hack??

---

A work around might be using this trick:
Hack your gateway hack so that it posts with a special username (a fake username
that does not belong to any member but a bot). Then you can exclude it in your
query to get accurate results: Eg.

PHP Code:

$getnewthread=$DB_site->query_first("SELECT COUNT(*) AS threads FROM thread
WHERE lastpost > '$bbuserinfo[lastvisit]' AND forumid!=X");

$getnewpost=$DB_site->query_first("SELECT count(*) AS posts FROM post WHERE
dateline > '$bbuserinfo[lastvisit]' AND username!=X"); 


In order to fix this, we need to know how to get the gateway script to post as a user, not as "guest" as it seems Guest is not really treated as a regular account.

Someone else asked the question, but I don't believe anyone has answered it.

Anyone know how to get the script to use a real userid when posting? Say I create a user "NNTPGateway" ??

Thanks - this could POSSIBLY allow me to turn this back on.

[gopherhockey]

I did it... you use the userid instead of username...

To ignore posts totals from the NNTP Newsgroup, simply edit your index.php and change:

 $countposts=$DB_site->query_first('SELECT COUNT(*) AS posts FROM post');

to:

 $countposts=$DB_site->query_first('SELECT COUNT(*) AS posts FROM post WHERE userid!=0');

If you run a script such as webwelcome where it picks out the number of posts since the user last logged in, just find the line:

 $getnewpost=$DB_site->query_first("SELECT count(*) AS posts FROM post WHERE dateline > '$bbuserinfo[lastvisit]'");

and change to:

 $getnewpost=$DB_site->query_first("SELECT count(*) AS posts FROM post WHERE dateline > '$bbuserinfo[lastvisit]' AND userid!=0");

You can do this for the threadcount as well - just use

thread WHERE forumid!=XX (where XX is the NNTP forum)


Now, I can turn this sucker back on!

[JF1980]

Why would I want to use this hack over the original when it lacks features of the original?

[gopherhockey]

What features are lacking?

IMHO this one was much better than the other, but I'm interested to see what you find in the other one that this one doesn't have.

[JF1980]

Well:

Does not support separate footers

Does not allow custom email address to be used via the users option

No spam control or string replacement support.

No purging or autoexpire.

So what advantage does this version offer over the original that makes it worth dropping these features?

[gopherhockey]

I'm not all that familiar with the "original" other than that it did not work with my 2.2.9 version and this one did. I also believe the pop3 features (email to a forum and to list) were not available in the other one at the time. Most the other features you list seem to be built into vb, so why would they need to be included in a hack?

I guess if you are so happy with the original, why not just stick with it.

Perhaps someone familiar with both can post here what the difference are. All I know is that this one worked without an issue and does everything I wanted (and more) in an NNTP gateway for vb.

[JF1980]

Well I don't have either yet and am trying to make a choice on which to go with. I was hoping someone who has used both could give advise.

What are these POP features then and why would I want them?

Basically, I want to be able to use 4 or 5 newsgroups on my site as forums. I want to have it syncronize with the news server every 10 mins (at longest, maybe less), I want to allow only registered members to post through the board and to have a footer put at the bottom of every post made from the board with a link to the site (e.g "Posted via www.thrill-seekers.net". Im not sure about expiry and deletion of out of date posts, I guess it would be ideal if they were removed from the board when they dropped off the news server, or maybe even if they stayed there for 30 days, then were automatically deleted (with useful threads being manually moved to other forums). Does this sound within the scope of this hack?

[gopherhockey]

This hack does what you are asking. There is a footer stuck in there showing a link to your profile and a link to the thread where the message was posted etc.

Expiring stuff can be handled by vb automatically - you can set how long you want to keep things and purge what you don't wish to keep. (lets you keep things around longer than your server, in case you want it there for reference etc.)

I use the pop3 feature to be able to email directly to a forum. (but only use this sparingly and don't advertise the address to many) - e.g. I subscribe to various sales or subscription emails that are related to my boards content and have those automatically show up in a forum as a post without having to do it manually. SPAM for this is controlled by my own server before it gets to the email account, then the hack pulls the messages using pop3 from my server.

I even set it up so posts going to my forums via NNTP are not counted in the rest of my vb systems statistics, so users aren't flooded with usenet garbage when they log in, but only see actual posts from local (registered) users.

But like I said, I haven't played with the other one much since it did not work for me initially. From what I remember I too compared them and liked some of the features this one had - plus it seems to get more support. I could be wrong, I'm sure the other one is good too. I know you won't go wrong with this one though, it is one of the most useful hacks I have (and may even cause me to wait before upgrading to vb3 until the hack is made compatible)

Hope that helps?