NNTP Gateway (Usenet/Newsgroups)

This hack provides a gateway to any NNTP newsgroup such as those from Usenet and other newsgroup servers. Selected newsgroups are imported from your news server and post on the forum are sent out to the newsgroup. This hack was greatly inspired by fastforward's usenet gateway hack. This one is a complete rewrite of that hack written in PHP and works with vB 2.2.2.

Installation:
Installation is pretty simple. No modifications to the vB php files are needed and it simply consists of dropping in the files to your forums directory, running the SQL modifications, and modifying the newsgroup settings for your desired newsgroups. For the complete installation details, read the readme.txt file.

Here are some of the features:

• Real cool vbcode to text conversion!
• All posts are inserted, even if parent thread cannot be found for replies
• Guests posts also get sent to the newsgroup.
• Importing of attachments from the newsgroup
• Poll info sent to newsgroup
• Attachment link sent to newsgroup
• Supports unlimited newsgroups and servers.
• email notification of new replies, just like regular forum posts
• Can import HTML messages and convert to limited vb code.

For control panel integration, please check out GameCrash's Graphical Interface addition.

What's not in it that was in fastforward's usenet gateway hack:

• No control panel integration at the moment. (Available separately)
• Does not support separate footers
• Does not allow custom email address to be used via the users option
• No spam control or string replacement support.
• No purging or autoexpire.

There is still lots that I'd like to add to it, but this is competely functional for basic newsgroup gateway support. It is most likely not free of bugs. If you have questions, suggestions or annoyances, please post in this thread.

vb3 support
lierduh modified this to provide support for the beta versions of vb3. When the release candidates of vb3 come out, I plan to add support for vb3 and add other features to this hack. If you have suggestions for new features, let me know.

E-mail gateway support
New in this version, you can now gateway with an email list. This will fetch emails from a POP email account and send via php's mail function to the mailing list's email address. To set up the list, in the newsgroup field, enter in the email address to send outgoing posts to. In the server field, enter in the pop3 server, and in the username and password fields enter in, you guessed it, the username and password to your pop account. Note: the email gateway capabilities has not been fully tested, so it might not work with your POP server. Also, it does not work in combo with a news<->forum gateway in the same forum (you can use either kind though in differnet forums within your forums).

[Ogmuk]

Thanks. I tried it with that file but the same result. Actually the output is even weirder (small part of the output):

Getting message number 49633: forum post
Getting message number 49634: forum post
Getting message number 49635: forum post
Getting message number 49636: Getting message number 49637: forum post
Getting message number 49638: forum post
Getting message number 49639: forum post
Getting message number 49640: Getting message number 49641: Getting message number 49642: forum post
Getting message number 49643: Getting message number 49644: Getting message number 49645: Getting message number 49646: Getting message number 49647: Getting message number 49648: Getting message number 49649: Getting message number 49650: Getting message number 49651: Getting message number 49652: Getting message number 49653: Getting message number 49654: Getting message number 49655: forum post
Getting message number 49656: forum post
Getting message number 49657: forum post
Getting message number 49658: forum post
Getting message number 49659: forum post
Getting message number 49660: forum post
Getting message number 49661: forum post
Getting message number 49662: forum post

To what else could this be related? Dated PHP/MySQL?

[allan grossman]

Originally Posted by Ogmuk

Thanks. I tried it with that file but the same result. Actually the output is even weirder (small part of the output):

To what else could this be related? Dated PHP/MySQL?

I have no idea

I know I'm not a lot of help here - but the thing should work. I don't know why it doesn't.

[Ogmuk]

Originally Posted by allan grossman

I have no idea

I know I'm not a lot of help here - but the thing should work. I don't know why it doesn't.

No problem I hope others still read this thread.

[Ogmuk]

Looking back my guess is that emptying the fields useragent and organization caused this odd behavior, that or it just can't handle a full forum prune. I completely emptied the database of my test board and reinstalled. Its working like it should so I won't change those two fields anymore.

Only one problem that I still have, it won't add the imported messages to the search index. It will only add the messages that were posted via the forum to the search index. Is there anything specific I have to do to enable this?

edit: I've just tried searching on your forum Allan but I can't search on any words from the Guest users. I assume that this feature isn't available yet so I'll just create the gateway on my live forum without a search option for now.

[allan grossman]

Originally Posted by Ogmuk

Looking back my guess is that emptying the fields useragent and organization caused this odd behavior, that or it just can't handle a full forum prune. I completely emptied the database of my test board and reinstalled. Its working like it should so I won't change those two fields anymore.

Only one problem that I still have, it won't add the imported messages to the search index. It will only add the messages that were posted via the forum to the search index. Is there anything specific I have to do to enable this?

edit: I've just tried searching on your forum Allan but I can't search on any words from the Guest users. I assume that this feature isn't available yet so I'll just create the gateway on my live forum without a search option for now.

I didn't know you couldn't search on guest users. I'm gonna quit indexing the Usenet forums

Can you search on guests from within the forum itself?

[allan grossman]

Originally Posted by allan grossman

I didn't know you couldn't search on guest users. I'm gonna quit indexing the Usenet forums

Can you search on guests from within the forum itself?

If you rebuild the search index it'll work, I think.

Head back to my forum and search on the keyword "bassist". You'll see all kindsa Usenet hits.

[Ogmuk]

Originally Posted by allan grossman

If you rebuild the search index it'll work, I think.

Yup, I think I mentioned that earlier too. Only if you update it manually it will work I was hoping that it would be done automatically.

Another thing I noticed with some quick tests. For example when people use 5 spaces, is there a way to get the 5 spaces to show on your forum too? Some ASCII signatures will not be displayed correctly on your forum is one reason that it would be handy

edit: and a bug I noticed if you edit some of the posts to view their content. This is one piece from a message:

PHP Code:

> test test test  test test test  test test test  test test test  test test
test[color=blue]
> test test test  test test test  test test test  test test test  test test[/color]
test[color=blue]
>  test test test  test test test  test test test  test test test[color=green]
> >  test test test  test test test  test test test
> >  test test test  test test test  test test test[color=darkred]
> > > test test test  test test test  test test test  test test test  test[/color][/color][/color]
test test[color=blue][color=green][color=darkred]
> > > test test test  test test test  test test test  test test test
> > > test test test  test test test  test test test  test test test  test[/color][/color][/color]
test test[color=blue][color=green][color=darkred]
> > > test test test  test test test  test test test

etc. 


I'm referring to the large amount of useless repeating COLOR vbcode if it wasn't obvious

[_color=blue][_color=green][_color=darkred]text[_/color][_/color][_/color]

It adds to the database's size when importing hundreds of messages a day

[Ogmuk]

Another thing. How difficult is it to make gateway.php, once it is loaded, convert the posts that were send via the forum into how they will be shown to the rest of the newsgroup, with 72 characters per line, signature, etc?

[allan grossman]

Originally Posted by Ogmuk

Another thing. How difficult is it to make gateway.php, once it is loaded, convert the posts that were send via the forum into how they will be shown to the rest of the newsgroup, with 72 characters per line, signature, etc?

Each quoted post will be a different color. Nested quotes, nested colors

I don't think you can hack a 72-character display without doing some major alterations on vBulletin files, Ogmuk - the gateway picks up the post *after* it's been posted to the forum. You'd have to hack both newthread.php and newreply.php - and both would be systemwide changes.

[Ogmuk]

I meant it in a way that it would edit the original post and replace it with a 'paste' of the 'echo' (if that makes any sense), so that you can see how it looks without using a newsreader. I think it would only need to remember the latest posts which were uploaded so it can download these with the next synchronization and replace the originals (preferably without turning them into unregistered users. so it would need to only replace the post's content).

No idea if such a thing is possible though or how much work it would be to change this