NNTP Gateway (Usenet/Newsgroups)

This hack provides a gateway to any NNTP newsgroup such as those from Usenet and other newsgroup servers. Selected newsgroups are imported from your news server and post on the forum are sent out to the newsgroup. This hack was greatly inspired by fastforward's usenet gateway hack. This one is a complete rewrite of that hack written in PHP and works with vB 2.2.2.

Installation:
Installation is pretty simple. No modifications to the vB php files are needed and it simply consists of dropping in the files to your forums directory, running the SQL modifications, and modifying the newsgroup settings for your desired newsgroups. For the complete installation details, read the readme.txt file.

Here are some of the features:

• Real cool vbcode to text conversion!
• All posts are inserted, even if parent thread cannot be found for replies
• Guests posts also get sent to the newsgroup.
• Importing of attachments from the newsgroup
• Poll info sent to newsgroup
• Attachment link sent to newsgroup
• Supports unlimited newsgroups and servers.
• email notification of new replies, just like regular forum posts
• Can import HTML messages and convert to limited vb code.

For control panel integration, please check out GameCrash's Graphical Interface addition.

What's not in it that was in fastforward's usenet gateway hack:

• No control panel integration at the moment. (Available separately)
• Does not support separate footers
• Does not allow custom email address to be used via the users option
• No spam control or string replacement support.
• No purging or autoexpire.

There is still lots that I'd like to add to it, but this is competely functional for basic newsgroup gateway support. It is most likely not free of bugs. If you have questions, suggestions or annoyances, please post in this thread.

vb3 support
lierduh modified this to provide support for the beta versions of vb3. When the release candidates of vb3 come out, I plan to add support for vb3 and add other features to this hack. If you have suggestions for new features, let me know.

E-mail gateway support
New in this version, you can now gateway with an email list. This will fetch emails from a POP email account and send via php's mail function to the mailing list's email address. To set up the list, in the newsgroup field, enter in the email address to send outgoing posts to. In the server field, enter in the pop3 server, and in the username and password fields enter in, you guessed it, the username and password to your pop account. Note: the email gateway capabilities has not been fully tested, so it might not work with your POP server. Also, it does not work in combo with a news<->forum gateway in the same forum (you can use either kind though in differnet forums within your forums).

[Gilby]

Originally posted by CJi
1 group(s) gatewayed.
200 Welcome to Usenet.com, 5.5g6, S8
381 PASS required
281 Ok
211 6 621 626 alt.sport.airsoft selected
Logging in to news.usenet.com, group alt.sport.airsoft

Hmmm... What is the lastmsg column set to in the database? It should be set to 0, but you can try setting it at 6.

[jarvis]

I've had this hack working great for over a month now, but now I'm getting the following error:

"mysql error: Lost connection to MySQL server during query

mysql error number: 2013"

Has my host possibly changed something that is killing this script? Like I said, I have had this working for some time now and just started getting these error messages. Does mySQL have some sort of timeout setting?

Thanks!

[patvdv]

I have been having the same problem when pulling in a big number of posts from the newsgroups. I wonder if this linked to PHP's max_execution_time? Is there anyway the post retrieval could be 'chunked' into distintinctive processes?

[akaler]

cool, gilby! thanks for all the tips.

if you need access to any of my news accounts, pm me. snews in particular has some insane retention on some groups. hopefully, the memory problems goes away when php or other components are updated. 4.1.2 IS known to be pretty buggy, and it doesn't look like anyone else is having any memory issues, per say.

and i'll definitely kick you some money next pay period! i hope everyone does.

[akaler]

Originally posted by patvdv
I have been having the same problem when pulling in a big number of posts from the newsgroups. I wonder if this linked to PHP's max_execution_time? Is there anyway the post retrieval could be 'chunked' into distintinctive processes?

well, i for one, DID need to increase php's max execution time. but i think mysql timeouts are something else entirely.

I changed my max_execution_time in /etc/php.ini to something crazy, like 6000 seconds. That's 6000 CPU seconds, though, so it's really overkill. If you look at CPU time with top for a really long process, you'll see cpu differs from real-time in a very large way.

Again, this shouldn't fix mysql timeouts. Message eva2000 and point him at this thread. He's the quicker picker upper for MySQL!

[akaler]

Hey, Gilby, got a question...

(heck, come to think of it, this may have been asked and answered before in the past 365 something messages -- i'm sorry if it has)

If someone is subscribed to a thread with email notification, do you know if importing messages is supposed to alert them? It seems like if users go to their CP panel, they'll sometimes see subscribed threads as being active, but they'll never actually be notified in any other way.

It'd be cool if they could be. While most users like Usenet via vB, some do say 'why not just use Google?'. The more normal vB options work for threads, forums, users, etc., the more edge you have over something like Google Groups. Google is so BASIC, while vB is more... amazing and far more usable on an everday basis. Especially with all the wonderful hacks!

Anyhoo -- next things I'm going to install are Community Bulletin Generator and some sort of Top 10 Active Threads hack. Maybe the syndicated content hack, I dunno.. Hopefully they work with usenet generated threads. I bet they will. A few other hacks I've installed work GREAT, I'll tell you that! The only one that has any weirdness is Quick Reply. I think it replies to the first message in a usenet thread, and not the last message. This confuses the normal usenet folks who are reading on a threaded basis.

Other than the weirdness with Quick Reply, and the non-updating of subscribed threads and forums... everything is quite fab vB integration wise. This is certainly a superior step up from using Google or your news reader!

[winkler1]

Great hack. We had difficulty getting lynx to invoke gateway.php properly - it would return the forum main page, although we'd visited the site w/lynx to get cookied. Log files show an HTTP/1.0 request; viewing gateway.php with IE5 is an HTTP/1.1, and works fine... any ideas? I just wrote a windows app in the meantime with an embedded IE control on a cron.

Here's a wierd case - at Time1) user posts to vbulletin, and at Time2) gateway.php ships the msg to news server..then @Time3) the user edits the message. Does the new content ever get repropagated? My tests indicate not. Sorry if this is an FAQ, 'edit' is a lousy search term!
Jeff

[Gilby]

Originally posted by akaler
If someone is subscribed to a thread with email notification, do you know if importing messages is supposed to alert them? It seems like if users go to their CP panel, they'll sometimes see subscribed threads as being active, but they'll never actually be notified in any other way.

It's supposed to, and it did when I first tested that feature after putting it in, however, I have begun to wonder if it's actually being sent out or not. It might not.

[Gilby]

Originally posted by winkler1
Great hack. We had difficulty getting lynx to invoke gateway.php properly - it would return the forum main page, although we'd visited the site w/lynx to get cookied. Log files show an HTTP/1.0 request; viewing gateway.php with IE5 is an HTTP/1.1, and works fine... any ideas? I just wrote a windows app in the meantime with an embedded IE control on a cron.

I don't understand your problem. If gateway.php is executed, then it'll import and export the newsgroup posts. However, with the added logging variable in the url that I have in the example, it will not output anything, unless there is a major error, in which case if this is a linux/unix setup, it'll send an email to the cronjob owner.

Originally posted by winkler1

Here's a wierd case - at Time1) user posts to vbulletin, and at Time2) gateway.php ships the msg to news server..then @Time3) the user edits the message. Does the new content ever get repropagated? My tests indicate not. Sorry if this is an FAQ, 'edit' is a lousy search term!
Jeff

It does not repropogate it. The message may have replies and cancelling the message and reposting it would break the threads. What I do on my forums is set it up to have a low edit time and leave it at that.

[winkler1]

I don't understand your problem.

it's pretty strange..gateway is at http://<host>.com/forums/gateway.php. When we use lynx to get the page (leaving off log=cron to keep things simple), the HTML shown belongs to the vBulletin main welcome page... the server log file shows a 200 code, with a big content length being sent back.
Gateway.php is set up, going and working just fine when we visit it with a normal web browser.
Ever seen this problem?