NNTP Gateway (Usenet/Newsgroups)

This hack provides a gateway to any NNTP newsgroup such as those from Usenet and other newsgroup servers. Selected newsgroups are imported from your news server and post on the forum are sent out to the newsgroup. This hack was greatly inspired by fastforward's usenet gateway hack. This one is a complete rewrite of that hack written in PHP and works with vB 2.2.2.

Installation:
Installation is pretty simple. No modifications to the vB php files are needed and it simply consists of dropping in the files to your forums directory, running the SQL modifications, and modifying the newsgroup settings for your desired newsgroups. For the complete installation details, read the readme.txt file.

Here are some of the features:

• Real cool vbcode to text conversion!
• All posts are inserted, even if parent thread cannot be found for replies
• Guests posts also get sent to the newsgroup.
• Importing of attachments from the newsgroup
• Poll info sent to newsgroup
• Attachment link sent to newsgroup
• Supports unlimited newsgroups and servers.
• email notification of new replies, just like regular forum posts
• Can import HTML messages and convert to limited vb code.

For control panel integration, please check out GameCrash's Graphical Interface addition.

What's not in it that was in fastforward's usenet gateway hack:

• No control panel integration at the moment. (Available separately)
• Does not support separate footers
• Does not allow custom email address to be used via the users option
• No spam control or string replacement support.
• No purging or autoexpire.

There is still lots that I'd like to add to it, but this is competely functional for basic newsgroup gateway support. It is most likely not free of bugs. If you have questions, suggestions or annoyances, please post in this thread.

vb3 support
lierduh modified this to provide support for the beta versions of vb3. When the release candidates of vb3 come out, I plan to add support for vb3 and add other features to this hack. If you have suggestions for new features, let me know.

E-mail gateway support
New in this version, you can now gateway with an email list. This will fetch emails from a POP email account and send via php's mail function to the mailing list's email address. To set up the list, in the newsgroup field, enter in the email address to send outgoing posts to. In the server field, enter in the pop3 server, and in the username and password fields enter in, you guessed it, the username and password to your pop account. Note: the email gateway capabilities has not been fully tested, so it might not work with your POP server. Also, it does not work in combo with a news<->forum gateway in the same forum (you can use either kind though in differnet forums within your forums).

[lierduh]

Originally Posted by Imhotep

What about my posts not showing up in the group? I did a test posting from my forum... and then later tried to view it using Outlook Express. (as I normally view the groups) Nothing appeared.

thanks

If you run the script manually and cut and paste the message on the screen to here. Someone might be able to help you.

[allan grossman]

Originally Posted by lierduh

That is good you have it fixed. It was error 381 related. Long time no show Allan.

Good to see you too, lierduh - I've been a bit busy

381 isn't an error, it's a password prompt - I checked.

Right after the 381 I got group and article numbers - so I did manage to authenticate to the server. I didn't keep the old gateway.php to see where it was broken but it couldn't have been too much because I glanced over the file a couple of times while I was trying to fix it.

[locatha]

Hey can anyone give me some ideas on how fast this hack works as far as importing msgs from usenet? I have been looking for something like this and have a few usenet groups i would like to have in my VB but they are fairly large and get almost 1000 each (there is three of them. Plus of course the backlog. about how long does it take, the server is a dedicated celeron 1.7 ghz. thanks and great looking hack.

[Odoin]

Back when I was using it on VB 2 it was really quick once it got the first batch of messages... the bottle neck is getting the messages from the NTTP server... Keep in mind a lot of messages will grow your DB quickly.

Mark

[locatha]

how many messages did you have in the initial batch or were doing per day. for my 3k per day if i ran it hourly it would have something like 100 msgs a run, would I need to up my php timeout from 30 seconds for that, or does it download 100 msgs in less than that? I might have to run it more often, any ideas?

[Imhotep]

Originally Posted by lierduh

If you run the script manually and cut and paste the message on the screen to here. Someone might be able to help you.

Hey, thanks for the reply. When I run gateway.php manually, I do not get a message but rather a blank white page. It does do something though... (pauses for a few seconds) and brings up new posts from the groups. I test my outgoing messages by posting a reply in my forum, running gateway.php manually, and then waiting for it to appear in Usenet. I have yet to get a reply to appear.

I am currently running...

VB 3.0.1
VB Portal 3.0.0 Beta 2
PhotoPost

site: www.cghaus.com

I installed the original hack from this thread

https://vborg.vbsupport.ru/showthrea...7&page=1&pp=15

and then followed up with the additions from this thread

https://vborg.vbsupport.ru/showpost....&postcount=658

I am still waiting for someone to reply to my posts on my host's forums regarding my cron job. In the meantime, I am running gateway.php manually.

http://www.lunarforums.com/forum/viewtopic.php?t=15006

TIA!

[Imhotep]

I just got a reply back from tech support and they say my cron job is correct. I check my web email and see this msg.

Status: 404
Content-type: text/html
X-Powered-By: PHP/4.3.3

No input file specified.

My cron line is...

php /home/cghaus2/public_html/forums/gateway.php?log=cron

I'm not sure about the "log=cron" part... I got it from the 1st hack.

[lierduh]

I have rewritten big chunks of the gateway.php. I will give it some test before uploading.

Improvement include:

1) TABLE_PREFIX is now supported.
2) Messages imported from usenet are now inserted into threads based on Reference in the header instead of the subject.
3) Built mechanism to avoid two instances of the gateway.php running at the same time. This can easily happen if someone clicks the link in the Who Is On Line page. This should also solve the problem of double up posts and wrong post counters.
4) Masive speed improvement!
5) Cleaned up the codes.
...

Stay tuned.

[hubba]

lierduh, perhaps any idea how to exclude the p´NG postings from the counter of new posts and themes?

[Imhotep]

Awesome! I'm hoping your new code fixes my problem of VB posts going out to Usenet. I removed the "log=cron" part of my cron job yesterday and now I am getting better results.

X-Powered-By: PHP/4.3.3
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Last-Modified: Sun, 09 May 2004 17:44:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Cache-Control: private, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: bblastvisit=1084124640; expires=Mon, 09-May-05 17:44:00 GMT; path=/
Set-Cookie: bblastactivity=1084124520; expires=Mon, 09-May-05 17:44:00 GMT; path=/
Set-Cookie: vbplang=english
Set-Cookie: showleftcolumn=1
Content-Type: text/plain

2 group(s) gatewayed.
Logging in to enews.newsguy.com, group alt.binaries.3d.lightwave
Logging in to enews.newsguy.com, group comp.graphics.apps.lightwave

Thans in advance for taking the time to work out some updated code.

[lierduh]

I have just uploaded the new package to page 44.

https://vborg.vbsupport.ru/showpost.php?p=444521

== TABLE_PREFIX is now supported.
== Messages imported from usenet are now inserted into threads based
on Reference in the header instead of the subject.
== Built mechanism to avoid two instances of the gateway.php running
at the same time. This can easily happen if someone clicks the link in
the Who Is On Line page. This should also solve the problem of double
up posts and wrong post counters.
== Massive speed improvement!
== Cleaned up the codes.
== Implemented parentid for imported posts
== lastmessage is updated now after each message retrieval. This way
no message will be imported twice, even if the script is crashed at
some stage.
== Fixed a bug which occurs in long header lines. Message may not be
retrieved due to this bug.

This time instead of a quick hack, I have gone through almost each line. So it should be a much better script.

[lierduh]

Originally Posted by hubba

lierduh, perhaps any idea how to exclude the p?NG postings from the counter of new posts and themes?

Sorry hubba, I don't quite get what you meant.

[Imhotep]

Awesome! This patch came through really fast.

I have one question though... it seems I'm getting some SQL errors when attemtping to "insert" the two new tables. They already exist. In fact, value 9 is showing Organization values. Should we just overwrite this stuff manually?

Error

SQL-query :  

INSERT INTO `nntp_settings` 
VALUES ( 9, 'Last Time Run', 'last_run', '1084316100', 'Last time the gateway.php was run', NULL , 0, 0 ) 

MySQL said: 


#1062 - Duplicate entry '9' for key 1

[lierduh]

Originally Posted by Imhotep

Awesome! This patch came through really fast.

I have one question though... it seems I'm getting some SQL errors when attemtping to "insert" the two new tables. They already exist. In fact, value 9 is showing Organization values. Should we just overwrite this stuff manually?

I see, just change the 8 and 9 in the insert to NULL. So use:

INSERT INTO `nntp_settings` VALUES (NULL, 'Is Gateway Running', 'is_running', '0', 'Check if another instance of gateway.php is running', NULL, 0, 0);
INSERT INTO `nntp_settings` VALUES (NULL, 'Last Time Run', 'last_run', '1084316100', 'Last time the gateway.php was run', NULL, 0, 0);

[lierduh]

I made a stupid mistake at the last minute. Please download the gateway.php again. I have attached a "a" to the end of the file name. The only thing changed was two lines in the gateway.php script. There is no need to replace the other files.