Usenet gateway

THIS THREAD IS CLOSED!
Posts in here will NOT be answered

This hack has undergone many major changes since this thread was started. Consequently, most of the posts have become dated and of little use. To coincide with the latest major release (20010712), a new one has been started.

See this thread for the latest version and discussion

This is a very cool hack.

Though I wonder how full the database might get with a busy newsgroup.

Have you tried it with a busy one?

And, how does it handle any binary attachments?

I think it doesn't support MIME. It would be cool tho

This is a very cool hack.

Though I wonder how full the database might get with a busy newsgroup.

Have you tried it with a busy one?

And, how does it handle any binary attachments?

Binary attachments are not supported. Although it would not be difficult to add a feature that saves the binary attachment to a directory and post a link in the forum.

There is a variable called $max_msg_length that can be set to the max number of lines that a message can be before it is assumed the post is a binary one. The default setting is 300 lines. In addition there is a spam filter to check for 'begin 644' in the body and reject the post if it is found (begin 644 means the post is binary).

As for busy groups. It all depends on your news server speed, your connection speed and the amount of space you have available for storage. But for practical use where you would maybe have 10-20 newsgroups, there will be no problem at all.

There was bug in the expire_article routine that was causing articles never to be removed from the usenet_ref and usenet_article tables. The fix is only in the newnews.pl script and affects only one line.

The link in the first post of this will always point to the latest release.

Hi...me again. I ran the SQL changes, no problems there (although the spam table isn't showing up on the database...any idea why? Is this required?)

Anyways, made the modifications to the newnews.pl file, uploaded to the cgi-bin directory (I don't believe my web host will allow to execute scripts outside of CGI-BIN), chmodded it to 755. Now I'm getting an Internal server error when I run the script from my browser.

I updated the first line of the script to my local path to perl (I know this works because I had UBB up and running before vbulletin, and I just copied that from the cgi script that was working). The only question in my mind is the following line in newnews.pl:

#use lib "/[path.to.your.local.perl.directory]/perl5/site_perl/5.005";

What is the proper syntax for this line? Could this be causing the internal server 500 error I'm seeing? My path to perl is #!/usr/bin/perl

Another question...why are there two newnews.pl files in the distribution? Which should I be using? The one in the Rcs folder looks different than the other one (header information) so I assumed that wasn't the correct one.

Thanks in advance!

The usenet_spam_filter table IS required. The script will fail if this is not here. It may be the cause of your error. If this is not being created, there may be an error in my SQL. Try running just the part that builds this table.

You don't need the '#use lib' line as your ISP has installed the modules for you.

The shebang line is probably correct. If you had ubb running using that line then it definately is.

The RCS sub directory is for revision control. You can ignore it. I just included it for completeness.

One point though, I did not intend this script to be executed via the browser and I have no idea whether it will work or what implications it may have. I will do some testing over the next few days and let you know the outcome.

regards

Not sure why the spam table wasn't added, but I ran just that part of your script and it did add the table to the database.

I'm kinda baffled at this one...I changed the newnews.pl again, re-uploaded it to my cgi-bin directory, Telnet'ed into my account, and I can't find the file there!!!! It shows on my FTP program, and I can even hit it with my browser (although I get that same 500 error), but looking for it through the telnet shell, I can't find it! I want to execute it from there so I can see if/what the error is, but can't find it.

Yes, I did CHMOD it to 755, even tried 777 and it just doesn't show up. Any thoughts?!?

stupid telnet program...found the program, so I've now run newnews.pl via telnet. Connects to the server and the right newsgroup, grabs the first 100 messages, when it does to dump them to the database, I'm getting the following error:

Requested 100 messages... 0 not available or rejected.
DBD::mysql::db do failed: You have an error in your SQL syntax near ''No User',9
79507486,'ALISHA has done the following\n____________________________' at line 1
at ./newnews.pl line 419.
Query failed (INSERT IGNORE INTO post (allowsmilie,threadid,username,dateline,pa
getext,visible,msgid,ord) VALUES ('1',,'No User',979507486,'ALISHA has done the
following\n_______________________________________ _\nSlept: 7 Touched: 31750 Dro
pped: 37\n________________________________________\nNetb ios Rulez \\GUYC \\C \\E
SC400 \\C \\ACI \\BOB PUBLIC \\C GATEWAY \\CUSTOMERS \\DOWNLOADS \\MY DOCUMENTS\
n\\MYLABEL \\PROGRAM FILE \\COMPAQ5695 \\MY MUSIC \\MYDOCS \\C \\PAVILLION A \\P
AVILLION C \\backup \\D \\C \\D\n\\E \\C_PAZR1 \\HOME C DRIVE \\TEMP \\C \\C \\C
\\C \\C \\D \\Downloads \\SHARED \\HHC \\HHPROGRAMS \\HHSHARED\n\n..and the wor
ms ate into his brain\n','1','<0e8cc39d915d766e63245db40b0119cb@an on.xg.nu>','0'
)) at ./newnews.pl line 419.

Any ideas here? I'm at a total loss here. It looks like it's having a problem dumping the messages into the post table, but I'm not an SQL expert).

On another note, a couple of things you might want to consider when you do your next documentation update:

1.) The SQL changes (textfile) - There are two usenet_outgoin_log table create commands

2.) You are using the table POST and post interchangably, which is causing errors in the alter table section toward the bottom of the query. Took me awhile to figure that mysql is case sensitive.

3.) You might want to make some recommendations about a mysql client. I'm using phpMyAdmin, and it's great. I couldn't imagine trying to do this stuff manually.

That's it for now! Any suggestions on the above?

The problem seems to be that 'null' is being inserted into the threadid column. As far as I can see, there are two possible reasons for this error:

Number 1 is line 292 that reads

$threadid = $dbh->{'mysql_insertid'};

is failing for some reason. mysql_insertid is a built in mysql function and returns the id of the last inserted record. I have no idea why this would not work on your setup. What version of Perl and MySQL are you using?

Number 2 is if the query on line 306 was pulling back null for the threadid column from the thread table. Now this is virtually impossible as threadid is an auto_increment column with a unique index on it.

I don't want to ask the obvious, but I will you are installing this on version 1.15 or 1.114 right?

I'll need to do some more investigation. I'll get back to you.

As for the SQL instructions. I'm an Oracle DBA so I'm used to tablenames only being case sensitive if you put double quotes around them. I will correct them asap.

regards