Usenet gateway

THIS THREAD IS CLOSED!
Posts in here will NOT be answered

This hack has undergone many major changes since this thread was started. Consequently, most of the posts have become dated and of little use. To coincide with the latest major release (20010712), a new one has been started.

See this thread for the latest version and discussion

Not a dumb question at all. I am running:

vb 1.1.5
mysql 3.23.32
perl version, well, as you know, hard to determine, but if I issue the command $perl -MDate::Calc -e 'print "$Date::Calc::VERSION \n"' the version returned is 4.2.

Anything else? I've tried a bunch of different stuff, manually wiped out the records in the usenet_article and usenet_ref tables, reinstalled everything (including vb), with no avail.

Thank you, looking very forward to getting this up and running, and your support is better than most technical support companies!!!!!!!!!!!!!!!!!

-RC

OK... looks like you have uncovered a little buggette. I think the problem is this.

When a usenet article is received that will become the thread starter, an entry is made in the thread table. The statement that does this is on line 264 and uses the INSERT IGNORE clause to prevent any possibility of duplicate messages if your news server decides to reindex and change message numbers on you.

However, the side effect to this is should the statement ever actually be ignored due to a duplicate messageid, the next line that gets the mysql_insertid will return null as nothing was actually inserted. So consequently, when it tries to put the corresponding entry into the post table on line 266, it will fail because threadid in the post table is specified as NOT NULL.

Now this is all very well and is easy to fix, but the real issue is why you came across a duplicate. The message id column is supposedly unique across all of usenet. You mentioned you tried again by emptying out the usenet_article table first. If so, the only way a duplicate could occur would be if the article came twice with the same messageid from your news server. But even this shouldn't matter as the usenet_article table has a unique index on the 'msgid' column.

I'll fix the script and post it in about an hour. In the meantime, check the following:

• make sure the only unique keys on the thread table are 'threadid' and 'msgid'
• make sure the usenet_article table has a unique index on 'msgid'

regards
PAJ

Well, glad (and sorry) to hear it doesn't look like it was me. Sorry to put you through this, but I guess that's the nature of the beast, huh?

Unique keys exist as you state in your message above. Really wierd I agree. At any rate, look forward to your reply.

Thanks again!


---------------------------------
Captnroger
Forum Moderator
Gofishohio.com
www.gofishohio.com

The fix has been incorporated into the script. The newnews.pl and the sql instructions are affected.

The change simply checks whether the $threadid is null prior to inserting into the post table.

If this doesn't work for you captnroger we'll need to review the entire setup.

If your problem was actually caused by a faulty mysql_id function you will not see any messages in your forums.

PAJ
--
the latest version of this script is always in the first post of this thread.

Here's the error I'm getting now (I did run the SQL query without an error though!) Modified the new newnews.pl, ran it, here's the message:

DBD::mysql::st execute failed: You have an error in your SQL syntax near '' at l
ine 1 at ./newnews.pl line 424.
Query failed (SELECT title FROM thread WHERE threadid=) at ./newnews.pl line 42
4.

Sorry to be a pain. Any thoughts?

OK.. assuming the problem is due to duplicate messageids the problem was my fault. It was trying to index that non-existent thread. I've added another check to stop it happening.

I still may have missed some so let me know if it fails anywhere else.

regards

FF -

First of all, I commend your efforts today to help me get this working. You truely are a champ.

The latest mods to the script got me through the importing process, YEAH! The script executed without an error.

What DID happen though is that while my forum has all the headers, along with the author, date and time, if you click on the actual post, there is nothing there. In looking at the database, nothing is actually being posted to the post table. I have three test messages in there that I created, that's it. The newsgroup messages don't make it to the post table. Looks like they are still in usenet_article.

So close!!!!!!!!!!

That indicates that something really is wrong with the 'mysql_insertid' function call or for some reason it's returning null.

Remove the IGNORE word from the SQL in line 303 so that it reads:

db_execute ("INSERT INTO thread (title,lastpost,forumid,postusername,lastposter,dateline,msgid,visible,open) VALUES ($subject,$dtm,$forumid,$poster,$poster,$dtm,$msgid,'1','1')");

and add the following debug line right after line 304 that reads $threadid = $dbh->{'mysql_insertid'};

print "DEBUG: threadid for msgid $msgid is $threadid\n";

Now if the SQL doesn't fail we know it is the 'mysql_insertid' function at fault. If the SQL fails then show me the error message as you did before.

The other thought I have is maybe your news server doesn't fully support the 'xhdr' function. This is unlikely but it would cause any header it couldn't find to be null.

What provider are you using? To check if this the problem I can give you temporary access to a test DNEWS server I have at home. It will be slow though as I only have 256K upload speed. Let me know your IP and I'll add access for testing.

regards.

hmmm....interesting. Never considered it may be the newsfeed. I'm using newsfeed.com, although I am using their spam-free newsfeed. Let me play around with this a bit...I'll get back with you.

okay, just to be safe I wiped out the database and started over again, and also changed news feeds. Here is the debug of the session as per your request:

Connecting to News.newsfeeds.com... Connected
Sending authentication info... Authenticated and logged in
Getting article batch from alt.test
Fetching headers of articles 848576 to 848675... done
Fetching article body 848576... OK
Fetching article body 848577... OK

much..
deleted..
here..

Processing article batch...
Requested 100 messages... 14 not available or rejected.

DEBUG: threadid for msgid '<20010207001313.30401.qmail@nym.alias.net>' is
DEBUG: threadid for msgid '<7daeb90ada844104ea268ca9b8bc25dd@mix2.hyperreal. pl>'
is
DEBUG: threadid for msgid '<mm0g6.140$Co5.19947@nntp2.onemain.com>' is
DEBUG: threadid for msgid '<8270e04f916966896207f88be635ef2b@anon.xg.nu>' is
DEBUG: threadid for msgid '<20010207000811.27948.qmail@nym.alias.net>' is
DEBUG: threadid for msgid '<Ek0g6.225629$w35.39920301@news1.rdc1.nj.home.com >' i
s
DEBUG: threadid for msgid '<Nk0g6.1085$D3.4198@tor-nn1.netcom.ca>' is
DEBUG: threadid for msgid '<ih0g6.129$0W5.38111@nntp3.onemain.com>' is
DEBUG: threadid for msgid '<Cc0g6.124$0W5.35089@nntp3.onemain.com>' is
DEBUG: threadid for msgid '<95q1pn$4id$1@lacerta.tiscalinet.it>' is
DEBUG: threadid for msgid '<JW%f6.118$0W5.21387@nntp3.onemain.com>' is
DEBUG: threadid for msgid '<S_%f6.127$Co5.9928@nntp2.onemain.com>' is
DEBUG: threadid for msgid '<A30g6.129$Co5.13100@nntp2.onemain.com>' is
DEBUG: threadid for msgid '<95q2nk$hv28s$1@ID-55073.news.dfncis.de>' is
DEBUG: threadid for msgid '<d80g6.131$Co5.13778@nntp2.onemain.com>' is
DEBUG: threadid for msgid '<3a808f06.949583@news.attcanada.ca>' is
DEBUG: threadid for msgid '<uR%f6.122$Co5.7773@nntp2.onemain.com>' is
DEBUG: threadid for msgid '<nM%f6.111$0W5.13742@nntp3.onemain.com>' is


Lots of these.... ends with...
DEBUG: threadid for msgid '<qk2g6.929$0W5.133222@nntp3.onemain.com>' is
DEBUG: threadid for msgid '<3p2g6.932$0W5.135574@nntp3.onemain.com>' is
DEBUG: threadid for msgid '<Bv2g6.395$Co5.72310@nntp2.onemain.com>' is
Processing outgoing messages


Does this help? I doubt it's the feed (or maybe it is, I dunno) because I'm using newsfeeds.com.