Usenet gateway

THIS THREAD IS CLOSED!
Posts in here will NOT be answered

This hack has undergone many major changes since this thread was started. Consequently, most of the posts have become dated and of little use. To coincide with the latest major release (20010712), a new one has been started.

See this thread for the latest version and discussion

[Gilby]

Originally posted by Dioxin
has nobody an idea to fix my problem?

there are a lot of posts, that aren't fetched by the script and wait for manual insertion.... what's the error?

please help

Posts that don't get inserted are not inserted because the references in the headers of the email do not match any of the message ids of existing posts in the forum. If your newsgroup is also a mailing list, as the one i use it on is, then this will occur more often (due to there being so many crappy mail readers that don't use the same headers). I also use this script on a private news server and those newsgroups usually never have a problem (unless the post before it was a binary post and therefore not inserted into the forum). Do these uninserted posts only occur on posts that are in reply to forum posts? If this is the case, this could be because you have changed your useragent setting in the control panel (but that shouldn't affect too many), or that your newserver that you are using is changing the headers and removing that useragent. If the nesgroup has a lot of binary posts, all posts in responce to that binary posts (and the rest of the thread branching from the binary post) will not be inserted into the forum.

[fastforward]

Originally posted by Gilby

If the nesgroup has a lot of binary posts, all posts in responce to that binary posts (and the rest of the thread branching from the binary post) will not be inserted into the forum.

This problem really needs to be fixed. But without risking the logical flow of a thread, I'm not sure how. Any suggestions?

[Gilby]

Three possibilities are:

1. Have the binary posts be inserted as attachments with the post.
2. Have a table that will keep track of the msgid of the thrown away binary posts and its corresponding thread that it was to go in (the subject too as some threads start with a binary post).
3. Have the first so many characters of a binary post be inserted and cut it off with a message saying that it was a binary post.

Ultimately, the first option is the best one... but that's also the hardest/longest one to implement. That can be combined with implementing html multipart posts as well.

The 2nd and 3rd options are easier to implement and can also be used as options for those that don't want to fill their database with binary files, so implementing that now would get the 'problem' fixed.

I have posting from the board disabled and get a lot of posts waiting for manual insertion. If I had to find a pattern I woud say majority of these posts were cross-posts to groups not present on my board (as in reply to message nr 11111 in rec.group.whatever, which I have and 22222 in alt.group.whatever which I don't). But (there's always but) many many times first and a number of cross-posted follow ups appear in vB and then next one just gets stopped. Some posts are kept on hold simply because first message in the thread wasn't fetched, although message is available on the server and if I reset the counter it will fetch it when forced.

Another bug would be mixup of posts from time to time - I had plenty of moments when full threads and replies to another group were inserted into different thread in different board. This is minor though.

Strangely enough everything worked perfect untill last versions for 2.0 betas (ones with all changes in the code)...
For the last five days I've been purgiung all posts from one of the groups and changing servers, suspecting there might be something weird about INN we have in the office, but pulling from private account and from public server also had the issues described above...

vBulletin 2.0.1 + Usenet hack 20010528

[Gilby]

Originally posted by v0n
Another bug would be mixup of posts from time to time - I had plenty of moments when full threads and replies to another group were inserted into different thread in different board. This is minor though.

I've had that happen... and i thought it might have been my database not being set correctly (probably sql not being executed from upgrade instructions). Some posts and the downline thread would show up in threads that only had the one post and was a forum post in another forum. Occasionally I would get the same post being inserted many times into the same thread (the correct thread). While the last problem may have been a one time thing, I fixed the other bug by making the database have a default value that was not null (in my case I used 'unicyclist') for the msgid field of the post. Since then, I haven't had any problems (that I've seen).

[shri]

Any ideas why I'd be getting the following error messages in the output?

DBD::mysql::db do failed: Duplicate entry 'reasonable' for key 2 at newnews.pl line 628.
DBD::mysql::db do failed: Duplicate entry 'interest' for key 2 at newnews.pl line 628.
DBD::mysql::db do failed: Duplicate entry 'rates' for key 2 at newnews.pl line 628.
DBD::mysql::db do failed: Duplicate entry 'amend' for key 2 at newnews.pl line 628.

Regards,
Shri

Hi

This look a great script on your Forum Fastforward.

I have couple of questions.

How do I execute the file newnews.pl via cron? would it be posible to set up a script so that when a member logs on, it runs the newnews.pl file?(I have also emailed my host this question).

vB code changes, (8) admin/functions.php.
You say to ADD the new code just before the closing brace of the indexpost() on or around line 1022,
My line 1022 is maked in red.
Where do I add your code? Is it at the green marked Add here????


if ($newwords) {
$newwords=trim($newwords);
$insertwords="(NULL,'".str_replace(" ","'),(NULL,'",addslashes($newwords))."')";
$DB_site->query("INSERT IGNORE INTO word (wordid,title) VALUES $insertwords");
$selectwords="title IN('".str_replace(" ","','",addslashes($newwords))."')";
$DB_site->query("INSERT IGNORE INTO searchindex (wordid,postid) SELECT DISTINCT wordid,$postid FROM word WHERE $selectwords");
}

if ($newtitlewords) {
$newtitlewords=trim($newtitlewords);
$insertwords="(NULL,'".str_replace(" ","'),(NULL,'",addslashes($newtitlewords))."') ";
$DB_site->query("INSERT IGNORE INTO word (wordid,title) VALUES $insertwords");
$selectwords="title IN('".str_replace(" ","','",addslashes($newtitlewords))."')";
$DB_site->query("REPLACE INTO searchindex (wordid,postid,intitle) SELECT DISTINCT wordid,$postid,1 FROM word WHERE $selectwords");
Add here????
}
}

// ###################### Start makeforumjump #######################


Thank You for any help that you give.
Mark Thornton

[webhost]

Loaded this morning everything looks good but I have 1 probelm for sure maybe 2. The first is when I run newnews.pl I get this error.

Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, webmaster@realwebhost.net and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log.


Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.

Apache/1.3.19 Server at www.realwebhost.net Port 80

Second I went to control panel made a forum called test and it shows up. Then I went to groups for usernet, I added a new group and the only thing I changed was I pointed it the new forum test. After I save I go to test forum on main page but the only thing that is there is test forum It does not show default newsgroup underneath. Can someone tell me what I doing wrong please. I know it is probably simple.

Thanks Joey
HTTP://forums.realwebhost.net test forum at the bottom

[fastforward]

Originally posted by shri
Any ideas why I'd be getting the following error messages in the output?


DBD::mysql::db do failed: Duplicate entry 'reasonable' for key 2 at newnews.pl line 628.
DBD::mysql::db do failed: Duplicate entry 'interest' for key 2 at newnews.pl line 628.
DBD::mysql::db do failed: Duplicate entry 'rates' for key 2 at newnews.pl line 628.
DBD::mysql::db do failed: Duplicate entry 'amend' for key 2 at newnews.pl line 628.

Regards,
Shri

Looks like you have problems with your word table. This problem has been reported before by people not using the usenet hack. The SQL that runs makes it impossible to try to insert a a duplicate key so this implies the table is somehow corrupt.

Your best bet would be to re-index. If you have a large board and that's not an option, then either;

1) Simply delete those words from the word table.
2) turn off the index option for the news for a single pull.

[fastforward]

Originally posted by markrt
How do I execute the file newnews.pl via cron? would it be posible to set up a script so that when a member logs on, it runs the newnews.pl file?(I have also emailed my host this question).

Possibly. But you'll have to write some sort of hack for vB that calls newnews.pl when someone logs on.

vB code changes, (8) admin/functions.php.
You say to ADD the new code just before the closing brace of the indexpost() on or around line 1022,

Your code shows the addition being made before the closing brace of the if statement. It needs to be just before the closing brace of the indexpost() function. see below.

if ($newtitlewords) {
$newtitlewords=trim($newtitlewords);
$insertwords="(NULL,'".str_replace(" ","'),(NULL,'",addslashes($newtitlewords))."') ";
$DB_site->query("INSERT IGNORE INTO word (wordid,title) VALUES $insertwords");
$selectwords="title IN('".str_replace(" ","','",addslashes($newtitlewords))."')";
$DB_site->query("REPLACE INTO searchindex (wordid,postid,intitle) SELECT DISTINCT wordid,$postid,1 FROM word WHERE $selectwords");
}
Add here????
}