Usenet gateway

THIS THREAD IS CLOSED!
Posts in here will NOT be answered

This hack has undergone many major changes since this thread was started. Consequently, most of the posts have become dated and of little use. To coincide with the latest major release (20010712), a new one has been started.

See this thread for the latest version and discussion

[fastforward]

DO NOT turn on autopuge. Tamarian has uncovered a serious bug that purges non-usenet groups aswell as the usenet groups. This is due to a left join being used to get the forumids.

The fix will be released ASAP.

[fastforward]

The auto-purge bug is now fixed. It will now wait until all news has been collected and then purge ONLY groups that are in the usenet_group table. The links have been updated to point to the new version 2.7.2.

To upgrade, you only need to replace the newnews.pl script.

[Stephan Whelan]

Just installed 2.7.2 and have to say that it all seems to work great...auto-purge as well.

Congratulations again fastforward!

[TechTalk]

You dont know how bad this is needed in vb 2.1!

[Stephan Whelan]

fastforward,

well it looks like I was too quick in saying that the auto-purge worked.

I am running v2.7.2 of your hack but it seems that the auto-purge still doesn't work the way it's supposed to.

It is currently deleting anything older than 28 days(what I set in the USENET settings) from all forums - not just the usenet ones.

[fastforward]

It's working on mine.

Check the article_purge function in newnews.pl. It's at the bottom starting at line 851.

It should like exactly like this:

sub article_purge {
  my $forums = db_fetch("SELECT forum, newsgroup FROM usenet_group");
  while (my($forumid,$newsgroup) = $forums->fetchrow_array) {
    my $purge = time() - ($config{auto_purge_days} * 86400);
    console("Purging usenet threads older than $config{auto_purge_days} days from $newsgroup...");
    my $threads = db_fetch("SELECT threadid FROM thread WHERE forumid=$forumid AND lastpost < $purge".(!$config{auto_purge_reg}?" AND regpost = 0":""));
    while (my($threadid) = $threads->fetchrow_array) {
      my $posts = db_fetch("SELECT postid FROM post WHERE threadid = $threadid");
      while (my($postid) = $posts->fetchrow_array) {
        db_execute("DELETE LOW_PRIORITY FROM searchindex WHERE postid = $postid");
      } 
      db_execute("DELETE LOW_PRIORITY FROM post WHERE threadid=$threadid");
    }
    db_execute("DELETE LOW_PRIORITY FROM thread WHERE forumid = $forumid AND lastpost < $purge");
    console(" OK\n");
  }
  console("Purge completed.\n");
}

[Stephan Whelan]

Here is the section in my newnews.pl...doesn't seem to be exactly the same...

sub article_purge {
my $forums = db_fetch("SELECT forum, newsgroup FROM usenet_group");
while (my($forumid,$newsgroup) = $forums->fetchrow_array) {
my $purge = time() - ($config{auto_purge_days} * 86400);
console("Purging usenet threads older than $config{auto_purge_days} days from $newsgroup...");
my $threads = db_fetch("SELECT threadid FROM thread WHERE forumid=$forumid AND lastpost < $purge".(!$config{auto_purge_reg}?" AND regpost = 0":""));
while (my($threadid) = $threads->fetchrow_array) {
my $posts = db_fetch("SELECT postid FROM post WHERE threadid = $threadid");
while (my($postid) = $posts->fetchrow_array) {
db_execute("DELETE LOW_PRIORITY FROM searchindex WHERE postid = $postid");
}
db_execute("DELETE LOW_PRIORITY FROM post WHERE threadid=$threadid");
db_execute("DELETE LOW_PRIORITY FROM thread WHERE forumid = $forumid AND lastpost < $purge");
}
console(" OK\n");
}
console("Purge completed.\n");

I'm going to change it to your code.

[fastforward]

You must have downloaded that within seconds of me uploading the version!

I replaced it within 2 or 3 minutes of initial upload.

Sorry about that. I didn't think anyone had got to it first. Did you lose any posts?

[fastforward]

This release just updates things for vB 2.0 RC1. There were very few changes made. The main changes were the control panel, to make it look like the new vB style, and the line numbers in the documentation.

Download latest version for vB 2.0 RC1

The latest version of this hack will always be in this first post of the thread.

[Stephan Whelan]

fastforward,

OK - it now is working fine - that will teach me to be too quick at downloading updates!! I didn't lose too much as we only changed to vB from UBB in the last month so just re-imported old posts from the old forums.

Anyway, I do have one question though (I feel you'll learn to hate me ) :

- How long does it take you to process the newsfeeds. I have 12 newsgroups running and I have to set my cron schedule to every 2 hours (I might put it back to every 3 hours) as it takes on average around 80 minutes to complete a run at processing all posts. I have around 500 posts in the usenet_article table and the expiry length is set to 2 days in my CP. I'm not to fussed in some respects, but I am a bit concerned that there is almost a constant load on the server...

What are other peoples experiences with this?