Just an update...

I'm still reloading articles into my forums...(I run 12 newsgroups off vB and some of them are pretty active...)

It looks like i'm almost there and no problems seem to have been encountered yet. I can't tell how much time the Orphaned Check adds to my inserts but i'll do some tests.

Finally, how often are people running cron jobs to update their boards? I was setting mine to around one per hour...

I'm running mine once an hour too. With the new code that ensures all possible replies are inserted at each batch, it's no longer necessary to stay 'ahead' of the newsgroup. (As long as you update more often than your expire date is set to :) )

I've found some more bugs I'm afraid. The main one has to do with the regular expression parsing of the spam filters. I believe I've also fixed the problem Gilby brought up regarding forum generated posts. There was also a small issue that could cause posts to be displayed in the wrong order if a thread contained both usenet posts and forum posts.

The good news is they're all fixed and I've also added an auto purge function.

I'll release it as soon as I've done a bit more testing. In the meantime, if your regular expressions don't appear to work, don't worry, it's not a result of your regex skills. :)

This sounds really interesting, but I dont install hacks because I like to upgrade and we all know its not fun with hacks installed.

My question is, is there any chance this will make it into 2.1? Also would you be willing to let John use the code so they can get this rolling?

I have heard talk about usenet being added before but I wasnt sure when it would take place.

Thanks
~Chris

Sorry, i didn't read the whole thread so please don't kill me if anyone has asked this before:
Is there a way to run the script on a server without having access to telnet and cron? Perhaps triggering the script every time the forums title page is loaded or something like that. i only have FTP access to my webspace account :(
I already contaced the sysob of my server but he said that he won't install the perl script for me :(

like I said earlier, you should open new threads when a new version comes out (from 1.x to 2.x) and have a moderator close the old thread...

- LTdata

options:

1. Get a new host. If you are paying more than $10/month there is no way you should be without telnet access. I think you need to install some perl modules to get it working anyway (see first post) so without telnet access you would not be able to get it to work.

2. Try the telnet emulators out there, try www.tucows.com to see if one will do what you want.

Jelsoft inclusion in 2.1

Unfortunatly i doubt they will do this, mainly because it is a flipping hard script to support - just look at this thread! but it would certainly be cool if they did. I can't wait until the final so I upgrade and then install this hack first thing!

This release adds a couple of new features and fixes a few bugs.
New Features:

• Added auto purge feature. (configurable via control panel)
• Added Gilby's mod to manually review/insert orphan replies (Thanks Gilby)
• Added additional option in spam & replace filters to allow operation in multi-line mode ('s' switch). There should be *nothing* you can't filter out of your messages now!

Bug Fixes:

1. Fixed problem with escaping of spam and replace filters when treated as regular expressions
2. Fixed problem of forum posts being sent without msgid of parent post if parent was also a forum post.
3. Tidied up indexes (again)
4. Fixed display ordering problem when usenet and forum posts are in same thread
5. A few other little things that forgot to write down.

Download version 2.7 for vB 2.0 beta 5

To upgrade from 2.6 you will need to run some SQL statements and make additional code changes. These are detailed in UPGRADE.txt.

You really should open a new thread....why post multiple versions in the same thread????

Because then there would be 20 or 30 different threads floating around. At least this way everything is one place. You can guarantee people would be posting in older threads if there were more than one. If people come across an old thread in a search or something, how do they know there's a new one? They then have to go searching backwards through all the hacks making sure they are reading the latest thread.

As mentioned in frequent posts, the latest version is always in the first post of this thread along with description, screenshots etc.

New readers to the thread will read the first post, see this statement then, if they want, they can read backwards from the end to see what's been happening recently.

Every time i have tried to view the examples or download the file, both http://britishexpats.com and http://www.dbforums.com are down :(

~Chris

eh? When?

According to my internetseer reports, BritishExpats.com has been down once over the past 2 months sometime between 9am and 10am a few days ago.

How often have you tried? Where from and when?

at this rate you are getting very close to running out of possible features to add :)

Im trying both as we speak.

Thats a trace from 24.217.12.87 (home)

From my dedicated box 209.61.188.107 (other end of the country)

About 2 mins after i started writing the above post, both sites became available.

~Chris

There are always new features to be added.

Here are a couple that are on the top of my list:

• Support for multi part messages, so only the text part is inserted into the post, and the html version is tossed.
• Attachments, if from a newsgroup, then it'll make it a regular attachment in vB. If posted on the forums, a link at the end of the newgroup post is given for where people can download it.
• Better text formatting int he newsgroup. (Possibly take the post, and parse the vB code into html and then use the html to text perl module). Good for stuff like lists.

I take it you mean the outgoing posts? This is something I haven't really spent much time on yet. But I agree, it's something that needs improving. I take the easy option at the moment and disable BB code & html for outgoing posts in the forum :)

As for incoming posts; there's not much else we can do with that. The posts that look bad in the forums look just as bad on usenet due to crap news clients or people posting using MS Outlook. I'm sure it wouldn't be too hard to handle the dual format messages correctly though. I'll get on it :)

Yup, I think it'd be cool if the posts that come out of the forum would have good formatting and are better than any other formatting of plain text documents.

Most the html posts that make it on my forums are from AOL users, now that AOL has AOL 6.0 out, they now send html formatted posts instead of plain text out to non-AOL users by default. Of course there is the occasional html post from other people as well.

Hi Fastforward,

When I switched autopurge on, I got an error from newnews.pl, for invalid isreg field?

Looking up the ugrade sql statements, this field was not added. Shoul it have been there, or is it from previous versions?

Sorry. my fault. It's the same in mine but I didn't notice :)

You need to change
isreg
in newnews.pl to
regpost

There is only one reference to it on line 856.

It's referring to a column in the thread table. If it's set to 1, it means the thread has been posted in be a registered member. It gets updated in newthread.php and newreply.php. Those files seem to be OK though, so you only need to change the one occurence in newnews.pl.

Now it purges fine, thanks.

DO NOT turn on autopuge. Tamarian has uncovered a serious bug that purges non-usenet groups aswell as the usenet groups. This is due to a left join being used to get the forumids.

The fix will be released ASAP.

The auto-purge bug is now fixed. It will now wait until all news has been collected and then purge ONLY groups that are in the usenet_group table. The links have been updated to point to the new version 2.7.2.

To upgrade, you only need to replace the newnews.pl script.

Just installed 2.7.2 and have to say that it all seems to work great...auto-purge as well.

Congratulations again fastforward!

:cool:

You dont know how bad this is needed in vb 2.1!

fastforward,

well it looks like I was too quick in saying that the auto-purge worked.

I am running v2.7.2 of your hack but it seems that the auto-purge still doesn't work the way it's supposed to.

It is currently deleting anything older than 28 days(what I set in the USENET settings) from all forums - not just the usenet ones.

It's working on mine.

Check the article_purge function in newnews.pl. It's at the bottom starting at line 851.

It should like exactly like this:

Here is the section in my newnews.pl...doesn't seem to be exactly the same...

I'm going to change it to your code.

You must have downloaded that within seconds of me uploading the version!

I replaced it within 2 or 3 minutes of initial upload.

Sorry about that. I didn't think anyone had got to it first. Did you lose any posts?

This release just updates things for vB 2.0 RC1. There were very few changes made. The main changes were the control panel, to make it look like the new vB style, and the line numbers in the documentation.

Download latest version for vB 2.0 RC1

The latest version of this hack will always be in this first post of the thread.

fastforward,

OK - it now is working fine - that will teach me to be too quick at downloading updates!! I didn't lose too much as we only changed to vB from UBB in the last month so just re-imported old posts from the old forums.

Anyway, I do have one question though (I feel you'll learn to hate me:D ) :

- How long does it take you to process the newsfeeds. I have 12 newsgroups running and I have to set my cron schedule to every 2 hours (I might put it back to every 3 hours) as it takes on average around 80 minutes to complete a run at processing all posts. I have around 500 posts in the usenet_article table and the expiry length is set to 2 days in my CP. I'm not to fussed in some respects, but I am a bit concerned that there is almost a constant load on the server...

What are other peoples experiences with this?

Woooah!!

You have a problem! I pull, process and load 70 groups in 5 minutes! When it was on a shared virtual server it used to take up to 15 minutes.

Even when pulling history with batch limit set to 150 it only takes 10-20 minutes! That's over 10,000 posts.

What version of Perl are you using?

fastforward,

Perl is 5.005_03.

I've watched the server pull newsgroups stuff through and it pulls the articles very quickly, the problem seems to be when it reaches "Checking for Orphans" and "Loading x Replies" stage.

I've followed details on optimising MySQL from various threads in these forums but doesn't seem to help very much.

It isn't a shared server, it is a cobalt raq 4i with 512 RAM.

Since trying to apply this hack our forums have been down with the following error:

Parse error: parse error in /home/sites/site3/web/forums/admin/functions.php on line 1729

Fatal error: Call to undefined function: vbsetcookie() in /home/sites/site3/web/forums/admin/sessions.php on line 198

I followed the directions exactly as specified in the INSTALL.TXT file. Modified the required files and then manually uploaded them to the server.

Any ideas on the problem? A prompt solution would be much appreciated since our boards are down at this moment. :eek:

first- replace functions.php with the backup you made to get forums working again.

then look inside functions.php the error might be beginning here:

function vbsetcookie ...

basically you have done something wrong in fuctions.php, missed of a closing bracket would be my guess

I uploaded the original functions.php and indeed the forum started working again. I then proceeded to go through the process of modifying the functions.php file again double and triple checking eachtime that the changes I made were absolutely correct. However, when I uploaded the file again the error I posted above immediately reappeared.

Maybe someone could attach a copy of their working modified functions.php file so I can take a look?

i havn't done it yet so can't help.

You are using the correct version of vb for the hack, release candidate 1. I could not tell from your forums as the version number is not there.

I believe this may be your problem. The email::find module requires version 5.005_63 to work at it's optimum. Although having said that I was using it with and older version with no problems. Here is what the module author states:
To confirm my suspicion, try commenting out line 828 of newnews.pl.
It currently reads
It won't make any difference to the functionality unless you are importing usenet users.

Let me know what happens.

This indicates there is a brace, semi colon or some other syntactical error. I'll have a look at my instructions again, although it worked when I did mine using my own instructions.

I've checked the instructions and it works for me. Not what you wanted to hear but it does :)

You might want to re-check the two edits in the deletethread() and deletepost() functions.

The edits replace the closing brace with another line of code AND a closing brace. Make sure you don't have two closing braces in there.

Thanks your solution worked for me. :D I created a new category forum called "usenet". Created the groups I wished to display on our VB. Ran the perl script, it connected fine, reported downloading all the messages fine, and disconnected fine. However, when I view our forum homepage the newsgroups are not showing up at all.

Should the newsgroup automatically appear under the chosen category when it's created in admin?

Any ideas?