Quote:
Originally Posted by SillyAlien
TosaInu, how did you make out with the re-indexing?
|
Hello SillyAlien,
It's not completed yet. I first used the default setting of 250 items per batch, and some 600,000 posts were indexed within a few hours. I left it running during the night and it got at some 700,000. Probably stuck. I let it run for the rest of the day (12+ hours) and it was still there. So, the PHP script probably timed out. The forum was running all the time and no glitches were reported from any member. I'm sure it would be said if there was any.
I launched it again at 50 per batch, and the script hung again, followed by a crash of the browser. There are some 720,000 items in the searchlog now. That's far from completed.
The problem is that several single posts on our forum are multi page essays. A few years ago, the PHP limit was set so that one of them could be just posted at a time without crashing PHP. The PHP settings are increased a few notches now, but it won't be able to process 50 at the same time. Setting the batch to 1 at a time, and assuming 1 second for a batch cyclus, it will take 12 days to hit the first million. So, I'll have to tweak to get it done.
There are 700,000 records now and I searched for a word there: results come up within 2 seconds, so performance isn't bad (for us).
I would say it's a decent, good or even very good implementation. The built script is a bottleneck when you have a large site and big items.
I'm guessing memory and timeouts is the problem for me. Would it be possible to run the build script from the commandline? Will that prevent issues?
02-03-2010, 10:18 AM
Hybrid Mode
