I too plan on tinkering extensively with sphinx once I get my site upgraded to 3.7. I literally could not run my forum search if it wasn't for sphinx. Also I want to tinker with the vB search code itself because I hate how they filter legit results and can sometimes even give "no results found" when in fact results are returned (before being processed through their miles of code that does god knows what for god knows why)...
Yes, people that use the similar threads option, I know vB 3.6 will look at the fulltext index when creating the similar threads so you have some code to modify to redirect it to sphinx. I would assume 3.7 would do the same. If you drop your fulltext indexes you will probably get an error when creating a new thread.
I have to disagree with the person who posted above saying that large vB sites are a small percentage. I think there are a lot more than you realize, but people attack the search problem in different ways. I've seen some people use lots of slave servers with some serious hardware to try and alievate the problem. Some people just disable their search entirely... or use google... I think someone might have hacked together dtsearch too... But I have to agree that Sphinx is the best (and fastest) choice...
I was really hoping that the vB team would implement some ability to use Sphinx with 3.7, but it seems like that request has gone unanswered...
The biggest problem I see when people do a search (not using sphinx), if it doesn't return results within a matter of seconds they start clicking again, and again, and again... which queues up the same search over and over in mysql... It's transparent to the other member too until someone posts a new message, which then it locks the whole table and anyone just wanting to read another thread has to wait... and they start clicking refresh over and over which again starts sending more and more requests to mysql... Eventually the server runs out of memory and things go ape... A serious problem indeed...
Anyhow, I would love to help out and contribute what I can once I get iTrader coded for 3.7 and those people off my back. I *think* I posted some cron scripts, log rotate, initd script, and other stuff on one of the previous pages. If I didn't and someone would like them let me know and I'll post what I have.
I have a cron script that updates the delta every 15min (you can change it to whatever) and once a day (like 5am) it rebuilds the whole indexes.
I also have an initd script (I run redhat) for starting & stopping sphinx. It creates a pid file and all that jazz... It's just a basic script but it works...
Along with the above initd script I wrote a logrotate script, since I keep logs for the searches & sphinx output.
05-07-2008, 01:28 PM
Threaded Mode