Mysql 4 Search hack
 

MySQL 4 Search hack for Vbulletin 2.x.x - version 1.2 beta
Author: PineappleClock (pineappleclock@clockcrew.cc) aim: Janitor61
-------------------------------------

Overview
The MySQL 4 search hack modifies the search functionality in Vbulletin to take advantage of the new SQL functions available in MySQL 4.

Synopsis
This hack allows MySQL to index the posts and threads in your BBS instead of requiring vbulletin to do it.
This hack, when installed, will change the BBS in the following ways:

• Vbulletin will not require the 'word' or 'searchindex' tables anymore - saving lots of space - instead it will use the 'post' table as an index.
• Vbulletin will not catalog, reference and index every word of every post upon insertion into the database - saving tons of processing.
• No need to regenerate the search index ever.
• The 'post' table will increase in size by about 35% :(
• Searches will be considerably faster, and posting will be somewhat faster.
• Search strings will be able to contain phrases, like "lets share a soda", which can include small words that would of been blocked in vbulletin 2.x searches.
• Advanced searches, containing included words, optional words, excluded words and exact phrases.

File Listing
install.txt -- the installation howto
simplesearch.txt -- a block of code that replaces a block of code in search.php
searchtemplate.txt -- a vbulletin template used for drawing the search form.

Requirements
MySQL 4.0.1 or newer - required
vbulletin 2.x.x - required (i've only tested it on 2.3.0 though)

How to check your MySQL version:
execute the following query:

Notes
The installation instructions assumes that you have access to database management software, like phpMyAdmin. If you can't edit your database directly, I can make a php script that will modify your database if there's enough demand.

The Vbulletin team aren't going to put in support for MySQL 4 until version 3.1 - so this hack should be obsolete by then.
If you'd like to see the modified search engine, go here:

http://www.clockcrew.cc/bbs/search.php

I was originally going to distribute the whole modified search.php file, but after reading the rules, im only distributing the modified part. Please let me know if you spot any bugs, etc. I've tested this, but there still are probably bugs i need to work out.

update [Apr 17]: added instructions on uninstalling

good luck!

Very interesting. What would this do with server loads? Would it increase overall load, or decrease it? I'm impressed.

Looks nice, but searching for an exact phrase doesn't seem to work on your site. I tried a couple of very common words and each time it said there were no results.

This seems a useful hack, somewhat similar to fastforward's (still unreleased).
Thanks for sharing it. :)

BTW, I'm getting a no permission error at http://www.clockcrew.cc/bbs/search.php. :(
I'll try this tomorrow. I'm off to bed now.
Good night folks. :)

try searching for 'break a leg' (no quotes) and put the option 'display results as posts'

Sorry - i had searching off for guests, it's on now.

From what i've seen, it's decreased my server load. They are using this hack over at forums.somethingawful.com as well - with ~ 1,800 concurrent users, and a lot of people there have mentioned that the site runs faster but YMMV.

I dont really have an accurate benchmark on which to gauge server performance though, but i've noticed my server load averages have been around 0.9 to 2.0 with an avg. of 80 members online - i remember them being a lot higher before, up in the 5s. I'm pretty sure the performance boost is since the server dosent have to do INSERT and SELECTs for each word in every new post thats added.

What's with the strange music on your site??!! LOL! :)

It looks VERY impressing indeed! As a matter of fact one of the most promising hacks that has been released for sometime. :)

I wish you could make a comparison before and after applying the hack. For instance if you use Teck's microstats hack, it shows searching performance quite good enough. The bottleneck in the default vb searching pushs badly when:
a) you have a big post database.
b) user searches a few common words at the same time. (For instance you have a DVD forum and user searches word "DVD AND MOVIE")
c) User uses wildcard in this search. eg. "dvd* AND movie*"

This search really kills the server in default vb search algorithm even if you are on dedicated. I have experienced searching taking upto 10-15 seconds in such searches in some large boards and when they have a few of such searches, the server gets on to its knees.

It would be terrific to learn the stats for such searches before and after you applied the hack. I really wonder how much it enhances the server load issue.

This is nothing to worry about! Tables "search" and "searchindex" has always larger than table "post" so when you get rid of them altogether, you'll still save 65% more space in the server even if you have 35% loss in post table.

Since these 3 tables are the largest tables in your database, %65 more space for them means more than %50 gain in your MYSQL space (thus server space) which is incredible! :)

What would happen in a vb2 to vb3 upgrade once this hack is installed?