I started on linux with the mod_auth_sql at this location:
http://sourceforge.net/projects/modauthmysql/ (this uses the crypt function, but vbulletin uses the md5 function - so it's possible to change vbulletin to use crypt... but then all your users would have to reenter new passwords ;( )
I didn't take the time to generalize this - the instructions above use the apxs program to link - but I needed to link in the md5.o library and just built my own shell script to do this. You'd need to make the appropriate changes for your system.
The attached file also include the httpd.conf file that defines the authorization. I started with /is to figure out what mod_auth is all about (use an .htaccess file created by htpasswd), then /is2 was using the instructions from the above, /is3 was a test using md5 passwords, /is4 was the final test using vbulletin tables (it ends up doing a join (result by specifying the fields this weird way)) that ties the group you belong to in vbulletin (you can say "require group Administrator" for example). (P.S. - it would be nice if vbulletin allowed you to belong to multiple groups - maybe another day)
Oh, if you're going to try /is2 - when you create the passwords, you need to "insert into user_info values("uname",encrypt("passwd"));" or for /is3 (md5) "insert into user_info values("uname",md5("passwd"));
Good luck!
Thanks to the original authors and contributors:
* http_auth_msql: authentication
*
* Rob McCool & Brian Behlendorf.
*
* Adapted to Shambhala by rst.
*
* converted to use MySQL by Vivek Khera <khera@kciLink.com>
*
* now maintained by Sam Brauer <sbrauer@users.sourceforge.net>