VB Search engine wanted to spider my entire site
 

Hi there,

I am looking for a search engine that could be used to spider my entire site (about 10,000 pages). VB's search engine works well but I would like to expand it to include ALL .htm pages as well as additional php/sql pages.

Is there a hack or way that I could do this? I have tried other search engines but they end up including multiple instances of the same page or inappropraite pages (e.g., profile pages, print pages, etc).

Also, by using one search engine, I could have the same engine on the VB pages as on the rest of my site.

Help would truly be appreciated.

Dave@pets.ca

As a fairly new user of vbulletin I would be interested in any thoughts on this as well. I would like to use it as my home page and if one search could catch the threads as well as my library of documents in word, access, powerpoint, etc. that would be great.

Thanks for any help, suggestions or guidance.

Well, you could try using another search engine to spider your site. There is an old tutorial on DevShed.com I used 2-3 years ago to create a PHP interface for ht://Dig. You can see my version in use at NissanPerformanceMag.com. The forums are on a separate site, so this doesn't search both the "static" content and the vB pages, though I suppose it could.

When I was searching on DevShed.com to dig up the link I came across another article that appears to be much more up to date.

My question, however, is will this approach use the exisitng VB Search? I only want one search engibe for my entire site (VB forums and the rest of the site)

No. I don't know how to modify the vB search to work as a spider so it will search an entire site. That's why I suggested looking into using something else. I used ht://Dig, but because I have my forums under a separate domain it doesn't search both, though I don't see why it isn't possible.