JimmyN
10-31-2006, 05:22 AM
Needing a new updated version of this mod which allows you to treat search engine robots (i.e. GoogleBot, Yahoo Slurp) as registered users instead of guests, as they are treated by default in VBulletin.
This is handy if your site allows limited access to guests, but full access to registered users. By default, a search engine robot would only be allowed to index what a guest could see, but with this mod, it would allow them to fully index your site as a registered user.
This mod uses the spiders_vbulletin.xml file for determining if a visitor is a bot or not, so make sure you have that file up to date. The one that comes with VB is pretty sparse, but it works. If you're only interested in GoogleBot, then the default file works great.
Original older version: here (https://vborg.vbsupport.ru/showthread.php?t=108070&highlight=bots+as+members)
let me know if you have anyquestions
thanks
This is handy if your site allows limited access to guests, but full access to registered users. By default, a search engine robot would only be allowed to index what a guest could see, but with this mod, it would allow them to fully index your site as a registered user.
This mod uses the spiders_vbulletin.xml file for determining if a visitor is a bot or not, so make sure you have that file up to date. The one that comes with VB is pretty sparse, but it works. If you're only interested in GoogleBot, then the default file works great.
Original older version: here (https://vborg.vbsupport.ru/showthread.php?t=108070&highlight=bots+as+members)
let me know if you have anyquestions
thanks