Quote:
Originally Posted by Videx
FYI, the "official" spiders.xml list is here.
|
Thanks. I know about that list but it's overkill for me. I don't need to see every spider in the world - just the major search engine spiders, which are included with the vBulletin spiders.xml file.
The point of this mod was to update the standard file to include Bingbot which is now crawling.