View Full Version : Search Bots
Milesian
09-14-2004, 09:34 AM
As yet, only MSN has spidered my forum.
On my old phpBB2 forum, Google automatically spidered it but no sign of it yet on my vBulletin 3.0.3 forum.
Is there something I need to do to encourage it or simply wait?
Did you install the board on a new domain, or on the same where you had the phpbb2 messageboard? And when was the last time a spider visited your site?
These are my 'Robots/Spiders visitors' stat:
MSNBot (hits: 5593) - (last: 13 Sep 2004 - 23:48)
Inktomi Slurp (hits: 900) - (last: 14 Sep 2004 - 00:07)
Unknown robot (identified by 'crawl') (hits: 181) - (last: 09 Sep 2004 - 07:09)
MSIECrawler (hits: 82) - (last: 13 Sep 2004 - 05:32)
Googlebot (Google) (hits: 63) - (last: 13 Sep 2004 - 22:02)
Jeeves (hits: 34) - (last: 04 Sep 2004 - 09:33)
Alexa (IA Archiver) (hits: 15) - (last: 05 Sep 2004 - 02:11)
Turn It In (hits: 10) - (last: Sep 2004 - 02:39)
GigaBot (hits: 6) - (last: 13 Sep 2004 - 05:32)
arks (hits: 3) - (last: 07 Sep 2004 - 12:13)
And don't forget to enable your archive (www.yourdomain.com/archive/), that will help you get indexed by spiders easier
Milesian
09-15-2004, 10:06 AM
Yeah, it's where the old phpBB2 board was located.
I don't have a stats hack installed yet so I'm not sure about the bot statistics. The only one's I've seen so far are MSN bots (I submitted the site to them).
The archive is also enabled.
I guess they just haven't found the forum yet :(
Only my index page can be found on google (and non vbulletin folders and pages), but none of the forumthreads can be found on google. There are vbulletin sites that where all threads are indexed by Google. So I guess it probably has something to do with just being lucky
MPM.com
11-14-2004, 08:35 PM
The same thing on my site, I have used to have lots of listings on google, now only the home page.
TCB where did you find these stats? How is it possible to find out which bot is which?
Thanks
Mark
vBulletin® v3.8.12 by vBS, Copyright ©2000-2025, vBulletin Solutions Inc.