seach engines

[Lionel]

Yesterday I was watching who's online. As I log out as one user and loggged back in as another, I saw the previous user logged in with a different IP. Today, just now, I am watching an IP following everybody's last move and doing the same. A trace IP revealed it was a Google Crawler. It is still following the last member to have an activity to display the same activity. Is that something new? Is that how search engines behave now?

[mheinemann]

Do you have Google Adsense on your site?

[Johnny]

doesnt matter if you have adsense or not, google or any other search engine just finally caught up with your site so their just spidering your site to be searched. its not new but really a bad thing since they do drain bandwidth alot

[mheinemann]

I mentioned the AdSense thing because it will crawl a page when a user visits that page to display relevent ads. I see it on my site all the time.
If you have access logs, you can tell if it's an AdSense crawl or a regular crawl by looking at the user agent.
AdSense is "Mediapartners-Google/2.1"
Regular google is "Googlebot/2.1 (+http://www.google.com/bot.html)"

[Lionel]

It so happened that I installed ad sense 2 days ago. Makes sense! :-)
Google and others have been invading for a while before... So I believe it is from AdSense.

[Michael Morris]

Whatever bandwidth spiders drain off is insignificant next to the value they provide in indexing your site for new visitors to find you.

[Natch]

There is an updated list of the robots and their user-agents you can find it on vbulletin.com: you paste the values into your Who's Onlin esection in your AdminCP, then you can tell who all the spiders are...