You will be surprised how many fake google bots are about
Some seo software runs a google view of things and also pretend to be google
Great way of finding your competition sniffing round your site
Something to watch out for and this happend to me last week
Some agents get blocked
I got hit about 1000 times in under an hour
All the guy did was set his scraper to rotate through a lot of agents
It was one of the most elaborate hits I have seen so far
The guy was still hitting while I added his ip to my htaccess and then the fire wall
I have also found that Project Honeypot gets busy and sometimes you might not get an ip blocked thats on their black list
I feel guilty with my war against the morons that hit me all day long with xrummer
All that money they have spent on it, we should happily let them have all our content
|