Are they coming at specific times or after you do certain things? I've noticed that immediately after I tweet a link, the bots descend on my site in droves. I have had some problems with particular bots always being around the site and looking like they are up to no good. After doing WHOIS searches on the IP addresses, I decided that for my site I had no reason to continue to allow them to come around and try to access prohibited areas, which is what they were doing. I just banned those particular IPs at the server level. Problem solved.
|