A couple of examples of ones in my list that (recently) show as online;
AhrefsBot
Exabot
gigabaz
Heritrix
Majestics MJ12bot
I though this hack picked up on all things similar (name based)?... ie "Majestics" should also take out "Majestics MJ12bot", regardless of the software they're using to access the site.
How do we include same name spiders in our list that are using different UA's?
|