Quote:
Originally Posted by Max Taxable
Harmless crawlers but if you want them blocked you can put them on the list.
|
It's not so much about if they are harmless or not, it's the amount of resources they sometimes use up crawling and the amount of sessions they leave open and the amount of spiders crawling at the same time ....for those 2 in the past I saw 23 different ips for proximic at one time and like 10 for magpie as well .. way too many sessions and spiders to be running around at the same time....