Quote:
Originally Posted by Alfa1
I am having no success with adding these bots to my blacklist:
Alexa
Artabus
BoardPulse
Deepnet Explorer
Radian6 FeedFetcher
Wget
Cna someone tell me what exactly I need to add to my blacklist.inc?
I see that adding 'Alexa' to my blacklist also blocks users with the alexa toolbar. It seems to me that if I want to allow users with the alexa toolbar installed, I will not be able to block the alexa crawler?
|
I see that alexa, deepnet explorer, radian6 feedfetcher and wget do get blocked, but are also listed on my spiders who visited list. Should that be?