Quote:
Originally Posted by Lee G
For some reason I get zero baidu hits
Either cloudflare block them or they take note of robots txt
Putting ip bans into the cf control panel saves killing my own server with htaccess bans
There seems to be a mass of junk with no user agent hitting from AmazonAWS at present
I need to find out if anything good comes from their servers, then consider a complete ban on all their ips. Any user agent being tracked is littered with AmazonAWS ips
|
Neither happens. I have CF and robots.txt.
AmazonAWS is a friendly spider, from what I've seen over the years.
And Simon - here I am, marked installed.