Quote:
Originally Posted by Alfa1
Do not be fooled by the massive number of fake Google bots that are around these days. If the IP/CIDR does not reveal a Google bot then its fake.
Add the following to your whitelist.ini to unblock Google, Baidu Japan and Facebook:
Code:
; Extra GOOGLE ranges
ip[] = "66.102.0.0/20"
ip[] = "173.194.0.0/16"
; Extra Baidu ranges
ip[] = "180.76.0.0/19"
ip[] = "119.63.196.0/19"
; Extra Facebook ranges
ip[] = "69.171.224.0/19"
ip[] = "173.252.64.0/18"
ip[] = "66.220.144.0/20"
Please be so kind to include the above into the next release.
The above CIDR ranges come from checking the whois on all bots that visited my big board while I had BB turned off for testing. The above IP ranges belong to valid spiders which are blocked if this is not added to the whitelist.
Does BB whitelist the IPs of pingdom?
https://www.pingdom.com/rss/probe_servers.xml
Pingdom should be whitelisted
|
The trouble is, you do not know whether something is a true GoogeBot or not as Google do not publish all of their IP ranges for obvious reasons. The only safeguard is to do a reverse dns lookup.