Quote:
Originally Posted by Alfa1
It makes me wonder about the capability of google to efficiently spider large boards.
|
Google uses artificial intelligence to spider pages, and may be up to 300 times more efficient than other spiders, using traditional algorithms. This efficiency is actually the main reason why Google needs so few bots compared to many other spiders. No matter what however, the bandwidth is the same