PDA

View Full Version : Yandex Crawler


oldengine
04-24-2018, 04:11 AM
The Yandex Russian crawler is trying to crawl me like this...

https://vbulletin.org/forum/index.php/read/72116

Vbulletin.org is showing the same effect as does my board.

The full URL looks like this...

https://vbulletin.org/forum/index.php/read/72116&expand/archive/chat/chat/chat/chat/chat/archive/archive/chat/chat/archive/chat/chat/archive/archive/archive/archive/archive/archive/archive/archive/archive/archive/archive/archive/archive/archive/archive/archive/archive/arc

https://vbulletin.org/forum/index.php/read/72116&expand/archive/chat/chat/chat/chat/chat/archive/archive/chat/chat/archive/chat/chat/archive/archive/archive/archive/archive/archive/archive/archive/archive/archive/archive/archive/archive/archive/archive/archive/archive/arc

Any clue about what they are attempting to accomplish?

Any way to halt this behavior?

IPs
141.8.43.172
52.90.254.43
100.43.90.161
5.255.250.121

Mattwhf
05-05-2018, 11:12 AM
Why you are worrying about Yandex Crawler is crawling your website?

If you want to block Malicious Bots / Crawlers using .htaccess then reading tips in this post

https://forumweb.hosting/9415-block-malicious-bots-crawlers-using-htaccess.html

Hope it helps!