![]() |
Quote:
But I am curious - why do you want to ban this bot? It's one of the better behaved ones out there and it doesn't flood. It obeys robots.txt as well. How can I block MJ12bot? MJ12bot adheres to the robots.txt standard. If you want the bot to prevent website from being crawled then add the following text to your robots.txt: User-agent: MJ12bot Disallow: / |
Quote:
BTW, Max is dead right :) |
Because I don't have use of spiders that no one is using. At least I don't know anyone that uses Majestics.
People usually use Google, Bing, Facebook... Other search engines or spiders don't interest me, as they don't interest my users. Same goes for Baidu or similar search engines. Does people use Majestics? And for what purposes? |
Installed Thanks :)
|
Quote:
It is a friendly, well behaved bot that helps your presence on the web. Baidu is none of these, it is a aggressive, leeching, unfriendly bot attached to a Chinese search engine. There's no comparison between the two. Ban whatever bots you like, no one's telling you not to. But MJ12 isn't hurting you, it's helping you and it is friendly. |
Quote:
|
Thank you for a great mod, Simon.
I want to block Deepnet Explorer Spiders. To do this I enter 'Deepnet Explorer' into the spide list? It's okay to block Deepnet Explorer? |
Blocking spiders is all about personal choice, do a little research and find out whether you want to cater for that country and whether they add value to your site!, when Deepnet Explorer are visiting go to who's online and at the bottom there's a dropdown box for "Show Useragent?" select Yes, then check out their useragent, you can enter any or all of the UA string, so if they actually do have Deepnet in the UA then you just enter that on its own line in the list :)
|
My current (updated) list of banned user agents entered into this Mod:
baiduspider beta.statsit.com statsit SiteIntel Yandex GomezAgent FunWebProducts MSIE 1 MSIE 2 MSIE 3 MSIE 4 MSIE 5 MSIE 6 Nesotebot DCPbot Opera/1 Opera/2 Opera/3 Opera/4 Opera/5 Opera/6 Opera/7 Opera/8 AOL Advertising R&D DataCha0s aiHitBot Apache-HttpClient Zend_Http_Client ReverseGet |
xpymep.exe
start.exe I seen these two as hosts so I added them to the list look strange |
All times are GMT. The time now is 09:08 PM. |
Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information | |
---|---|
|
|
![]() |
|
Template Usage:
Phrase Groups Available:
|
Included Files:
Hooks Called:
|