How to get rid of the Majestics MJ12bot ?
 

On one of the Sites I run i have lately a huge amount of visits of the "Majestics MJ12bot". Any idea how to get rid of it ? I think I already banned a few IP Ranges where he was in but he seems to come back with other IP?s anytime.

As you can see, he is much more there than any other Bot. And though I don?t know if that Bot is bad really, it just concerns me a little bit.

http://www.abload.de/img/damn-bot8e33.jpg

I have a .htaccess File which is filled with a lot of banned IP Ranges already. Anyway I can get rid of the bot with the .htaccess File as well ? I just don?t know which IP/IP Ranges that Bot uses in total.

Anyone experience with that Bot ?

UPDATE:

For some reasons I cant get the image to show it in here as its supposed to be, so here is the direct link to the Screenshot.

The image is quite tiny :D

Well, do you've access to the Webserver that you can install things like mod_security ? Usually blocking IP ranges won't help much if the bot is always coming back and it might end you're blocking legimate members from accessing your board. :(

Dunno if the bot has a special useragent that can be blocked or if the made requests won't differ - you could block based on these or maybe as a much better solution slow each reply down to a maximum like Teergrubing for spam mailservers. :)

I dont have access to the Webserver, its just a Webspace account. .htaccess is the best I can do. And yes I know, banning IP Ranges often hits also the wrong people, but then sometimes its better safe than sorry.

There is only so much one can do on a Webspace.

robots.txt 'nuff said..

Have a read up here buddy, http://www.majestic12.co.uk/projects...ch/mj12bot.php

The IP address is 88.38.211.149, so add that to your IP ban, it does not obey robots.txt and may overload websites.

If you see this fake bot from that IP, please report this abuse to: abusedesk@interbusiness.it and abuse@business.telecomitalia.it - this is their ISP who provides them with upstream connectivity and they were very responsive.

I think if such abuse continues they will cut them off for good.
:up:

With a robots.txt i basically stop all the Bots that I put in it, from going through the site´s content, right ? Someone has a good up to date robots.txt with most of the bad search engines blocked, except maybe Google because I heard they don´t like when you do that. And I want at least Google do the Search stuff.

At that point, use the referrer info in .htaccess. In fact, I'd most likely block all instances of majestic just to hammer the point home.

# deny majestic
SetEnvIfNoCase User-Agent "^MJ12" block

As soon as I put that into my .htaccess File i get a "Internal Server Error 500".

This is, except from the actual IP Ranges I blocked (list would be too long ;) ), how my .htaccess File looks.

Links have been changed in this example but this is how the .htaccess File looks without the actual IP Bans.

Try this

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^MJ12
RewriteRule ^.* - [F,L]

Place that after your "allows".

That other part of Rewrite Engine i have there is for protecting Images not being hotlinked on my Site. Can I use both those parts or does that cause problems ?