How to get rid of the Majestics MJ12bot ?

[tlwwolfseye]

On one of the Sites I run i have lately a huge amount of visits of the "Majestics MJ12bot". Any idea how to get rid of it ? I think I already banned a few IP Ranges where he was in but he seems to come back with other IP?s anytime.

As you can see, he is much more there than any other Bot. And though I don?t know if that Bot is bad really, it just concerns me a little bit.

http://www.abload.de/img/damn-bot8e33.jpg

I have a .htaccess File which is filled with a lot of banned IP Ranges already. Anyway I can get rid of the bot with the .htaccess File as well ? I just don?t know which IP/IP Ranges that Bot uses in total.

Anyone experience with that Bot ?

UPDATE:

For some reasons I cant get the image to show it in here as its supposed to be, so here is the direct link to the Screenshot.

[Angel-Wings]

The image is quite tiny :D

Well, do you've access to the Webserver that you can install things like mod_security ? Usually blocking IP ranges won't help much if the bot is always coming back and it might end you're blocking legimate members from accessing your board. :(

Dunno if the bot has a special useragent that can be blocked or if the made requests won't differ - you could block based on these or maybe as a much better solution slow each reply down to a maximum like Teergrubing for spam mailservers. :)

[tlwwolfseye]

I dont have access to the Webserver, its just a Webspace account. .htaccess is the best I can do. And yes I know, banning IP Ranges often hits also the wrong people, but then sometimes its better safe than sorry.

There is only so much one can do on a Webspace.

[Biker_GA]

robots.txt 'nuff said..

[UKBusinessLive]

Have a read up here buddy, http://www.majestic12.co.uk/projects/dsearch/mj12bot.php

The IP address is 88.38.211.149, so add that to your IP ban, it does not obey robots.txt and may overload websites.

If you see this fake bot from that IP, please report this abuse to: abusedesk@interbusiness.it and abuse@business.telecomitalia.it - this is their ISP who provides them with upstream connectivity and they were very responsive.

I think if such abuse continues they will cut them off for good.
:up:

[tlwwolfseye]

With a robots.txt i basically stop all the Bots that I put in it, from going through the site´s content, right ? Someone has a good up to date robots.txt with most of the bad search engines blocked, except maybe Google because I heard they don´t like when you do that. And I want at least Google do the Search stuff.

[Biker_GA]

At that point, use the referrer info in .htaccess. In fact, I'd most likely block all instances of majestic just to hammer the point home.

# deny majestic
SetEnvIfNoCase User-Agent "^MJ12" block

[tlwwolfseye]

# deny majestic
SetEnvIfNoCase User-Agent "^MJ12" block

As soon as I put that into my .htaccess File i get a "Internal Server Error 500".

This is, except from the actual IP Ranges I blocked (list would be too long ;) ), how my .htaccess File looks.


php_value memory_limit 64M
php_value upload_max_filesize 50M
php_value max_execution_time 900

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http(s)?://(www\.)?mydomain.com [NC]
RewriteRule \.(jpg|jpeg|png|gif)$ http://www.myotherdomain.com/hotlinkxutp.jpg [NC,R,L]

ErrorDocument 403 /error/403.html
ErrorDocument 404 /error/403.html

<Files .htaccess>
order allow,deny
deny from all
</Files>

Links have been changed in this example but this is how the .htaccess File looks without the actual IP Bans.

[Biker_GA]

Try this

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^MJ12
RewriteRule ^.* - [F,L]

Place that after your "allows".

[tlwwolfseye]

That other part of Rewrite Engine i have there is for protecting Images not being hotlinked on my Site. Can I use both those parts or does that cause problems ?

[Biker_GA]

Should be able to just add the applicable lines to that section. :)

[tlwwolfseye]

Found that on a Website where someone posted a list of Bad Bots. The only difference in the syntax i see with yours and this is that he has the [NC, also before OR] at the end of each line. Does that make a difference or would I have just to adapt his list the way you posted that above ?


RewriteEngine On
RewriteBase /

RewriteCond %{HTTP_USER_AGENT} ^(aesop_com_spiderman|alexibot|backweb|bandit|batc hftp|bigfoot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(black.?hole|blackwidow|blowfish|botalot|buddy|bu iltbottough|bullseye) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(cheesebot|cherrypicker|chinaclaw|collector|copie r|copyrightcheck) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(cosmos|crescent|curl|custo|da|diibot|disco|ditto spyder|dragonfly) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(drip|easydl|ebingbong|ecatch|eirgrabber|emailcol lector|emailsiphon) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(emailwolf|erocrawler|exabot|eyenetie|filehound|f lashget|flunky) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(frontpage|getright|getweb|go.?zilla|go-ahead-got-it|gotit|grabnet) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(grafula|harvest|hloader|hmview|httplib|httrack|h umanlinks|ilsebot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(infonavirobot|infotekies|intelliseek|interget|ir ia|jennybot|jetcar) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(joc|justview|jyxobot|kenjin|keyword|larbin|leech ftp|lexibot|lftp|libweb) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(likse|linkscan|linkwalker|lnspiderguy|lwp|magnet |mag-net|markwatch) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(mata.?hari|memo|microsoft.?url|midown.?tool|miix pc|mirror|missigua) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(mister.?pix|moget|mozilla.?newt|nameprotect|navr oad|backdoorbot|nearsite) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(net.?vampire|netants|netcraft|netmechanic|netspi der|nextgensearchbot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(attach|nicerspro|nimblecrawler|npbot|octopus|off line.?explorer) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(offline.?navigator|openfind|outfoxbot|pagegrabbe r|papa|pavuk) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(pcbrowser|php.?version.?tracker|pockey|propowerb ot|prowebwalker) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(psbot|pump|queryn|recorder|realdownload|reaper|r eget|true_robot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(repomonkey|rma|internetseer|sitesnagger|siphon|s lysearch|smartdownload) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(snake|snapbot|snoopy|sogou|spacebison|spankbot|s panner|sqworm|superbot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(superhttp|surfbot|asterias|suzuran|szukacz|takeo ut|teleport) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(telesoft|the.?intraformant|thenomad|tighttwatbot |titan|urldispatcher) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(turingos|turnitinbot|urly.?warning|vacuum|vci|vo ideye|whacker) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(libwww-perl|widow|wisenutbot|wwwoffle|xaldon|xenu|zeus|zy borg|anonymouse) [NC,OR]



--------------- Added 1243606963 at 1243606963 ---------------

Also tried this one.


RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^$|\<|\>|\'|\%|\_iRc|\_Works|\@\$x|\<\?|\$x0e|\+select\+|\+union\+|1\,\1\,1\,|2icommerc e|3GSE|4all|59\.64\.153\.|88\.0\.106\.|98|85\.17\. |A\_Browser|ABAC|Abont|abot|Accept|Access|Accoo|Ac eFTP|Acme|ActiveTouristBot|Address|Adopt|adress|ad ressendeutschland|ADSARobot|agent|ah\-ha|Ahead|AESOP\_com\_SpiderMan|aipbot|Alarm|Albert |Alek|Alexibot|Alligator|AllSubmitter|alma|almaden |ALot|Alpha|aktuelles|Akregat|Amfi|amzn\_assoc|Ana l|Anarchie|andit|Anon|AnotherBot|Ansearch|AnswerBu s|antivirx|Apexoo|appie|Aqua_Products|Arachmo|arch ive|arian|ASPSe|ASSORT|aster|Atari|ATHENS|AtHome|A tlocal|Atomic_Email_Hunter|Atomz|Atrop|^attach|att rib|autoemailspider|autohttp|axod|batch|b2w|Back|B ackDoorBot|BackStreet|BackWeb|Badass|Baid|Bali|Ban dit|Baidu|Barry|BasicHTTP|BatchFTP|bdfetch|beat|Be come|Beij|BenchMark|berts|bew|big.brother|Bigfoot| Bilgi|Bison|Bitacle|Biz360|Black|Black.Hole|BlackW idow|bladder.fusion|Blaiz|Blog.Checker|Blogl|BlogP eople|Blogshares.Spiders|Bloodhound|Blow|bmclient| Board|BOI|boitho|Bond|Bookmark.search.tool|boris|B ost|Boston.Project|BotRightHere|Bot.mailto:craftbo t@yahoo.com|BotALot|botpaidtoclick|botw|brandwatch |BravoBrian|Brok|Bropwers|Broth|browseabit|BrowseX |Browsezilla|Bruin|bsalsa|Buddy|Build|Built|Bulls| bumblebee|Bunny|Busca|Busi|Buy|bwh3|c\-spider|CafeK|Cafi|camel|Cand|captu|Catch|cd34|Ceg| CFNetwork|cgichk|Cha0s|Chang|chaos|Char|char\(32\, 35\)|charlotte|CheeseBot|Chek|CherryPicker|chill|C hinaClaw|CICC|Cisco|Cita|Clam|Claw|Click.Bot|clipp ing|clshttp|Clush|COAST|ColdFusion|Coll|Comb|comme ntreader|Compan|contact|Control|contype|Conc|Conv| Copernic|Copi|Copy|Coral|Corn|core-project|cosmos|costa|cr4nk|crank|craft|Crap|Crawle r0|Crazy|Cres|cs\-CZ|cuill|Curl|Custo|Cute|CSHttp|Cyber|cyberalert|^ DA$|daoBot|DARK|Data|Daten|Daum|dcbot|dcs|Deep|Dep S|Detect|Deweb|Diam|Digger|Digimarc|digout4uagent| DIIbot|Dillo|Ding|DISC|discobot|Disp|Ditto|DLC|Dnl oadMage|DotBot|Doubanbot|Download|Download.Demon|D ownload.Devil|Download.Wonder|Downloader|drag|Drea mPassport|Drec|Drip|dsdl|dsok|DSurf|DTAAgent|DTS|D ual|dumb|DynaWeb|e\-collector|eag|earn|EARTHCOM|EasyDL|ebin|EBM-APPLE|EBrowse|eCatch|echo|ecollector|Edco|edgeio|e fp\@gmx\.net|EirGrabber|email|Email.Extractor|Emai lCollector|EmailSearch|EmailSiphon|EmailWolf|Emer| empas|Enfi|Enhan|Enterprise\_Search|envolk|erck|Er oCr|ESurf|Eval|Evil|Evere|EWH|Exabot|Exact|EXPLOIT ER|Expre|Extra|ExtractorPro|EyeN|FairAd|Fake|FANG| FAST|fastlwspider|FavOrg|Favorites.Sweeper|Faxo|FD M\_1|FDSE|fetch|FEZhead|Filan|FileHound|find|Fireb at|Firefox.2\.0|Firs|Flam|Flash|FlickBot|Flip|fluf fy|flunky|focus|Foob|Fooky|Forex|Forum|ForV|Fost|F oto|Foun|Franklin.Locator|freefind|FreshDownload|F rontPage|FSurf|++++|Fuer|futile|Fyber|Gais|GalaxyB ot|Galbot|Gamespy\_Arcade|GbPl|Gener|geni|Geona|Ge t|gigabaz|Gira|Ginxbot|gluc|glx.?v|gnome|Go.Zilla| Goldfire|Google.Wireless.Transcoder|Googlebot\-Image|Got\-It|GOFORIT|gonzo|GornKer|GoSearch|^gotit$|gozilla| grab|Grabber|GrabNet|Grub|Grup|Graf|Green.Research |grub|grub\-client|gsa\-cra|GSearch|GT\:\:WWW|GuideBot|guruji|gvfs|Gyps|ha ck|haha|hailo|Harv|Hatena|Hax|Head|Helm|herit|hgre |hhjhj\@yahoo|Hippo|hloader|HMView|holm|holy|HomeP ageSearch|HooWWWer|HouxouCrawler|HMSE|HPPrint|htdi g|HTTPConnect|httpdown|http.generic|HTTPGet|httpli b|HTTPRetriever|HTTrack|human|Huron|hverify|Hybrid |Hyper|ia\_archiver|iaskspi|IBM\_Planetwide|iCCra| ichiro|ID\-Search|IDA|IDBot|IEAuto|IEMPT|iexplore\.exe|iGette r|Ilse|Iltrov|Image|Image.Stripper|Image.Sucker|im agefetch|iimds\_monitor|Incutio|IncyWincy|Indexer| Industry.Program|Indy|InetURL|informant|InfoNav|In foTekies|Ingelin|Innerpr|Inspect|InstallShield.Dig italWizard|Insuran\.|Intellig|Intelliseek|InterGET |Internet.Ninja|Internet.x|Internet\_Explorer|Inte rnetLinkagent|InternetSeer.com|Intraf|IP2|Ipsel|Ir ia|IRLbot|Iron33|Irvine|ISC\_Sys|iSilo|ISRCCrawler |ISSpi|IUPUI.Research.Bot|Jady|Jaka|Jam|^Java|java \/|Java\(tm\)|JBH.agent|Jenny|JetB|JetC|jeteye|jiro| JoBo|JOC|jupit|Just|Jyx|Kapere|kash|Kazo|KBee|Kenj in|Kernel|Keywo|KFSW|KKma|Know|kosmix|KRAE|KRetrie ve|Krug|ksibot|ksoap|Kum|KWebGet|Lachesis|lanshan| Lapo|larbin|leacher|leech|LeechFTP|LeechGet|leipzi g\.de|Lets|Lexi|lftp|Libby|libcrawl|libcurl|libfet ch|libghttp|libWeb|libwhisker|libwww|libwww\-FM|libwww\-perl|LightningDownload|likse|Linc|Link|Link.Sleuth |LinkextractorPro|Linkie|LINKS.ARoMATIZED|LinkScan |linktiger|LinkWalker|Lint|List|lmcrawler|LMQ|LNSp iderguy|loader|LocalcomBot|Locu|London|lone|looksm art|loop|Lork|LTH\_|lwp\-request|LWP|lwp-request|lwp-trivial|Mac.Finder|Macintosh\;.I\;.PPC|Mac\_F|magi |Mag\-Net|Magnet|Magp|Mail.Sweeper|main|majest|Mam|Mana| MarcoPolo|mark.blonin|MarkWatch|MaSagool|Mass|Mass .Downloader|Mata|mavi|McBot|Mecha|MCspider|mediapa rtners|^Memo|MEGAUPLOAD|MetaProducts.Download.Expr ess|Metaspin|Mete|Microsoft.Data.Access|Microsoft. URL|Microsoft\_Internet\_Explorer|MIDo|MIIx|miner| Mira|MIRE|Mirror|Miss|Missauga|Missigua.Locator|Mi ssouri.College.Browse|Mist|Mizz|MJ12|mkdb|mlbot|ML M|MMMoCrawl|MnoG|moge|Moje|Monster|Monza.Browser|M ooz|Moreoverbot|MOT\-MPx220|mothra\/netscan|mouse|MovableType|Mozdex|Mozi\!|^Mozilla$| Mozilla\/1\.22|Mozilla\/22|^Mozilla\/3\.0.\(compatible|Mozilla\/3\.Mozilla\/2\.01|Mozilla\/4\.0\(compatible|Mozilla\/4\.08|Mozilla\/4\.61.\(Macintosh|Mozilla\/5\.0|Mozilla\/7\.0|Mozilla\/8|Mozilla\/9|Mozilla\:|Mozilla\/Firefox|^Mozilla.*Indy|^Mozilla.*NEWT|^Mozilla*MSI ECrawler|Mp3Bot|MPF|MRA|MS.FrontPage|MS.?Search|MS FrontPage|MSIE\_6\.0|MSIE6|MSIECrawler|msnbot\-media|msnbot\-Products|MSNPTC|MSProxy|MSRBOT|multithreaddb|musc| MVAC|MWM|My\_age|MyApp|MyDog|MyEng|MyFamilyBot|MyG etRight|MyIE2|mysearch|myurl|NAG|NAMEPROTECT|NASA. Search|nationaldirectory|Naver|Navr|Near|NetAnts|n etattache|Netcach|NetCarta|Netcraft|NetCrawl|NetMe ch|netprospector|NetResearchServer|NetSp|Net.Vampi re|netX|NetZ|Neut|newLISP|NewsGatorInbox|NEWT|NEWT .ActiveX|Next|^NG|NICE|nikto|Nimb|Ninja|Ninte|NIPG Crawler|Noga|nogo|Noko|Nomad|Norb|noxtrumbot|NPbot |NuSe|Nutch|Nutex|NWSp|Obje|Ocel|Octo|ODI3|oegp|Of fline|Offline.Explorer|Offline.Navigator|OK.Mozill a|omg|Omni|Onfo|onyx|OpaL|OpenBot|Openf|OpenTextSi teCrawler|OpenU|Orac|OrangeBot|Orbit|Oreg|osis|Out f|Owl|P3P|PackRat|PageGrabber|PagmIEDownload|pansc i|Papa|Pars|Patw|pavu|Pb2Pb|pcBrow|PEAR|PEER|PECL| pepe|Perl|PerMan|PersonaPilot|Persuader|petit|PHP| PHP.vers|PHPot|Phras|PicaLo|Piff|Pige|pigs|^Ping|P ingd|PingALink|Pipe|Plag|Plant|playstarmusic|Pluck |Pockey|POE\-Com|Poirot|Pomp|Port.Huron|Post|powerset|Preload|p ress|Privoxy|Probe|Program.Shareware|Progressive.D ownload|ProPowerBot|prospector|Provider.Protocol.D iscover|ProWebWalker|Prowl|Proxy|Prozilla|psbot|PS urf|psycheclone|^puf$|Pulse|Pump|PushSite|PussyCat |PuxaRapido|PycURL|Pyth|PyQ|QuepasaCreep|Query|Que st|QRVA|Qweer|radian|Radiation|Rambler|RAMP|RealDo wnload|Reap|Recorder|RedCarpet|RedKernel|ReGet|rel evantnoise|replacer|Repo|requ|Rese|Retrieve|Rip|Ri x|RMA|Roboz|Rogue|Rover|RPT\-HTTP|Rsync|RTG30|.ru\)|ruby|Rufus|Salt|Sample|SAPO |Sauger|savvy|SBIder|SBP|SCAgent|scan|SCEJ\_|Sched |Schizo|Schlong|Schmo|Scout|Scooter|Scorp|ScoutOut |SCrawl|screen|script|SearchExpress|searchhippo|Se archme|searchpreview|searchterms|Second.Street.Res earch|Security.Kol|Seekbot|Seeker|Sega|Sensis|Sept |Serious|Sezn|Shai|Share|Sharp|Shaz|shell|shelo|Sh erl|Shim|Shiretoko|ShopWiki|SickleBot|Simple|Siph| sitecheck|SiteCrawler|SiteSnagger|Site.Sniper|Site Sucker|sitevigil|SiteX|Sleip|Slide|Slurpy.Verifier |Sly|Smag|SmartDownload|Smurf|sna\-|snag|Snake|Snapbot|Snip|Snoop|So\-net|SocSci|sogou|Sohu|solr|sootle|Soso|SpaceBison| Spad|Span|spanner|Speed|Spegla|Sphere|Sphider|spid er|SpiderBot|SpiderEngine|SpiderView|Spin|sproose| Spurl|Spyder|Squi|SQ.Webscanner|sqwid|Sqworm|SSM\_ Ag|Stack|Stamina|stamp|Stanford|Statbot|State|Stee l|Strateg|Stress|Strip|studybot|Style|subot|Suck|S ume|sun4m|Sunrise|SuperBot|SuperBro|Supervi|Surf4M e|SuperHTTP|Surfbot|SurfWalker|Susi|suza|suzu|Swee p|sygol|syncrisis|Systems|Szukacz|Tagger|Tagyu|tAk e|Talkro|TALWinHttpClient|tamu|Tandem|Tarantula|ta rspider|tBot|TCF|Tcs\/1|TeamSoft|Tecomi|Teleport|Telesoft|Templeton|Tenc ent|Terrawiz|Test|TexNut|trivial|Turnitin|The.Intr aformant|TheNomad|Thomas|TightTwatBot|Timely|Titan |TMCrawler|TMhtload|toCrawl|Todobr|Tongco|topic|To rrent|Track|translate|Traveler|TREEVIEW|True|Tunne l|turing|Turnitin|TutorGig|TV33\_Mercator|Twat|Twe ak|Twice|Twisted.PageGetter|Tygo|ubee|UCmore|UdmSe arch|UIowaCrawler|Ultraseek|UMBC|unf|UniversalFeed Parser|unknown|UPG1|UtilMind|URLBase|URL.Control|U RL\_Spider\_Pro|urldispatcher|URLGetFile|urllib|UR LSpiderPro|URLy|User\-Agent|UserAgent|USyd|Vacuum|vagabo|Valet|Valid|Vam p|vayala|VB\_|VCI|VERI\~LI|verif|versus|via|Viewer |virtual|visibilitygap|Visual|vobsub|Void|VoilaBot |voyager|vspider|VSyn|w\:PACBHO60|w0000t|W3C|w3m|w 3search|walhello|Walker|Wand|WAOL|WAPT|Watch|Wavef ire|wbdbot|Weather|web.by.mail|Web.Data.Extractor| Web.Downloader|Web.Ima|Web.Mole|Web.Sucker|Web2Mal |Web2WAP|WebaltBot|WebAuto|WebBandit|Webbot|WebCap ture|WebCat|webcraft\@bea|Webclip|webcollage|WebCo llector|WebCopier|WebCopy|WebCor|webcrawl|WebDat|W ebDav|webdevil|webdownloader|Webdup|WebEMail|WebEM ailExtrac|WebEnhancer|WebFetch|WebGo|WebHook|Webin ator|WebInd|webitpr|WebFilter|WebFountain|WebLea|W ebmaster|WebmasterWorldForumBot|WebMin|WebMirror|w ebmole|webpic|WebPin|WebPix|WebReaper|WebRipper|We bRobot|WebSauger|WebSite|Website.eXtractor|Website .Quester|WebSnake|webspider|Webster|WebStripper|we bsucker|WebTre|WebVac|webwalk|WebWasher|WebWeasel| WebWhacker|WebZIP|Wells|WEP\_S|WEP.Search.00|WeRel ateBot|wget|Whack|Whacker|whiz|WhosTalking|Widow|W in67|window.location|Windows.95\;|Windows.95\)|Win dows.98\;|Windows.98\)|Winodws|Wildsoft.Surfer|Win HT|winhttp|WinHttpRequest|WinHTTrack|Winnie.Poh|wi re|WISEbot|wisenutbot|wish|Wizz|WordP|Works|world| WUMPUS|Wweb|WWWC|WWWOFFLE|WWW\-Collector|WWW.Mechanize|www.ranks.nl|wwwster|^x$|X 12R1|x\-Tractor|Xaldon|Xenu|XGET|xirq|Y\!OASIS|Y\!Tunnel|y acy|YaDirectBot|Yahoo\-MMAudVid|YahooSeeker|YahooYSMcm|Yamm|Yand|yang|Yet i|Yoono|yori|Yotta|YTunnel|Zade|zagre|ZBot|Zeal|Ze Bot|zerx|Zeus|ZIPCode|Zixy|zmao|Zyborg [NC]
RewriteRule ^(.*)$ - [F,L]


But this gave me a "INTERNAL SERVER ERROR 500" again. Though the guy that made that line said it worked.

[Biker_GA]

NC is no case. Tells the file to ignore case and apply. You can do it that way as well. :)

If you're getting server errors, rewrite may be an invalid command. What OS and software is your server running?

[tlwwolfseye]

No, you don?t understand. Of course "Rewrite" and that stuff works, i was just saying that for example the one you posted about the MAJESTIC Bot works fine, the other 2 below don?t. So I wonder why that is.

[Biker_GA]

Ahh, my bad.. Use similar syntax to what I provided. It will entail a lot of editing for that long bot list, but you should be able to adapt it. For example:

RewriteCond %{HTTP_USER_AGENT} ^MJ12 [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^ alexibot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^wisenutbot
RewriteRule ^.* - [F,L]

And so on and so forth. :)