The Arcive of Official vBulletin Modifications Site.It is not a VB3 engine, just a parsed copy! |
|
Ban Spiders by User Agent Details »» | |||||||||||||||||||||||||||
What this mod does
With this mod you can enter User Agents to watch or ban, you can also recieve emails or have an Output.txt created and updated with time and date of visits. It doesn't just have to be spiders, you can watch, log or ban any useragent! How to install Simply import the product ban_spider, the mod is active by default but none of the other options are turned on. What is a UserAgent? http://en.wikipedia.org/wiki/User_agent Understanding a UserAgent string http://user-agent-string.info/parse Genuine User Getting Blocked? https://vborg.vbsupport.ru/showpost....&postcount=105 Tools to help http://whatsmyuseragent.com/SwitchingUserAgents.asp http://www.botsvsbrowsers.com/SimulateUserAgent.asp FAQ https://vborg.vbsupport.ru/showpost....&postcount=137 How does it work? https://vborg.vbsupport.ru/showpost....&postcount=381 What's a bot? http://en.wikipedia.org/wiki/Spambot How do i ban a bot? https://vborg.vbsupport.ru/showpost....&postcount=318 https://vborg.vbsupport.ru/showpost....7&postcount=51 Where's output.txt located? https://vborg.vbsupport.ru/showpost....&postcount=216 Bad bot lists https://vborg.vbsupport.ru/showpost....&postcount=259 https://vborg.vbsupport.ru/showpost....&postcount=224 https://vborg.vbsupport.ru/showpost....&postcount=281 Tested on vb3.7.x, vB3.8.x , vB4.x.x but should work on any version. __________________________________________________ __________________ Special thanks to: Lior KH99 BoP5 for helping me sort out a few issues ...and beta testers ForceHSS (Special thanks to Force for latest testing) ozzy47 GreyHost If you use this please mark as INSTALLED History 9th June 2011 Orginal xml added 12th June 2011 Added both email notification and text file logging 22nd June 2011 Version 2.0.0, Added create thread on activity
08th October Beta testing started for thread creation. 20th October Beta testing started for emailing. 21st October Beta testing complete Ver 3.0.0 uploaded 29th October minor fix added to cope with empty userid on thread creation 30th October Beta testing automatic redirection to spiders/bots IP 31st October New xml uploaded with automatic redirect to IP 25th November Minor fix for blank forumid fixed 26th November 2011 Fixed version check & create thread Off by default 17th December 2014 Version 3.1.0 uploaded, Hook changed extra logging and statistics added by Ozzy47 (Chris) 18th December 2014 Version 3.1.1 uploaded, prevented spiders being counted when mod turned off. 17th December 2014 Version 3.1.2 uploaded, due to rogue code from another mod The Bad Bots list is now included in the product Please prune out all those that you wish to be able to see your site (i suggest you definately prune out "DA" and "Custo" : Support will now only be given to those who have this mod marked as INSTALLED Download Now
Supporters / CoAuthors Show Your Support
|
16 благодарности(ей) от: | ||
ASCONAN, BlueCheri, Disco_Dave, ForceHSS, Gn_Snake, GreyGhost, htscpl, Michael.A, midnz, obglobal.net, puertoblack2003, RichieBoy67, rootsxrocks, tanzeelniazi, too_cool_3, victorvu |
Comments |
#642
|
|||
|
|||
I never looked before but it is also getting through on my site.
|
Благодарность от: | ||
Max Taxable |
#643
|
||||
|
||||
Baidu has kissed my "gluteus maximus" for almost 2 years and change if not more ... So has Yandex and a handful of others as well ... I must have a "magic" forum
|
#644
|
||||
|
||||
I am re working a couple of things, and then need to test further, I can then share my findings with Simon.
|
2 благодарности(ей) от: | ||
Black Snow, Max Taxable |
#645
|
|||
|
|||
I would be happy to test on my site if it helps the community.
D. |
#646
|
||||
|
||||
I don't wanna flap it, already flapped too much. But I am pretty sure the problem is solved.
|
#647
|
|||
|
|||
If this helps anyone.... this is a list of what I am seeing in terms of spiders on my site with this installed.
Bing Spiders (6), Google Favicon Spiders (9), Proximic Spiders (135), Baidu Spiders (175), WinHTTP Spiders (12), Facebook Spiders (20), Google AdSense Spiders (7), Magpie Spiders (9), linkdexbot/2.0 Spiders (7), AhrefsBot Spiders (14), Coccoc Spiders (2), Google AppEngine Spiders (6), Google Spiders (40), Sucuri Spiders (3), Twitterbot Spiders (4), Google FeedFetcher Spiders (3), Apple RSS Spiders (1), WordPress.com mShots Spiders (1), Google Web Preview Spiders (3), Grapeshot Spiders (2), James BOT WebCrawler Spiders (5), Netseer crawler/2.0 Spiders (2), Google Images Spiders (3), Galaxy Spiders (2), Feedly Spiders (2), DotBot Spiders (1), Yahoo! Slurp Spiders (1), 360Spider Spiders (4), Netcraft Web Server Survey Spiders (1), NerdyBot Spiders (2), Exabot Spiders (1), Integrity Bot Spiders (1), ContextAd Bot Spiders (2), Twitturls.com (Python-urllib) Spiders (1) I am happy to supply any information that you may find useful to assist in the work you are doing. D. |
#648
|
||||
|
||||
I need a snapshot of your settings for the mod as there is no way all those being entered in the mod would get past the mod!
|
#649
|
||||
|
||||
This is a snapshot of the spiders that are showing up in the whos online:
What exactly do you need a snapshot in the settings Simon? This is my list of spiders I have banned with your mod: almaden Anarchie Artabus ASPSeek attach autoemailspider BackWeb Baidu Bandit BatchFTP BlackWidow Bot\mailto:craftbot@yahoo.com Buddy bumblebee CherryPicker ChinaClaw CICC Collector Copier Copyscape Crescent DIIbot DISCo DISCo\Pump dotbot Download\Demon Download\Wonder Downloader Drip DSurf15a eCatch EasyDL/2.99 EirGrabber EmailCollector EmailSiphon EmailWolf Express\WebPictures ExtractorPro EyeNetIE FileHound FlashGet FrontPage GetRight GetSmart GetWeb! gigabaz GNIP Go\!Zilla Go!Zilla Go-Ahead-Got-It gotit Grabber GrabNet Grafula grub-client HMView HTTrack httpdown .*httrack.* ia_archiver Ichiro Image\Stripper Image\Sucker Indy*Library Indy\Library InterGET InternetLinkagent Internet\Ninja InternetSeer.com Iria JBH*agent JetCar JOC\Web\Spider JustView larbin LeechFTP LexiBot lftp Link*Sleuth likse //Link LinkWalker Mag-Net Magnet Magpie magpie Mass\Downloader Memo Microsoft.URL MIDown\tool Mirror Mister\PiX Mozilla.*Indy Mozilla.*NEWT Mozilla*MSIECrawler MS\FrontPage* MSFrontPage MSIECrawler MSProxy Navroad NearSite NetAnts NetMechanic NetSpider Net\Vampire NetZIP NICErsPRO Ninja Nutch Octopus Offline\Explorer Offline\Navigator omgili Openfind PageGrabber Papa\Foto PaperLiBot pavuk pcBrowser Ping PingALink Pockey psbot Pump QRVA RealDownload Reaper Recorder ReGet Scooter Seeker Siphon sitecheck.internetseer.com SiteSnagger SlySearch SmartDownload Snake sogou Soso SpaceBison speedy Spinn3r sproose Stripper Sucker SuperBot SuperHTTP Surfbot Szukacz tAkeOut Teleport\Pro URLSpiderPro Vacuum VoidEYE Web\Image\Collector Web\Sucker WebAuto [Ww]eb[Bb]andit webcollage WebCopier Web\Downloader WebEMailExtrac.* WebFetch WebGo\IS WebHook WebLeacher WebMiner WebMirror WebReaper WebSauger Website Website\eXtractor Website\Quester Webster WebStripper WebWhacker WebZIP Wget Whacker Widow WWWOFFLE x-Tractor Xaldon\WebSpider Xenu Yandex Yeti YOUDAOBOT Zeus.*Webster Zeus baiduspider beta.statsit.com statsit SiteIntel Yandex GomezAgent FunWebProducts Nesotebot DCPbot AOL Advertising R&D DataCha0s aiHitBot Apache-HttpClient Zend_Http_Client ReverseGet XXX bot Content vBSEO spbot OffByOne thyroidbuzz AcoonBot coccoc xpymep proxyproxy2884 AppEngine start.exe Semiocast HTTP client Firefox/3.6.23 TurnitinBot curl SwpLc/1.6 GrepNetstat.com news bot AskTbPTV checks panopta App3le PhantomJS AlwaysOnline SISTRIX proximic CRAWL-E/0.6.4 WebMoney Maxthon HTMLParser oBot UnisterBot ERACrawler Butterfly Topsy Butterfly Topsy Crawler Ezooms Deepnet Alexa Bitlybot Seznam Fulltext Sunrise Communications AG crawl Crawl MJ12bot Bimbot Snapbot thunderstone Thunderstone grub-client Bing MSN OOZBOT Wayback Machine Crowsnest Spider FlipboardProxy Feedly |
#650
|
|||
|
|||
Here is my stuff:
|
#651
|
||||
|
||||
Hi Gadget Guy, remove the second picture as it has your email address in it. I see the settings are ok, now can you just copy the list as you have it (copy straight out of the textbox in the mod) sitck it in a wordpad document, zip it and attach it here so i can check that please.
|
|
|
X vBulletin 3.8.12 by vBS Debug Information | |
---|---|
|
|
More Information | |
Template Usage:
Phrase Groups Available:
|
Included Files:
Hooks Called:
|