![]() |
Quote:
|
Baidu has kissed my "gluteus maximus" for almost 2 years and change if not more ... So has Yandex and a handful of others as well ... I must have a "magic" forum :)
|
I am re working a couple of things, and then need to test further, I can then share my findings with Simon. :)
|
I would be happy to test on my site if it helps the community.
D. |
Quote:
|
If this helps anyone.... this is a list of what I am seeing in terms of spiders on my site with this installed.
Bing Spiders (6), Google Favicon Spiders (9), Proximic Spiders (135), Baidu Spiders (175), WinHTTP Spiders (12), Facebook Spiders (20), Google AdSense Spiders (7), Magpie Spiders (9), linkdexbot/2.0 Spiders (7), AhrefsBot Spiders (14), Coccoc Spiders (2), Google AppEngine Spiders (6), Google Spiders (40), Sucuri Spiders (3), Twitterbot Spiders (4), Google FeedFetcher Spiders (3), Apple RSS Spiders (1), WordPress.com mShots Spiders (1), Google Web Preview Spiders (3), Grapeshot Spiders (2), James BOT WebCrawler Spiders (5), Netseer crawler/2.0 Spiders (2), Google Images Spiders (3), Galaxy Spiders (2), Feedly Spiders (2), DotBot Spiders (1), Yahoo! Slurp Spiders (1), 360Spider Spiders (4), Netcraft Web Server Survey Spiders (1), NerdyBot Spiders (2), Exabot Spiders (1), Integrity Bot Spiders (1), ContextAd Bot Spiders (2), Twitturls.com (Python-urllib) Spiders (1) I am happy to supply any information that you may find useful to assist in the work you are doing. D. |
I need a snapshot of your settings for the mod as there is no way all those being entered in the mod would get past the mod!
|
1 Attachment(s)
This is a snapshot of the spiders that are showing up in the whos online:
https://vborg.vbsupport.ru/external/2014/12/30.jpg What exactly do you need a snapshot in the settings Simon? This is my list of spiders I have banned with your mod: almaden Anarchie Artabus ASPSeek attach autoemailspider BackWeb Baidu Bandit BatchFTP BlackWidow Bot\mailto:craftbot@yahoo.com Buddy bumblebee CherryPicker ChinaClaw CICC Collector Copier Copyscape Crescent DIIbot DISCo DISCo\Pump dotbot Download\Demon Download\Wonder Downloader Drip DSurf15a eCatch EasyDL/2.99 EirGrabber EmailCollector EmailSiphon EmailWolf Express\WebPictures ExtractorPro EyeNetIE FileHound FlashGet FrontPage GetRight GetSmart GetWeb! gigabaz GNIP Go\!Zilla Go!Zilla Go-Ahead-Got-It gotit Grabber GrabNet Grafula grub-client HMView HTTrack httpdown .*httrack.* ia_archiver Ichiro Image\Stripper Image\Sucker Indy*Library Indy\Library InterGET InternetLinkagent Internet\Ninja InternetSeer.com Iria JBH*agent JetCar JOC\Web\Spider JustView larbin LeechFTP LexiBot lftp Link*Sleuth likse //Link LinkWalker Mag-Net Magnet Magpie magpie Mass\Downloader Memo Microsoft.URL MIDown\tool Mirror Mister\PiX Mozilla.*Indy Mozilla.*NEWT Mozilla*MSIECrawler MS\FrontPage* MSFrontPage MSIECrawler MSProxy Navroad NearSite NetAnts NetMechanic NetSpider Net\Vampire NetZIP NICErsPRO Ninja Nutch Octopus Offline\Explorer Offline\Navigator omgili Openfind PageGrabber Papa\Foto PaperLiBot pavuk pcBrowser Ping PingALink Pockey psbot Pump QRVA RealDownload Reaper Recorder ReGet Scooter Seeker Siphon sitecheck.internetseer.com SiteSnagger SlySearch SmartDownload Snake sogou Soso SpaceBison speedy Spinn3r sproose Stripper Sucker SuperBot SuperHTTP Surfbot Szukacz tAkeOut Teleport\Pro URLSpiderPro Vacuum VoidEYE Web\Image\Collector Web\Sucker WebAuto [Ww]eb[Bb]andit webcollage WebCopier Web\Downloader WebEMailExtrac.* WebFetch WebGo\IS WebHook WebLeacher WebMiner WebMirror WebReaper WebSauger Website Website\eXtractor Website\Quester Webster WebStripper WebWhacker WebZIP Wget Whacker Widow WWWOFFLE x-Tractor Xaldon\WebSpider Xenu Yandex Yeti YOUDAOBOT Zeus.*Webster Zeus baiduspider beta.statsit.com statsit SiteIntel Yandex GomezAgent FunWebProducts Nesotebot DCPbot AOL Advertising R&D DataCha0s aiHitBot Apache-HttpClient Zend_Http_Client ReverseGet XXX bot Content vBSEO spbot OffByOne thyroidbuzz AcoonBot coccoc xpymep proxyproxy2884 AppEngine start.exe Semiocast HTTP client Firefox/3.6.23 TurnitinBot curl SwpLc/1.6 GrepNetstat.com news bot AskTbPTV checks panopta App3le PhantomJS AlwaysOnline SISTRIX proximic CRAWL-E/0.6.4 WebMoney Maxthon HTMLParser oBot UnisterBot ERACrawler Butterfly Topsy Butterfly Topsy Crawler Ezooms Deepnet Alexa Bitlybot Seznam Fulltext Sunrise Communications AG crawl Crawl MJ12bot Bimbot Snapbot thunderstone Thunderstone grub-client Bing MSN OOZBOT Wayback Machine Crowsnest Spider FlipboardProxy Feedly |
1 Attachment(s)
Here is my stuff:
|
Hi Gadget Guy, remove the second picture as it has your email address in it. I see the settings are ok, now can you just copy the list as you have it (copy straight out of the textbox in the mod) sitck it in a wordpad document, zip it and attach it here so i can check that please.
|
All times are GMT. The time now is 05:08 PM. |
Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information | |
---|---|
|
|
![]() |
|
Template Usage:
Phrase Groups Available:
|
Included Files:
Hooks Called:
|