Thread: Miscellaneous Hacks - Ban Spiders by User Agent
View Single Post
  #647  
Old 12-11-2014, 02:27 AM
Gadget_Guy Gadget_Guy is offline
 
Join Date: Jun 2010
Posts: 271
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

If this helps anyone.... this is a list of what I am seeing in terms of spiders on my site with this installed.

Bing Spiders (6),
Google Favicon Spiders (9),
Proximic Spiders (135),
Baidu Spiders (175),
WinHTTP Spiders (12),
Facebook Spiders (20),
Google AdSense Spiders (7),
Magpie Spiders (9),
linkdexbot/2.0 Spiders (7),
AhrefsBot Spiders (14),
Coccoc Spiders (2),
Google AppEngine Spiders (6),
Google Spiders (40),
Sucuri Spiders (3),
Twitterbot Spiders (4),
Google FeedFetcher Spiders (3),
Apple RSS Spiders (1),
WordPress.com mShots Spiders (1),
Google Web Preview Spiders (3),
Grapeshot Spiders (2),
James BOT WebCrawler Spiders (5),
Netseer crawler/2.0 Spiders (2),
Google Images Spiders (3),
Galaxy Spiders (2),
Feedly Spiders (2),
DotBot Spiders (1),
Yahoo! Slurp Spiders (1),
360Spider Spiders (4),
Netcraft Web Server Survey Spiders (1),
NerdyBot Spiders (2),
Exabot Spiders (1),
Integrity Bot Spiders (1),
ContextAd Bot Spiders (2),
Twitturls.com (Python-urllib) Spiders (1)

I am happy to supply any information that you may find useful to assist in the work you are doing.

D.
Reply With Quote
 
X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.02994 seconds
  • Memory Usage 1,761KB
  • Queries Executed 11 (?)
More Information
Template Usage:
  • (1)SHOWTHREAD_SHOWPOST
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)post_thanks_box
  • (1)post_thanks_button
  • (1)post_thanks_javascript
  • (1)post_thanks_navbar_search
  • (1)post_thanks_postbit_info
  • (1)postbit
  • (1)postbit_onlinestatus
  • (1)postbit_wrapper
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • reputationlevel
  • showthread
Included Files:
  • ./showpost.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_post_thanks.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_postinfo_query
  • fetch_postinfo
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • showpost_start
  • bbcode_fetch_tags
  • bbcode_create
  • postbit_factory
  • showpost_post
  • postbit_display_start
  • post_thanks_function_post_thanks_off_start
  • post_thanks_function_post_thanks_off_end
  • post_thanks_function_fetch_thanks_start
  • post_thanks_function_fetch_thanks_end
  • post_thanks_function_thanked_already_start
  • post_thanks_function_thanked_already_end
  • fetch_musername
  • postbit_imicons
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • post_thanks_function_can_thank_this_post_start
  • showpost_complete