Thread: Miscellaneous Hacks - Ban Spiders by User Agent
View Single Post
  #525  
Old 11-09-2013, 10:09 PM
bzcomputers's Avatar
bzcomputers bzcomputers is offline
 
Join Date: Apr 2012
Location: TX
Posts: 503
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally Posted by Toxic2 View Post
Simon can you come up with a Viable List of Bad UA's more appropriate for 2013? MSIE1 blocked one of my members, so i deleted MSIE1 from the List.
Removing MSIE1 is about the only thing that would definitely be recommended to remove from the old list for this mod. MSIE1 will block users on IE10 and IE11 (which means A LOT of users and growing daily). I haven't seen anything prior to IE5 try to access my sites in a long time, so removing MSIE1 should not negatively impact your site in any way either (but blocking IE10 & 11 will!).

As was mentioned above it is a personal choice on what you block. There are thousands of sites out there that don't block any and are doing just fine. You need to figure out what's important for your site. Being hit hundreds of times a day from bots from regions of the world you probably don't even cater too like China or countries of the former Soviet Union? or Blocking those bots and in turn reducing server load and load times for those users you do want to cater too?

For instance if you block IE6 (MSIE6) you will currently block over 20% of Chinese internet users (average everywhere else in the World is less than 1% using IE6). When you consider China has 1.3 billion people that is a lot of potential website traffic. At the same time China also accounted for nearly 40% of the world's computer hacking attack traffic during 2012. If you have content that is viable for Chinese users which can possibly turn into potential profit (sales / ad clicks) for you then blocking MSIE6 and chinese specific bots like baiduspider, iaskspider, sogou spider, Sosospider, YoudaoBot and others is probably not recommended.

There are plenty of good resources of information in this thread and other places on the net on useragents. Just remember copying someones elses useragent blocking list directly and not specifically taking a little time and tweaking it for your own needs is almost guaranteed to negatively impact your site traffic.

Here are a couple of additional resources:
http://www.useragentstring.com/pages/Crawlerlist/
http://www.user-agents.org/
Reply With Quote
3 благодарности(ей) от:
Max Taxable, Simon Lloyd, Toxic2
 
X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01203 seconds
  • Memory Usage 1,771KB
  • Queries Executed 11 (?)
More Information
Template Usage:
  • (1)SHOWTHREAD_SHOWPOST
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)bbcode_quote
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)post_thanks_box
  • (3)post_thanks_box_bit
  • (1)post_thanks_button
  • (1)post_thanks_javascript
  • (1)post_thanks_navbar_search
  • (1)post_thanks_postbit
  • (1)post_thanks_postbit_info
  • (1)postbit
  • (1)postbit_onlinestatus
  • (1)postbit_wrapper
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • reputationlevel
  • showthread
Included Files:
  • ./showpost.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_post_thanks.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_postinfo_query
  • fetch_postinfo
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • showpost_start
  • bbcode_fetch_tags
  • bbcode_create
  • postbit_factory
  • showpost_post
  • postbit_display_start
  • post_thanks_function_post_thanks_off_start
  • post_thanks_function_post_thanks_off_end
  • post_thanks_function_fetch_thanks_start
  • fetch_musername
  • post_thanks_function_fetch_thanks_end
  • post_thanks_function_thanked_already_start
  • post_thanks_function_thanked_already_end
  • post_thanks_function_fetch_thanks_bit_start
  • post_thanks_function_show_thanks_date_start
  • post_thanks_function_show_thanks_date_end
  • post_thanks_function_fetch_thanks_bit_end
  • post_thanks_function_fetch_post_thanks_template_start
  • post_thanks_function_fetch_post_thanks_template_end
  • postbit_imicons
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • post_thanks_function_can_thank_this_post_start
  • showpost_complete