Thread: Miscellaneous Hacks - Ban Spiders by User Agent
View Single Post
  #42  
Old 09-12-2011, 05:52 AM
TheWhite TheWhite is offline
 
Join Date: Nov 2006
Posts: 147
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Well guys, I have a decent forum with 185k members with a decent dedicated server and an acceptable Google adsense income.

In the last few years I've tried to "ban" the vicious Baidu Spider by using the robot text (useless because it don't obey it), then by using the .htacess (was great) which worked fine for a couple of years but since a month ago it has somehow found a way in not obeying that either.

In the last week or so I started banning the Ips as a last resort but these guys come out with a new one in a couple of days so this morning I did a Google search and came to this WONDERFUL VB MOD, this coder deserves a medal and not only the MOD OF THE MONTH (so vote for him!!). After an hour, Baidu is NO MORE!! I hit 253GB of bandwidth this morning which is over my average monthly (time period) rate by at least 75GB not mentioning the server slowdowns.

Baidu doesn't bring any good traffic (adsense wise), it only does harm by eating up your resources and slowing down your forum thus causing the Google Crawlers (GOOD BOTS) to take more time on indexing your forum which is bad.

I haven't tested the logging and emailing reportings yet but I will in the next few days.

Cheers!!
Reply With Quote
 
X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01188 seconds
  • Memory Usage 1,762KB
  • Queries Executed 11 (?)
More Information
Template Usage:
  • (1)SHOWTHREAD_SHOWPOST
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)post_thanks_box
  • (1)post_thanks_button
  • (1)post_thanks_javascript
  • (1)post_thanks_navbar_search
  • (1)post_thanks_postbit_info
  • (1)postbit
  • (1)postbit_onlinestatus
  • (1)postbit_wrapper
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • reputationlevel
  • showthread
Included Files:
  • ./showpost.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_post_thanks.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_postinfo_query
  • fetch_postinfo
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • showpost_start
  • bbcode_fetch_tags
  • bbcode_create
  • postbit_factory
  • showpost_post
  • postbit_display_start
  • post_thanks_function_post_thanks_off_start
  • post_thanks_function_post_thanks_off_end
  • post_thanks_function_fetch_thanks_start
  • post_thanks_function_fetch_thanks_end
  • post_thanks_function_thanked_already_start
  • post_thanks_function_thanked_already_end
  • fetch_musername
  • postbit_imicons
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • post_thanks_function_can_thank_this_post_start
  • showpost_complete