vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 3.8 Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=235)
-   -   Miscellaneous Hacks - Ban Spiders by User Agent (https://vborg.vbsupport.ru/showthread.php?t=264932)

Midohash 01-18-2012 10:50 AM

Thanks a lot Simon :up: ... I still have same problem with Baidu :( ... I have checked my raw access log today and still having loads of Baidu spiders in my site 48 hours after installing the mod :eek: ... this is how they appear in my log file:

Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)

Any recommendations?

Regarding Alexa, I have certified my site and have pasted their javascript code in the header of my forum as they recommend so they can analyse my website traffic directly regardless of visitors with/without alexa toolbar ... I don't know if my alexa rank will drop due to drop in the overall visitor numbers after banning spiders?

Regards

Simon Lloyd 01-19-2012 10:47 AM

of course your raw logs from cpanel, plesk or whathaveyou will show them visiting your site, remember this mod redirects them when trying to load your site, it does that regardless of whether they are trying to load the homepage or a thread, your Alexa shouldn't drop on account of redirecting spiders, in fact i've been redirecting these spiders for ages on one of my sites that was Page Rank 2 and it's now PR3, so if it's not hurting Google rank it has to be good :)

Can you give me a screenshot of your settings for ths mod?

Midohash 01-19-2012 10:58 PM

Thanks Simon, enclosed is a screenshot of my settings for the mod. My Alexa rank dropped from 83000 to 84000 but not sure if that related to the drop in the visitors count after applying the mod or just an incidental finding :confused:

http://img29.imageshack.us/img29/8616/spidersq.jpg

Simon Lloyd 01-20-2012 02:28 PM

Midohash, if you want to pm me admin access i'll take a look.

Doug Nelson 01-27-2012 08:29 AM

Excellent mod, thank you.

What is the advantage/disadvantage to redirecting spiders back to their own IP?

Simon Lloyd 01-27-2012 02:18 PM

Quote:

Originally Posted by Doug Nelson (Post 2292982)
Excellent mod, thank you.

What is the advantage/disadvantage to redirecting spiders back to their own IP?

You have to send them somewhere and whilst the example i gave you is a fun place to send them and they expect a certain amount of bots there's no guarantees that it will always be there, however, if you redirect to their own IP they will always have somewhere to be redirected to :)

gsmlover4u 01-30-2012 07:56 AM

to block spiders are benificial or not ?

Simon Lloyd 01-30-2012 12:40 PM

Blocking spiders will save you tons on your bandwidth and server load, take the Baiduspider they send hundreds upon hindreds of bots to index your site and they don't obey robots.txt

gsmlover4u 01-31-2012 05:14 AM

working on vbulletin 4.1.10

steviewonder44 02-07-2012 06:12 PM

The ips for baidu I also ban: 123, 124 125 and 58


All times are GMT. The time now is 02:16 AM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01149 seconds
  • Memory Usage 1,735KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (1)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (10)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete