Go Back   vb.org Archive > vBulletin 4 Discussion > vB4 General Discussions
FAQ Community Calendar Today's Posts Search

Reply
 
Thread Tools Display Modes
  #1  
Old 10-08-2010, 11:51 AM
tattooz tattooz is offline
 
Join Date: Aug 2010
Posts: 76
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default Block bots

Is there anyway to block bots like Baidu from accessing my site?
Reply With Quote
  #2  
Old 10-08-2010, 08:31 PM
your24hourstore your24hourstore is offline
 
Join Date: Feb 2010
Posts: 1,226
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally Posted by tattooz View Post
Is there anyway to block bots like Baidu from accessing my site?
sure just use htaccess file or block them in robots.txt
Reply With Quote
  #3  
Old 12-23-2010, 09:34 PM
GONUMBER6's Avatar
GONUMBER6 GONUMBER6 is offline
 
Join Date: Jan 2010
Location: Sunny Arizona
Posts: 352
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Great question!! But how do you do this exactly? I usually have 100 of these spiders on my site at all times of the day!! I just want to make sure I don't block the other bots as well.

Here is my current robots.txt I have just made this recently so let me know if there is somethings you would change in it....

PHP Code:
User-agent: *
Allow: /

User-agent: *
Disallow: /forums/clientscript/
Disallow: /forums/cpstyles/
Disallow: /forums/customavatars/
Disallow: /forums/customprofilepics/
Disallow: /forums/images/
Disallow: /forums/ajax.php
Disallow
: /forums/arcade.php
Disallow
: /forums/attachment.php
Disallow
: /forums/awards.php
Disallow
: /forums/calendar.php
Disallow
: /forums/cron.php
Disallow
: /forums/editpost.php
Disallow
: /forums/faq.php
Disallow
: /forums/global.php
Disallow
: /forums/image.php
Disallow
: /forums/inlinemod.php
Disallow
: /forums/joinrequests.php
Disallow
: /forums/login.php
Disallow
: /forums/market.php
Disallow
: /forums/market_bank.php
Disallow
: /forums/market_convert.php
Disallow
: /forums/member.php
Disallow
: /forums/memberlist.php
Disallow
: /forums/misc.php
Disallow
: /forums/moderator.php
Disallow
: /forums/newattachment.php
Disallow
: /forums/newreply.php
Disallow
: /forums/newthread.php
Disallow
: /forums/online.php
Disallow
: /forums/poll.php
Disallow
: /forums/postings.php
Disallow
: /forums/printthread.php
Disallow
: /forums/private.php
Disallow
: /forums/profile.php
Disallow
: /forums/register.php
Disallow
: /forums/report.php
Disallow
: /forums/reputation.php
Disallow
: /forums/search.php
Disallow
: /forums/sendmessage.php
Disallow
: /forums/showgroups.php
Disallow
: /forums/subscription.php
Disallow
: /forums/threadrate.php
Disallow
: /forums/usercp.php
Disallow
: /forums/usernote.php
Disallow
: /forums/vbactivity.php
Disallow
: *.js
Disallow
: /forums/includes/
Disallow: /forums/install/
Disallow: /forums/customavatars/
Disallow: /css
Reply With Quote
Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT. The time now is 11:10 PM.


Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.03606 seconds
  • Memory Usage 2,211KB
  • Queries Executed 13 (?)
More Information
Template Usage:
  • (1)SHOWTHREAD
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)ad_showthread_beforeqr
  • (1)ad_showthread_firstpost
  • (1)ad_showthread_firstpost_sig
  • (1)ad_showthread_firstpost_start
  • (1)bbcode_php
  • (1)bbcode_quote
  • (1)footer
  • (1)forumjump
  • (1)forumrules
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (1)navbar
  • (3)navbar_link
  • (120)option
  • (3)post_thanks_box
  • (3)post_thanks_button
  • (1)post_thanks_javascript
  • (1)post_thanks_navbar_search
  • (3)post_thanks_postbit_info
  • (3)postbit
  • (3)postbit_onlinestatus
  • (3)postbit_wrapper
  • (1)spacer_close
  • (1)spacer_open
  • (1)tagbit_wrapper 

Phrase Groups Available:
  • global
  • inlinemod
  • postbit
  • posting
  • reputationlevel
  • showthread
Included Files:
  • ./showthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_post_thanks.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_postinfo_query
  • fetch_postinfo
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • showthread_start
  • showthread_getinfo
  • forumjump
  • showthread_post_start
  • showthread_query_postids
  • showthread_query
  • bbcode_fetch_tags
  • bbcode_create
  • showthread_postbit_create
  • postbit_factory
  • postbit_display_start
  • post_thanks_function_post_thanks_off_start
  • post_thanks_function_post_thanks_off_end
  • post_thanks_function_fetch_thanks_start
  • post_thanks_function_fetch_thanks_end
  • post_thanks_function_thanked_already_start
  • post_thanks_function_thanked_already_end
  • fetch_musername
  • postbit_imicons
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • post_thanks_function_can_thank_this_post_start
  • tag_fetchbit_complete
  • forumrules
  • navbits
  • navbits_complete
  • showthread_complete