Go Back   vb.org Archive > Community Discussions > Forum and Server Management
FAQ Community Calendar Today's Posts Search

 
 
Thread Tools Display Modes
Prev Previous Post   Next Post Next
  #9  
Old 10-12-2008, 11:34 PM
SEOvB's Avatar
SEOvB SEOvB is offline
 
Join Date: May 2007
Location: Indianapolis
Posts: 2,451
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally Posted by ATVTorture View Post
Wowser, no simple copy and paste from yo0ur robots.txt?
Blocking robots or even delaying them is a really piss poor work around for what shouldn't be a problem. You're only going to hurt your results in the search engines by doing this but if you aren't concerned with such trickery then its a non-issue at that point.

Code:
# Allow Archiver
User-agent: ia_archiver
Allow: /


User-agent: Slurp
Crawl-delay: 60

User-agent: GoogleBot
Crawl-delay: 60


User-agent: *

Disallow: *.php
Disallow: *.js
Disallow: *.jsp
Disallow: *.cfm
Disallow: *.asp
Disallow: *.html
Disallow: *.htm
Disallow: *.aspx
Disallow: *.cgi

Disallow: /forum/includes/
Disallow: /forum/install/
Disallow: /forum/customavatars/
Disallow: /forum/archive/
Disallow: /forum/sitemap/
Disallow: /forum/members/
Disallow: /wp-includes/
Disallow: /wp-content/
Disallow: /wp-admin/
Disallow: /forum/images
And just out of curiosity what hosting company are you using
Reply With Quote
 


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT. The time now is 02:18 PM.


Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.04687 seconds
  • Memory Usage 2,719KB
  • Queries Executed 12 (?)
More Information
Template Usage:
  • (1)SHOWTHREAD
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)ad_showthread_beforeqr
  • (1)bbcode_code
  • (4)bbcode_quote
  • (1)footer
  • (1)forumjump
  • (1)forumrules
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (1)navbar
  • (3)navbar_link
  • (120)option
  • (17)post_thanks_box
  • (17)post_thanks_button
  • (1)post_thanks_javascript
  • (1)post_thanks_navbar_search
  • (17)post_thanks_postbit_info
  • (17)postbit
  • (17)postbit_onlinestatus
  • (17)postbit_wrapper
  • (1)showthread_list
  • (1)spacer_close
  • (1)spacer_open
  • (1)tagbit_wrapper 

Phrase Groups Available:
  • global
  • inlinemod
  • postbit
  • posting
  • reputationlevel
  • showthread
Included Files:
  • ./showthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_threadedmode.php
  • ./includes/functions_post_thanks.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_postinfo_query
  • fetch_postinfo
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • showthread_start
  • showthread_getinfo
  • forumjump
  • showthread_post_start
  • showthread_query_postids_threaded
  • showthread_threaded_construct_link
  • showthread_query
  • bbcode_fetch_tags
  • bbcode_create
  • showthread_postbit_create
  • postbit_factory
  • postbit_display_start
  • post_thanks_function_post_thanks_off_start
  • post_thanks_function_post_thanks_off_end
  • post_thanks_function_fetch_thanks_start
  • post_thanks_function_fetch_thanks_end
  • post_thanks_function_thanked_already_start
  • post_thanks_function_thanked_already_end
  • fetch_musername
  • postbit_imicons
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • post_thanks_function_can_thank_this_post_start
  • tag_fetchbit_complete
  • forumrules
  • navbits
  • navbits_complete
  • showthread_complete