Go Back   vb.org Archive > vBulletin 3 Discussion > vB3 General Discussions
FAQ Community Calendar Today's Posts Search

Reply
 
Thread Tools Display Modes
  #1  
Old 07-08-2005, 07:07 AM
yinyang's Avatar
yinyang yinyang is offline
 
Join Date: May 2004
Location: Los Angeles
Posts: 580
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default Closing Board keeps out Spiders

I've always wondered about the following:

If you close a board, i.e. make it so posts can only be read if you register, does it keep out spiders/search engines?

If so, is there a way to close a board and allow spiders/search engines?
Reply With Quote
  #2  
Old 07-08-2005, 07:16 AM
Colin F's Avatar
Colin F Colin F is offline
 
Join Date: Jul 2004
Location: Switzerland
Posts: 1,551
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally Posted by yinyang
I've always wondered about the following:

If you close a board, i.e. make it so posts can only be read if you register, does it keep out spiders/search engines?

If so, is there a way to close a board and allow spiders/search engines?
Yes, it does.
Spiders generally see the same content as guests.

Be aware that displaying different content to search engine spiders and regular users is against a lot of the search engines rules...
Reply With Quote
  #3  
Old 07-08-2005, 07:17 AM
Andreas's Avatar
Andreas Andreas is offline
 
Join Date: Jan 2004
Location: Germany
Posts: 6,863
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
If you close a board, i.e. make it so posts can only be read if you register, does it keep out spiders/search engines?
Yes

Quote:
If so, is there a way to close a board and allow spiders/search engines?
Not really ... you could check UAs and give access if you find a match for a Spider, but UAs can be set by the User => everybody has access.
Or you can check IPs. This is more difficult to be faked, but still possible - and it would IMHO be hard to get a list of Spider IPs.

Quote:
Originally Posted by ColinF
Be aware that displaying different content to search engine spiders and regular users is against a lot of the search engines rules...
Right. I always wonderes if assigning an ultra-light style (like the Archive) to Spiders would also be considered Cloaking?
Reply With Quote
  #4  
Old 07-08-2005, 07:24 AM
Marco van Herwaarden Marco van Herwaarden is offline
 
Join Date: Jul 2004
Posts: 25,415
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally Posted by yinyang
If you close a board, i.e. make it so posts can only be read if you register, does it keep out spiders/search engines?
Yes this will keep out spiders. Spiders follow the permissions that are set for the 'Unregistered / Not logged in' Usergroup. They can only index pages they can see.

Quote:
Originally Posted by yinyang
If so, is there a way to close a board and allow spiders/search engines?
Yes it could be done with some modifications. It would however be against the TOS of all search engines and if you are caught, you would be totally removed and blocked by them.
Reply With Quote
  #5  
Old 07-08-2005, 07:47 AM
yinyang's Avatar
yinyang yinyang is offline
 
Join Date: May 2004
Location: Los Angeles
Posts: 580
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

ok. thanks for the info guys.

i just wanted the spiders to see what the regular members saw, and not what the unregistered members saw. but you've answered my questions. thanks!
Reply With Quote
Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT. The time now is 02:14 PM.


Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2024, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.06183 seconds
  • Memory Usage 2,200KB
  • Queries Executed 13 (?)
More Information
Template Usage:
  • (1)SHOWTHREAD
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)ad_showthread_beforeqr
  • (1)ad_showthread_firstpost
  • (1)ad_showthread_firstpost_sig
  • (1)ad_showthread_firstpost_start
  • (6)bbcode_quote
  • (1)footer
  • (1)forumjump
  • (1)forumrules
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (1)navbar
  • (3)navbar_link
  • (120)option
  • (5)post_thanks_box
  • (5)post_thanks_button
  • (1)post_thanks_javascript
  • (1)post_thanks_navbar_search
  • (5)post_thanks_postbit_info
  • (5)postbit
  • (5)postbit_onlinestatus
  • (5)postbit_wrapper
  • (1)spacer_close
  • (1)spacer_open
  • (1)tagbit_wrapper 

Phrase Groups Available:
  • global
  • inlinemod
  • postbit
  • posting
  • reputationlevel
  • showthread
Included Files:
  • ./showthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_post_thanks.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_postinfo_query
  • fetch_postinfo
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • showthread_start
  • showthread_getinfo
  • forumjump
  • showthread_post_start
  • showthread_query_postids
  • showthread_query
  • bbcode_fetch_tags
  • bbcode_create
  • showthread_postbit_create
  • postbit_factory
  • postbit_display_start
  • post_thanks_function_post_thanks_off_start
  • post_thanks_function_post_thanks_off_end
  • post_thanks_function_fetch_thanks_start
  • post_thanks_function_fetch_thanks_end
  • post_thanks_function_thanked_already_start
  • post_thanks_function_thanked_already_end
  • fetch_musername
  • postbit_imicons
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • post_thanks_function_can_thank_this_post_start
  • tag_fetchbit_complete
  • forumrules
  • navbits
  • navbits_complete
  • showthread_complete