vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vB4 General Discussions (https://vborg.vbsupport.ru/forumdisplay.php?f=251)
-   -   Can Private forum Locations Be crawled? (https://vborg.vbsupport.ru/showthread.php?t=287501)

invitezone 09-05-2012 05:20 PM

Can Private forum Locations Be crawled?
 
I know that when google spider and baidu etc, try to crawl a forum they dont have permission to enter they only see the permission denied page as seen on who's online.
But the fact that they are seeing the permission denied thing does that mean they will cache the location.

For eg.
I make a section called www.myforum.com/private-section
I make the section hidden from view,
I see google bot trying to crawl it, but seeing the no permission thing.

Now google bot has been denied seeing the contents of that forum, but does google bot now know that the location exists?
I want to make it so that the no-one even bots ever knows a private forum exists within my public forum.

I know I can deny via the robots.txt but some bots choose to ignore that.
How can I make sure of it? is permissions enough?

Thanks for your help.

--------------- Added [DATE]1346944839[/DATE] at [TIME]1346944839[/TIME] ---------------

anyone know this?
thanks


All times are GMT. The time now is 05:00 AM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01118 seconds
  • Memory Usage 1,706KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (1)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete