vb.org Archive - robots.txt for keeping certain forums from being indexed/spidered?

vb.org Archive (https://vborg.vbsupport.ru/index.php)

- vB3 General Discussions (https://vborg.vbsupport.ru/forumdisplay.php?f=111)

- - robots.txt for keeping certain forums from being indexed/spidered? (https://vborg.vbsupport.ru/showthread.php?t=164591)

DFC99

12-07-2007 12:43 PM

robots.txt for keeping certain forums from being indexed/spidered?

Hi, I've got a few forums (e.g. f=46) on my forum (such as the sandbox and off-topic) which are open to everyone to see, but I don't want them spidered by google and other search engines. Is there a way of doing this using robots.txt? Thanks :)!

ssslippy

12-07-2007 05:54 PM

Not all search engines follow the robots.txt. So this wouldnt quite work.

All times are GMT. The time now is 09:33 PM.

X vBulletin 3.8.12 by vBS Debug Information
Page Generation 0.00976 seconds Memory Usage 1,699KB Queries Executed 10 (?)
More Information
Template Usage: (1)ad_footer_end (1)ad_footer_start (1)ad_header_end (1)ad_header_logo (1)ad_navbar_below (1)footer (1)gobutton (1)header (1)headinclude (6)option (1)post_thanks_navbar_search (1)printthread (2)printthreadbit (1)spacer_close (1)spacer_open Phrase Groups Available: global postbit showthread	Included Files: ./printthread.php ./global.php ./includes/init.php ./includes/class_core.php ./includes/config.php ./includes/functions.php ./includes/class_hook.php ./includes/modsystem_functions.php ./includes/class_bbcode_alt.php ./includes/class_bbcode.php ./includes/functions_bigthree.php Hooks Called: init_startup init_startup_session_setup_start init_startup_session_setup_complete cache_permissions fetch_threadinfo_query fetch_threadinfo fetch_foruminfo style_fetch cache_templates global_start parse_templates global_setup_complete printthread_start bbcode_fetch_tags bbcode_create bbcode_parse_start bbcode_parse_complete_precache bbcode_parse_complete printthread_post printthread_complete
Messages: