vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   Modification Requests/Questions (Unpaid) (https://vborg.vbsupport.ru/forumdisplay.php?f=112)
-   -   Search Engine Trap (https://vborg.vbsupport.ru/showthread.php?t=64184)

smestas 04-21-2004 07:19 AM

Search Engine Trap
 
I noticed on my site (currently active users) that Google Bot and others always tries to follow the reply and search links. It would be great if someone could write a HACK that would catch listed search engines by user agent to redirect to the archive pages when they are doing stuff like this.

vbmechanic 04-21-2004 11:24 AM

Easier than that... create a file called "robots.txt" and place it in the root directory of your site. The file should contain the following:

User-agent: *
Disallow: /forum/newreply.php
Disallow: /forum/newthread.php
Disallow: /forum/search.php

etc etc.. just list out any files that you do not want search engines to read. Most good engines will obey this file.


All times are GMT. The time now is 07:32 AM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.00927 seconds
  • Memory Usage 1,699KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (2)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete