vb.org Archive - Bots following links

vb.org Archive (https://vborg.vbsupport.ru/index.php)

- vB3 General Discussions (https://vborg.vbsupport.ru/forumdisplay.php?f=111)

- - Bots following links (https://vborg.vbsupport.ru/showthread.php?t=73377)

MeNeedHelp

12-27-2004 10:00 PM

Bots following links

Hi,

How do I prevent Bots from following links on my site.
I currently count it when someone follows a link on my site but Bots seem to be able to follow all links and thus my count sky-rockets.

Thanks for any input.
MeNeedHelp (I really do :squareeyed: ).

manguish

12-29-2004 08:46 PM

robots.txt

Try something like this :

Code:

User-agent: *

Disallow: /vb/modcp/

Disallow: /vb/admincp/

Disallow: /cgi-bin

Disallow: /scgi-bin

Add in the links you want them to not follow.

I'm guessing this will work, but will stand to be corrected ;)

MeNeedHelp

12-29-2004 09:09 PM

Thanks for the reply, however, I have almost 300 links and will not be able to maunally disallow each link (well, do not want to, is a better word :rolleyes: ).

All times are GMT. The time now is 12:36 AM.

X vBulletin 3.8.12 by vBS Debug Information
Page Generation 0.00889 seconds Memory Usage 1,707KB Queries Executed 10 (?)
More Information
Template Usage: (1)ad_footer_end (1)ad_footer_start (1)ad_header_end (1)ad_header_logo (1)ad_navbar_below (1)bbcode_code_printable (1)footer (1)gobutton (1)header (1)headinclude (6)option (1)post_thanks_navbar_search (1)printthread (3)printthreadbit (1)spacer_close (1)spacer_open Phrase Groups Available: global postbit showthread	Included Files: ./printthread.php ./global.php ./includes/init.php ./includes/class_core.php ./includes/config.php ./includes/functions.php ./includes/class_hook.php ./includes/modsystem_functions.php ./includes/class_bbcode_alt.php ./includes/class_bbcode.php ./includes/functions_bigthree.php Hooks Called: init_startup init_startup_session_setup_start init_startup_session_setup_complete cache_permissions fetch_threadinfo_query fetch_threadinfo fetch_foruminfo style_fetch cache_templates global_start parse_templates global_setup_complete printthread_start bbcode_fetch_tags bbcode_create bbcode_parse_start bbcode_parse_complete_precache bbcode_parse_complete printthread_post printthread_complete
Messages: