Log in

View Full Version : Bots following links


MeNeedHelp
12-27-2004, 10:00 PM
Hi,

How do I prevent Bots from following links on my site.
I currently count it when someone follows a link on my site but Bots seem to be able to follow all links and thus my count sky-rockets.

Thanks for any input.
MeNeedHelp (I really do :squareeyed: ).

manguish
12-29-2004, 08:46 PM
robots.txt

Try something like this :


User-agent: *
Disallow: /vb/modcp/
Disallow: /vb/admincp/
Disallow: /cgi-bin
Disallow: /scgi-bin


Add in the links you want them to not follow.

I'm guessing this will work, but will stand to be corrected ;)

MeNeedHelp
12-29-2004, 09:09 PM
Thanks for the reply, however, I have almost 300 links and will not be able to maunally disallow each link (well, do not want to, is a better word :rolleyes: ).