Quote:
Originally Posted by xenite
Most of that traffic may have been crawlers.
Make sure your robots.txt is up-to-date.
|
PHP Code:
User-agent:
Mediapartners-Google
Disallow:
User-agent: *
Disallow: /go/
Disallow: /cgi-bin/
Disallow: /images/
Disallow: /admincp/
Disallow: /modcp/
Disallow: /attachment.php
Disallow: /search.php
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /editpost.php
Disallow: /profile.php
Disallow: /register.php
Disallow: /login.php
Disallow: /subscription.php
Disallow: /private.php
Disallow: /report.php
Disallow: /sendmessage.php
Disallow: /member.php
Disallow: /memberlist.php
Disallow: /misc.php
Disallow: /moderator.php
Disallow: /postings.php
Disallow: /sendtofriend.php
Disallow: /threadrate.php
Disallow: /usercp.php
Disallow: /showgroups.php
Disallow: /announcement.php
Hi, this is what the code in my robots.txt file. Is there any other option to update it?