Quote:
Originally Posted by Antivirus
You need to add nofollow tag to any links which reference the page you want to prevent spiders from accessing like so:
<a href="spidersforbidden.php" rel="nofollow">link</a>
This will prevent spiders from crawling the frbidden script.
|
You are referring to "good" spiders? If the script needs to be off-limits to spiders, would that require a rule in .htaccess ? Just a thought.