Bots crawling private forums

[hornos]

Hi,

I've noticed that googlebot also crawls private forums. Of course, access to a specific thread in a private forum will be denied.

- However, is there a way do disable spidering on specific forums, private ones for instance ?
- Is there a chance that those private threads will be indexed in Google ? ... hope not :nervous:

Thanks.

[amykhar]

If you have adsense banners showing in the private forums, the adsense spider will try to view the page immediately after the banner is displayed for the first time. But, it will only see the no permission message.

Amy

[Princeton]

search engines follow all links on a page ... what they find on that page depends on permissions set for unregistered users