Closing Board keeps out Spiders

[yinyang]

I've always wondered about the following:

If you close a board, i.e. make it so posts can only be read if you register, does it keep out spiders/search engines?

If so, is there a way to close a board and allow spiders/search engines?

[Colin F]

I've always wondered about the following:

If you close a board, i.e. make it so posts can only be read if you register, does it keep out spiders/search engines?

If so, is there a way to close a board and allow spiders/search engines?
Yes, it does.
Spiders generally see the same content as guests.

Be aware that displaying different content to search engine spiders and regular users is against a lot of the search engines rules...

[Andreas]

If you close a board, i.e. make it so posts can only be read if you register, does it keep out spiders/search engines?
Yes

If so, is there a way to close a board and allow spiders/search engines?
Not really ... you could check UAs and give access if you find a match for a Spider, but UAs can be set by the User => everybody has access.
Or you can check IPs. This is more difficult to be faked, but still possible - and it would IMHO be hard to get a list of Spider IPs.

Be aware that displaying different content to search engine spiders and regular users is against a lot of the search engines rules...

Right. I always wonderes if assigning an ultra-light style (like the Archive) to Spiders would also be considered Cloaking?

[Marco van Herwaarden]

If you close a board, i.e. make it so posts can only be read if you register, does it keep out spiders/search engines?
Yes this will keep out spiders. Spiders follow the permissions that are set for the 'Unregistered / Not logged in' Usergroup. They can only index pages they can see.

If so, is there a way to close a board and allow spiders/search engines?
Yes it could be done with some modifications. It would however be against the TOS of all search engines and if you are caught, you would be totally removed and blocked by them.

[yinyang]

ok. thanks for the info guys.

i just wanted the spiders to see what the regular members saw, and not what the unregistered members saw. but you've answered my questions. thanks!