PDA

View Full Version : SE Spider access


doogie88
07-07-2007, 06:16 PM
I don't know if this is possible or not.
I have a VIP section of my site that is for paying memebers.
You get access to hundreds of articles.
But your access needs to be upgraded.
I was wondering if it would be possible to grant the SE spiders access to this area to spider all these articles for search engines.
Now I guess when a visitor in the SE clicks the link it will say they can't access the page, but it would still be listed.

Is this possible?

doogie88
07-09-2007, 06:13 PM
up.

The_VarMaster
07-09-2007, 06:28 PM
It is possible, but if your visitor change his HTTP USER AGENT header to header of SE he will see all of articles.

doogie88
07-10-2007, 08:18 PM
I don't really understand what that means?

magnus
07-11-2007, 06:37 PM
This is against Google's TOS. You're not allow to spider content which is not readily available to all users.