Limey-YMR
07-07-2004, 01:57 AM
I help run a car club forum, it's something of a membership by association of car colour, therefore we aren't too bothered about having too many new (and potential non-same-car owning members)
I was wondering if it was possible by .htaccess or by hack of configuring our server so that spiders, mainly MSN, google and Inktomi/Yahoo could only "see" the Archive.
We are at 1and1 hosting - 100MB mySQL limit :( and today our web server was blocked from the database server for exactly two hours (to the minute), I believe this was their own firewall blocking us, possibly due to "over spidering" or our forum was just plain too busy for 1and1's sensitive firewall / IDS rules whatever the block cause, I would like to see if someone knows of an efficient way of herding the spiders :)
so I can spend time on admin/installing hacks and not have to learn how to code them or .htaccess files
Any help is greatly appreciated.
you know what they say - you can lead a spider to an archive, but you can't make it index.
I was wondering if it was possible by .htaccess or by hack of configuring our server so that spiders, mainly MSN, google and Inktomi/Yahoo could only "see" the Archive.
We are at 1and1 hosting - 100MB mySQL limit :( and today our web server was blocked from the database server for exactly two hours (to the minute), I believe this was their own firewall blocking us, possibly due to "over spidering" or our forum was just plain too busy for 1and1's sensitive firewall / IDS rules whatever the block cause, I would like to see if someone knows of an efficient way of herding the spiders :)
so I can spend time on admin/installing hacks and not have to learn how to code them or .htaccess files
Any help is greatly appreciated.
you know what they say - you can lead a spider to an archive, but you can't make it index.