google spiders 60% of my threads

[SkuZZy]

Originally posted by indiamike
Though it's a little off topic...and I just want to chime in a little bit here....
I use vbportal and google spiders my site almost everyday (usually 20 robots at a time), somtimes twice a day. It doesn't list all the posts in the search engine, at one time it did but dropped a bunch, however the big pitfall is that Google uses up around one gig of my bandwidth a month. I have a small site and that is still way to high.

I may start blocking Google just because of this....just a little warning if your inviting Google.

Cheers Anyway
Mike

Good point being made here. I can see how some people might be concerned with this. I should point out, the archive scripts i'm releasing are very, very small in size the way they come. Of course, if you add images and styles into them, then they get much bigger. But using them the way they are, the bandwidth used will be virtually un-noticable. When it comes to getting google to archive your posts, the simpler = the better