Does Google hate big boards?
 

Hi, I was wondering what makes some big boards be excluded from Google. Does Google penalize these boards for some reason?

Gaia Online, 1,074,929,715 posts:
Google: about 132,000 from gaiaonline.com

FaceTheJury, 135,719,579 posts:
Google: about 33,100 from forums.facethejury.com

Offtopic.com, 79,896,449 posts:
Google: about 141 from forums.offtopic.com
(!!!)

nope,
www.atraf-f.net - 1,200,000~ posts
Results 1 - 10 of about 172,000 from www.atraf-f.net. (0.19 seconds)

in google israel we have 233,000 results

That's the thing. Smaller forums seem to have more indexed pages that bigger ones.

The forum you mentioned has just over 1 million posts, and still has more indexed pages that the forum with 1 BILLION posts.

And that's not the only example. I've heard Google banning some sites with inappropriate content, but all big forums can't fall into this category... And still... Big postcount, small number of indexed pages.

???

I think some big boards ask google to exclude them from their index - because they don't WANT to get bigger.

Hmm, I never thought of that... This sounds resonable...

some sites exclude web crawlers in general not only because of the advertisement, but also because it is resource expensive...

when a web crawler visit your site, it hits hundreds of pages in a minute, compared to a member who can hit 3 or 4 pages per minute...

i have on my site yahoo that crawl my site once per day, 58 to 100 hits per minute... and go after 10 minutes... i have 0 members... just infos... imagine a site with 1 million posts... yahoo would crawl the site like 1000 times the count i have... webhosts hates that...

not at all.

I know yahoo has this horrible habbit of sending 50-100 bots at a time, had 50 the other day on my page.

Maybe bad SEO ;)

Most of Offtopic.com's forum are not visible to guests, so it makes sense that they don't have many indexed pages. I don't know about the rest of them though.