Google - robots.txt - vb.org Archive

		vb.org Archive > vBulletin 3 Discussion > vB3 General Discussions
Google - robots.txt

Reply

#1

Old

10-29-2009, 08:40 PM

Wifey

Wifey is offline

Join Date: Mar 2004

Posts: 250

Благодарил(а): 0 раз(а)

Поблагодарили: 0 раз(а) в 0 сообщениях

Default

Google - robots.txt

So I've just FINALLY gotten around to doing this. My question is, what files should I restrict from the Google Crawl?

Reply With Quote

#2

Old

10-29-2009, 09:00 PM

Lynne

Lynne is offline

Join Date: Sep 2004

Location: California/Idaho

Posts: 41,180

Благодарил(а): 0 раз(а)

Поблагодарили: 0 раз(а) в 0 сообщениях

Default

Restrict those that you don't want spidered. I think there are a few threads here where users have posted their robots.txt files. Just do a search and you should find them.

Reply With Quote

Reply

« Previous Thread | Next Thread »

Posting Rules
You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules

Forum Jump

All times are GMT. The time now is 08:15 PM.