Quote:
Originally Posted by perfphysio
Can you give us a bit more insite into what that is and how one goes about it?
:cross-eyed:
|
3. Restrict Search Engine Crawling
Those spiders..they index your content, but there are always places that you don't want or need them to go.
To restrict the pesky critters to only eating up your Content, do the following.
In your root (public_html) directory, upload the following robots.txt::
Code:
User-agent: *
Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/member.php
Disallow: /forums/memberlist.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/profile.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/showgroups.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php
this assumes your forum is in the /forums/ folder
also.. to the thread starter,, your site doesn't have a bunch of links to the forum.. check out my forum, its in my sig.. look at the footer and see the archive links.. I'd suggest doing something like this with links to your forum sections
This will help a Lot !
good luck