Yet another ROBOTS.txt file question (sorry)

[zascok]

Originally Posted by Nichtofen

Curiously,
Within your code you do not have a "forums/" prefix on your items. That would be required if your forum was located within a forums folder in the root, correct?

all the same with /forums on front of each line for the forum itself, the rest is up to what you have in the root. I just don't have anything else but forum so it's right in the top.

Disallow: /forums/*.js
...
...
Disallow: /forums/xmlsitemap.php

[Nichtofen]

To clarify:

Disallow: /forums/*.js
...
...
Disallow: /forums/xmlsitemap.php

Has the same exact effect as:

Disallow: /*.js
...
...
Disallow: /xmlsitemap.php

Just wanted to make sure I understood correctly. I will not bother with changing unless it is necessary. If it automatically seeks a sub-folder on the server anywhere with that name then I will just leave it.

Thanks in advance zascok!

--------------- Added [DATE]1344206047[/DATE] at [TIME]1344206047[/TIME] ---------------

Originally Posted by zascok

I just don't have anything else but forum so it's right in the top.

Gotcha, thanks!

[invitezone]

Originally Posted by zascok

yup and nope you can't leave it at that you gotta up it into the root of your forum

hehehe, erm yeah I understand that much
I meant if I don't want to ban any bots or spiders from my site, I just want to limit them to the usual stuff, I can just leave your example file as it is, unedited, and upload that to forum root right?

Thanks

[Nichtofen]

That is correct. Put robots.txt into your root with the contents that zascok gracefully provided.