perfect robots.txt for vb 3.7?

[punchbowl]

Originally Posted by Boofo

If they can get on the server, they can find the directory. There is no foolproof way to stop someone who wants to get to it bad enough. All you can do is slow them down. But good luck with your theory.

It's not my theory. I think it may have been suggested on here. (Edit: Maybe not. Mentioned here http://www.vbulletin.com/forum/showthread.php?t=194701 also remember this?

//	****** PATH TO ADMIN & MODERATOR CONTROL PANELS ******
	//	This setting allows you to change the name of the folders that the admin and
	//	moderator control panels reside in. You may wish to do this for security purposes.
	//	Please note that if you change the name of the directory here, you will still need

)

I only allow people http access to my server. Apart from guessing I'm a big fan of the flintstones it does provide extra hoops that any hacker will have to jump through. Not theoretical hoops either : They have to guess another defacto password. If someone has access to my server through ssh or whatever then it's already too late.

From reading on here 90% of board hacks are through a browser simply putting admincp after the board url and guessing. A large percentage of these 'hacks' are inside jobs by disgruntled mods who presumably know their way around a standard installation.

Why exclude something by name when googlebot only follows links and the admin link will never appear for guests? Are you putting your admin cp in a sitemap too?! (j/k re sitemap!)

Either way it's not a major issue but I just don't see the point in listing it.

[Brandon Sheley]

Originally Posted by jambo_1969

Make sure you add in

User-agent: Fasterfox
Disallow: /

I managed to catch a thread on my board where people were discussing have Fasterfox set as turbo charged...my loads went through the roof !!

Does anyone have any info on this? Does it really work? What's the point?

I've never heard of this before, I'll add it if it helps, but I really hate messing with my robots.txt file, I don't want to stop the bots from me site

thanks
-Brandon

[punchbowl]

Originally Posted by Loco.M

Does anyone have any info on this? I've never heard of this before

thanks
-Brandon

Yeah I use it. Worth it.

There's a few other download accelerators too but some allow surfers spoof their useragent which is unfortunate.

[Phooey]

Does the fasterfox addon in your robots.txt completely block anyone using fasterfox or just blocks the functionality of the add-on on the site?

[s3nt3nc3d]

@Phooey
I'd assume it just blocks the functionality of the add-on

I run multiple websites on my server...only one is semi-large and having issues with spiders hogging all the resources (crashing my database multiple times a day due to having too many connections). Should I upload that file into the root folder of that one site or should I modify it and upload it into my server root folder where my xxxx.com, yyyyy.com, zzzzz.com, etc folders are located for my individual websites? I'm using 3.6.9 (soon to upgrade to 3.6.10--not quite ready for the jump to 3.7 but eventually will do so), so I'm gonna have to modify this a little to suit my needs prolly.

[Jase2]

Originally Posted by Bradley_Wint

don't mind me sounding like a newbie here but i added the content to a robots.txt file, upped it to my server html root folder like 8 hrs ago..now when i check my who's online list, i still see google bots murdering my calender.php, login.php...etc...

anything i did wrong?

Give it a while. But are you sure they don't have the red error icon next to them?

Here's what I use :

User-agent: *
Disallow: /forum/admincp/
Disallow: /forum/album.php
Disallow: /forum/announcement.php
Disallow: /forum/calendar.php
Disallow: /forum/cron.php
Disallow: /forum/editpost.php
Disallow: /forum/faq.php
Disallow: /forum/showgroups.php
Disallow: /forum/images/
Disallow: /forum/joinrequests.php
Disallow: /forum/login.php
Disallow: /forum/memberslist.php
Disallow: /forum/misc.php
Disallow: /forum/modcp/
Disallow: /forum/moderator.php
Disallow: /forum/newreply.php
Disallow: /forum/newthread.php
Disallow: /forum/online.php
Disallow: /forum/poll.php
Disallow: /forum/printthread.php
Disallow: /forum/private.php
Disallow: /forum/register.php
Disallow: /forum/report.php
Disallow: /forum/search.php
Disallow: /forum/sendmessage.php
Disallow: /forum/showpost.php
Disallow: /forum/subscription.php
Disallow: /forum/subscriptions.php
Disallow: /forum/threadrate.php
Disallow: /forum/usercp.php
Disallow: /forum/usernote.php

[SEOvB]

Heres more information on Robots.txt for vBulletin including some other files and folders you should block out.

After you upload it, it could take up to 24hrs for spiders to find and start to follow the rules of robots.txt most only download it once a day.

[Boofo]

Too bad that link doesn't work.

[Jase2]

It does work....