vbSpiderFriend - Search Engine Friendliness

I am tired of my 200,000 posts not being listed in Google. I was inspired by phpbuilder.com this morning and I wrote:

vbSpiderFriend - the search engine indexer for all of your posts

Purpose: Allow search engine spiders to crawl a linked list of all of your posts.

Project Requirements:

-Friendly URLs (no query strings)
-Good dynamic meta tags
-Never have to touch the script again.. It is Y3K compliant, simply re-submit to the engines to update your listings

Install Requirements:

-vBulletin 1.x or 2.x
-about 10 minutes


1) Download the attached Zip.

2) Open class.mysql.php and put your database login info at the top.

3) Create a new directory called archive under your forum, like /forum/archive

4) Open the included .htaccess and change the Error 404 to your new archive path.

5) Open index.php and change the self-explanatory variables at the top of the file.

6) Upload all 3 files to your archive directory.

7) Submit /forum/archive/index.php to search engines and watch em crawl


DISCLAIMER: I don't use 2.x but I checked the schema and this should work fine.

NOTES: This uses ErrorDocument and query string parsing to get the variables needed. I do not have the time or energy to troubleshoot this if it does not work on your server. Sorry!

[Admin]

OmniSlash31 please enter your license information in your profile. Thanks.

[Chamber]

Excellent hack idea!! Wanna install it - is it good to go on V2.2.0 ?

[MrLister]

it sure is... it would work on any..

[JJR512]

Does anybody know how to get this to work with sub-domain addresses (like forums.jjr512.com, instead of www.jjr512.com/forums)?

[eva2000]

Originally posted by Chamber
Excellent hack idea!! Wanna install it - is it good to go on V2.2.0 ?

yup working on my 2.2.0 forums

as to subdomain not sure i think it's the same as domain just need to put the modrewrite stuff in the virtualhost entry for the subdomain?

[Logtenberg]

Has anyone had their archive site crawled by google yet?

[mjames]

[edit]Got it working![/edit]

[djr]

Although I installed this a few days ago (which was a breeze actually!), I'm a little confused now where to put the .htaccess file.
In the instructions Overgrow put it like this:

1) Download the attached Zip.
2) Open class.mysql.php and put your database login info at the top.
3) Create a new directory called archive under your forum, like /forum/archive
4) Open the included .htaccess and change the Error 404 to your new archive path.
5) Open index.php and change the self-explanatory variables at the top of the file.
6) Upload all 3 files to your archive directory.
7) Submit /forum/archive/index.php to search engines and watch em crawl

However, in this thread I see several mentions of putting the .htaccess in the forums path (/forums instead of /forums/archive).

Where do I put the .htaccess? In /forums or in /forums/archive?

- djr

[FWC]

Originally posted by djr
Where do I put the .htaccess? In /forums or in /forums/archive?

- djr

Put Overgrow's .htaccess file in /forums/archive.

[djr]

FWC,

Thanks, that clarifies a bit :-) Another question though: do I need some setting (.htaccess/meta) elsewere to prevent Google/others from spidering the normal forums, so it spiders the /archive directory instead?

- djr