vbSpiderFriend - Search Engine Friendliness

I am tired of my 200,000 posts not being listed in Google. I was inspired by phpbuilder.com this morning and I wrote:

vbSpiderFriend - the search engine indexer for all of your posts

Purpose: Allow search engine spiders to crawl a linked list of all of your posts.

Project Requirements:

-Friendly URLs (no query strings)
-Good dynamic meta tags
-Never have to touch the script again.. It is Y3K compliant, simply re-submit to the engines to update your listings

Install Requirements:

-vBulletin 1.x or 2.x
-about 10 minutes


1) Download the attached Zip.

2) Open class.mysql.php and put your database login info at the top.

3) Create a new directory called archive under your forum, like /forum/archive

4) Open the included .htaccess and change the Error 404 to your new archive path.

5) Open index.php and change the self-explanatory variables at the top of the file.

6) Upload all 3 files to your archive directory.

7) Submit /forum/archive/index.php to search engines and watch em crawl


DISCLAIMER: I don't use 2.x but I checked the schema and this should work fine.

NOTES: This uses ErrorDocument and query string parsing to get the variables needed. I do not have the time or energy to troubleshoot this if it does not work on your server. Sorry!

[James Cridland]

All the links are supposed to link to non-existent folders, that's how this hack works.

Read carefully the bit about .htaccess

[Learner29]

James,

first thank you for your reply. I really appreciate that you are trying to help.


second, this hack supposed to generate an index.html file, something like a site map with links towards every week of every month of every year.... see this example http://www.overgrow.com/edge/archive/2/index.html

in my case, the directory "2" and all other same level directories do NOT exist...

Please help.

[Learner29]

James,

At the end of the day, the script or the hack is supposed to generate TEXT ONLY versions of your forums and your threads.

see this example and look well at the subfolders of the "archive" folder

http://www.overgrow.com/edge/archive/2/1999/08/1/2274



N.B. : Overgrow has put a "adult only" mark for his site.

[James Cridland]

Again: the pages are not supposed to exist.

The .htaccess thing catches the "this page doesn't exist" error, and runs this script instead. If the .htaccess thing doesn't work on your server, then you need to ask your ISP nicely for a custom 404 error page.

[Learner29]

I thank you very much. I will check and tell you if it works.

[Learner29]

Originally posted by James Cridland
Again: the pages are not supposed to exist.

The .htaccess thing catches the "this page doesn't exist" error, and runs this script instead. If the .htaccess thing doesn't work on your server, then you need to ask your ISP nicely for a custom 404 error page.

I have placed a .htaccess on the root, and another .htaccess in the public_html folder, and another .htacess in its subfolder "vb".

Should I make them all the same .htaccess file with that custom 404 line

ErrorDocument 404 http://www.mydomain.com/vb/archive/index.php

or should I simply delete those .htaccess files in the subfolders

I have SSH / Telnet access to my server, and can delete eventual .htaccess files.

[Learner29]

Ok, I have asked and got an answer about .htaccess files.

finally, I deleted ALL of my .htaccess files, except for the one on the root.

that one, holds this ErrorDocument 404 http://www.mydomain.com/vb/archive/index.php correctly.

it should work then now....

I tried, it does not

[FWC]

You need to put an .htaccess file like this in the /forums/archive folder:

ErrorDocument 404 /forums/archive/index.php

Change forums to the correct folder name for you board directory. This is all that's required along with the index.php and the class.mysql.php files.

[mvigod]

People...I guess you missed my earlier post in this thread...the 404 not found error WILL NOT WORK! Yes you will get the content page but the engines receive a 404 header thus thinking the page they are getting is an error page and thus is NOT indexed nor are links followed within it. Don't waste your time on this with the hack as is...

[Learner29]

People...I guess you missed my earlier post in this thread...the 404
not found error WILL NOT WORK! Yes you will get the content page but the
engines receive a 404 header thus thinking the page they are getting is an
error page and thus is NOT indexed nor are links followed within it. Don't
waste your time on this with the hack as is...

my friend this hack is one of the best hacks about search engine
friendliness. I have searched well on vbulletin.org, and did not find any
serious as competitive alternative.

Plus, this hack worked VERY fine for sooooo many people, why not us then ???
I would highly appreciate any help.....