vbSpiderFriend - Search Engine Friendliness

I am tired of my 200,000 posts not being listed in Google. I was inspired by phpbuilder.com this morning and I wrote:

vbSpiderFriend - the search engine indexer for all of your posts

Purpose: Allow search engine spiders to crawl a linked list of all of your posts.

Project Requirements:

-Friendly URLs (no query strings)
-Good dynamic meta tags
-Never have to touch the script again.. It is Y3K compliant, simply re-submit to the engines to update your listings

Install Requirements:

-vBulletin 1.x or 2.x
-about 10 minutes


1) Download the attached Zip.

2) Open class.mysql.php and put your database login info at the top.

3) Create a new directory called archive under your forum, like /forum/archive

4) Open the included .htaccess and change the Error 404 to your new archive path.

5) Open index.php and change the self-explanatory variables at the top of the file.

6) Upload all 3 files to your archive directory.

7) Submit /forum/archive/index.php to search engines and watch em crawl


DISCLAIMER: I don't use 2.x but I checked the schema and this should work fine.

NOTES: This uses ErrorDocument and query string parsing to get the variables needed. I do not have the time or energy to troubleshoot this if it does not work on your server. Sorry!

[eva2000]

great works perfectly now

one last thing.. how do i prevent private forums from being listed/outputted and indexed ?

thanks much appreciated

[Chris Schreiber]

Excellent and easy to install hack, thanks

I even liked the URL submission tool you linked to!

[Overgrow]

There ya go.. that is an important addition, eh?

There's a new zip file up top yonder.. the index.php has a new variable that holds your private forumid's.

[Brian]

One suggestion would be to show a suffix (.php , .shtml etc) so that is dynamic so the search engines know to spider it slower than if it was html. If they assume its html they might take on a tone at once as doing that to normal html files should be fine but this is actually doing all the db calls etc.

Just a suggestion but this is very nice!!

-Brian

[Streicher]

I have tested your hack and my private forums are not hidden.

And also some thread are not found, by the index.php

When i click on some forumlinks the forumpage is simply reloaded.

[Overgrow]

Thanks guys,

Streicher, you need to specify which forums are private at the top of index.php. Look for

$privateForums="|17|18|";

and put whatever forums in there that should be hidden. Enclose them with |pipes|. Anyone else having these same problems with reloading, etc?

Brian, my main target here is google. It does fine on other parts of my site where I do these type of query strings. If you do find that some engines are hesitant, let me know or make the change and tell us what you did.

[Stephan Whelan]

Overgrow,

I'm interested in the hack but can't seem to get it to work.

I've followed your instructions but am getting an Error 500 Internal Server Error when I try and run the script.

I'm running on a Cobalt RAQ4i so I don't know if there is a problem with the .htaccess file and the way the RAQ handles it.

Any ideas?

[NickyNet]

hi..

great hack..

but not working for me..

http://www.nicky.net/foren/archiv/

[Overgrow]

Nicky:

Here is your error

"The requested URL /foren/archiv/11 was not found on this server.
Additionally, a 404 Not Found
error was encountered while trying to use an ErrorDocument to handle the request."

That bottom line is the key.. it says your .htaccess is not pointing to the proper file. Edit the .htaccess and play around with it until you get it right. Sometimes the link has to be ../relative sometimes you can make it absolute /foren/archiv/index.php

Stephan can you give me a link? I use a Raq4i as well and it works for me. I did have to change the access.conf and put Allow Override ALL in there so htaccess could control the errordocument.

[Stephan Whelan]

Originally posted by Overgrow
Stephan can you give me a link? I use a Raq4i as well and it works for me. I did have to change the access.conf and put Allow Override ALL in there so htaccess could control the errordocument.

http://forums.deeperblue.net/archive/index.php