vbSpiderFriend - Search Engine Friendliness

I am tired of my 200,000 posts not being listed in Google. I was inspired by phpbuilder.com this morning and I wrote:

vbSpiderFriend - the search engine indexer for all of your posts

Purpose: Allow search engine spiders to crawl a linked list of all of your posts.

Project Requirements:

-Friendly URLs (no query strings)
-Good dynamic meta tags
-Never have to touch the script again.. It is Y3K compliant, simply re-submit to the engines to update your listings

Install Requirements:

-vBulletin 1.x or 2.x
-about 10 minutes


1) Download the attached Zip.

2) Open class.mysql.php and put your database login info at the top.

3) Create a new directory called archive under your forum, like /forum/archive

4) Open the included .htaccess and change the Error 404 to your new archive path.

5) Open index.php and change the self-explanatory variables at the top of the file.

6) Upload all 3 files to your archive directory.

7) Submit /forum/archive/index.php to search engines and watch em crawl


DISCLAIMER: I don't use 2.x but I checked the schema and this should work fine.

NOTES: This uses ErrorDocument and query string parsing to get the variables needed. I do not have the time or energy to troubleshoot this if it does not work on your server. Sorry!

[michaelab]

07-10-03 at 02:45 PM KevinM said this in Post #319
I have installed this hack on my test forum, and it works great on 15 out of 20 of my forums.

Of the ones that don't work, all the forum id's are single digit e.g.
http://www.ukgardeningchat.co.uk/archive/5 where the forum id is 5.

However, if I manually type in 05 at the end of the url instead, it works great. eg.

http://www.ukgardeningchat.co.uk/archive/05

but it does not do this automatically. All the forums from 10 upwards are fine. Any suggestions on how to make it work or force the url to read 05?

Kevin - see my post above. I think my fix will solve your problem also (looks like the same problem).

BTW, on the 404 issue and Google not liking it, I just checked my archive URLs using a little bit of Java I just knocked up and the response codes are all 200 OK so shouldn't be an issue.

Michael.

[Till]

I was wondering, what exactly is the use of those pages that list the months and week 1 through week 4 for each one? Or is that just your "design"?

[smestas]

Im getting the following when clicking on any of the archive links.

"No posts, please go back."

I have the newest version (v1.1b) which I just d-loaded. If anyone can lead me in the right direction I'd appreciate it.

Link to my archive pages:
http://www.alfabb.com/bb/forums/archive/index.php

[Till]

Originally Posted by smestas

Im getting the following when clicking on any of the archive links.

"No posts, please go back."

I have the newest version (v1.1b) which I just d-loaded. If anyone can lead me in the right direction I'd appreciate it.

Link to my archive pages:
http://www.alfabb.com/bb/forums/archive/index.php

Could be multiple reasons.

1) no posts
2) certain requirements not fulfilled

If I remember correctly, there's some sort of debug mode, maybe turn it on and see what it says.

In any case, no offense to the author, I would not recommend running this script. It never worked for me (although posts showed up, none were indexed due to a bug that's been fixed now though (see: status code)) and there's a ton of scripts on vB.org that do the same or maybe a better job. (I wrote one myself to get the job done.)

[smestas]

Till,

Thanks for the reply. My main problem is that I just moved to a new host and they do not support the mod_mime/ForceType directive. I previously was running (old server) a archive hack called "easy archive" and it worked great. I currently have around 4,000 pages getting indexed and I need to get somthing back up there for the bots to chew on.

I guess im going back to the drawing board. Any suggestions?

[Till]

Originally Posted by smestas

Till,

Thanks for the reply. My main problem is that I just moved to a new host and they do not support the mod_mime/ForceType directive. I previously was running (old server) a archive hack called "easy archive" and it worked great. I currently have around 4,000 pages getting indexed and I need to get somthing back up there for the bots to chew on.

I guess im going back to the drawing board. Any suggestions?

I am not familiar with easy archive, but why not rename the file to whatever.html and use that? Instead of "whatever". Then search and replace the source code and append the extension. Would be the easiest work around in order to not use forcetype.

I would also add a rewrite rule, so everything from /whatever gets rewritten to /whatever.html. You won't lose search engine traffic this way.

Then again, I don't see why they would not enable it for you. Especially if they want your business. Just remind them that there are 3 million companies out there offering webhosting.

[MrNase]

Hey

Is someone using that hack for 2.3.2 ?? Does it work?
I don't want to mess my forums

[Skyline_GT]

Can anyone help me ??

http://www.gamerzonline.net/forum/archive/

[michaelab]

Originally Posted by michaelab

BTW, on the 404 issue and Google not liking it, I just checked my archive URLs using a little bit of Java I just knocked up and the response codes are all 200 OK so shouldn't be an issue.

Confirmed! :banana: My vbSpiderFriend archive pages are showing up on Google. Took their time but they're definitely there now so there is no 404 issue. YES!

Michael.

[michaelab]

Originally Posted by MrNase

Hey

Is someone using that hack for 2.3.2 ?? Does it work?
I don't want to mess my forums

Yes - works fine with 2.3.2

Michael.