The Arcive of Official vBulletin Modifications Site.

About Developer · **Released**: 04-29-2001

I am tired of my 200,000 posts not being listed in Google. I was inspired by phpbuilder.com this morning and I wrote:

vbSpiderFriend - the search engine indexer for all of your posts

Purpose: Allow search engine spiders to crawl a linked list of all of your posts.

Project Requirements:

-Friendly URLs (no query strings)
-Good dynamic meta tags
-Never have to touch the script again.. It is Y3K compliant, simply re-submit to the engines to update your listings

Install Requirements:

-vBulletin 1.x or 2.x
-about 10 minutes

1) Download the attached Zip.

2) Open class.mysql.php and put your database login info at the top.

3) Create a new directory called archive under your forum, like /forum/archive

4) Open the included .htaccess and change the Error 404 to your new archive path.

5) Open index.php and change the self-explanatory variables at the top of the file.

6) Upload all 3 files to your archive directory.

7) Submit /forum/archive/index.php to search engines and watch em crawl

DISCLAIMER: I don't use 2.x but I checked the schema and this should work fine.

NOTES: This uses ErrorDocument and query string parsing to get the variables needed. I do not have the time or energy to troubleshoot this if it does not work on your server. Sorry!

mvigod · #**232** 07-13-2002, 06:10 PM

Is the 404 page it's sending out going to be archived by Google? When google spiders it they will get a page of text (the error page delivered by this script) but with a 404 header which is "not found"....because of this will they bother to index it? Shouldn't this be a rewrite rule to redirect google transparently rather than a 404 error?

mvigod · #**233** 07-18-2002, 02:51 PM

Has anyone ever witnessed Google going more than one level deep? Since they get the 404 respose code with this hack they don't index those pages or go any deeper. They are not in the habit of indexing "Not Found 404 error" pages or following the links in them. They will follow the first set of links from the index.php but each of those really doesn't exist so the 404 error is the end of the line even though the pages have content.

I think this has to be rewritten with mod rewrite to rewrite the requests for all pages back to index.php so a 200 OK response code is given and it will be archived.

With all these posts didn't anybody realize this?

James Cridland · #**234** 07-21-2002, 01:32 PM

What might be interesting is to check for Google - or "search" - in the referrer, and don't give a <META REFRESH, but properly redirect them to the proper version of the page automatically.

That means that Google won't be given the changed location when it spiders, but any normal user, following a Google search, will get the proper redirect.

You can do this by...

1. Changing $refresh=0 to $refresh=1

2. Find $forwarding=1

Replace:

PHP Code:


			
            $forward= "<script language=\"javascript\">document.location.href='$forumURL/showthread.php?threadid=$threadID';";

            $forward.= "document.write('<font face=verdana,arial size=2><b><center>Please wait while the new version is loading...</center></b></font><br>&nbsp;<br>');</script>";

            $metas="<META HTTP-EQUIV=\"REFRESH\" CONTENT=\"0;$forumURL/showthread.php?threadid=$threadID\"></head><body></body></html>";

            $forwarding=1;

with

PHP Code:


			
header ("Location: $forumURL/showthread.php?threadid=$threadID");

exit;

3. Just above that chunk of text, find

PHP Code:


			
if ((!stristr(getenv(HTTP_REFERER),$homeURL)) or (strlen(getenv(HTTP_REFERER)) < 1)) {

with

PHP Code:


			
if ((!stristr(getenv(HTTP_REFERER),"text")) or (strlen(getenv(HTTP_REFERER)) < 1)) {

where text is a bit from your URL. I.e. if you're hosted on www.mydomain.com then put "mydomain" in there.

What does the panel think?

veedee · #**235** 07-21-2002, 02:57 PM

I hate to keep asking this in all the hacks but..

I'm running 2.2.6 will it work for this?

Cheers,

veedee

James Cridland · #**236** 07-21-2002, 06:47 PM

Yes it will.

Incidentally, I've used this hack as a basis to create an Avantgo / iSilo PDA version of my forums... check them out at http://forums-lite.mediauk.com/

I've rather heavily hacked this out, though.

(Amusingly, this is not search-engine friendly...!!)

EchoHype.com · #**237** 07-25-2002, 08:39 AM

Great hack mate!

Learner29 · #**238** 07-27-2002, 06:18 PM

oh. I am sorry to bother but I really tried to find by myself but could not !!

I installed the hack and the index.php page of the /vb/archive directory is displaying nicely.

Yet all the links on this index.php page point toward non-existent folders!!!

example

http://www.mydomain.com/vb/archive/29
and
http://www.mydomain.com/vb/archive/13

but checking that archive folder by FTP, there are NO Folders called 13 or 29 ....

I am so frustrated as I DID install this hack in the past and it worked nicely then.....

Whoever would help, I would be more than grateful and thankful.

Learner29 · #**239** 07-30-2002, 07:22 PM

I can't believe nobody bothered to help me in almost 3 days now....

Anybody ??? Any ideas ???

tiger · #**240** 08-03-2002, 07:35 AM

I have the same problem

Learner29 · #**241** 08-03-2002, 12:56 PM

will any kind person step out and help a bit.. ??

X vBulletin 3.8.12 by vBS Debug Information
Page Generation 0.13254 seconds Memory Usage 2,316KB Queries Executed 25 (?)
More Information
Template Usage: (1)SHOWTHREAD (1)ad_footer_end (1)ad_footer_start (1)ad_header_end (1)ad_header_logo (1)ad_navbar_below (1)ad_showthread_beforeqr (4)bbcode_php (1)footer (1)forumjump (1)forumrules (1)gobutton (1)header (1)headinclude (1)modsystem_post (1)navbar (6)navbar_link (120)option (1)pagenav (1)pagenav_curpage (4)pagenav_pagelink (1)pagenav_pagelinkrel (11)post_thanks_box (11)post_thanks_button (1)post_thanks_javascript (1)post_thanks_navbar_search (11)post_thanks_postbit_info (10)postbit (11)postbit_onlinestatus (11)postbit_wrapper (1)spacer_close (1)spacer_open (1)tagbit_wrapper Phrase Groups Available: global inlinemod postbit posting reputationlevel showthread	Included Files: ./showthread.php ./global.php ./includes/init.php ./includes/class_core.php ./includes/config.php ./includes/functions.php ./includes/class_hook.php ./includes/modsystem_functions.php ./includes/functions_bigthree.php ./includes/class_postbit.php ./includes/class_bbcode.php ./includes/functions_reputation.php ./includes/functions_post_thanks.php Hooks Called: init_startup init_startup_session_setup_start init_startup_session_setup_complete cache_permissions fetch_threadinfo_query fetch_threadinfo fetch_foruminfo style_fetch cache_templates global_start parse_templates global_setup_complete showthread_start showthread_getinfo forumjump showthread_post_start showthread_query_postids showthread_query bbcode_fetch_tags bbcode_create showthread_postbit_create postbit_factory postbit_display_start post_thanks_function_post_thanks_off_start post_thanks_function_post_thanks_off_end post_thanks_function_fetch_thanks_start post_thanks_function_fetch_thanks_end post_thanks_function_thanked_already_start post_thanks_function_thanked_already_end fetch_musername postbit_imicons bbcode_parse_start bbcode_parse_complete_precache bbcode_parse_complete postbit_display_complete post_thanks_function_can_thank_this_post_start pagenav_page pagenav_complete tag_fetchbit_complete forumrules navbits navbits_complete showthread_complete
Messages:

The Arcive of Official vBulletin Modifications Site.

It is not a VB3 engine, just a parsed copy!