The Arcive of Official vBulletin Modifications Site.

About Developer · **Released**: 04-29-2001

I am tired of my 200,000 posts not being listed in Google. I was inspired by phpbuilder.com this morning and I wrote:

vbSpiderFriend - the search engine indexer for all of your posts

Purpose: Allow search engine spiders to crawl a linked list of all of your posts.

Project Requirements:

-Friendly URLs (no query strings)
-Good dynamic meta tags
-Never have to touch the script again.. It is Y3K compliant, simply re-submit to the engines to update your listings

Install Requirements:

-vBulletin 1.x or 2.x
-about 10 minutes

1) Download the attached Zip.

2) Open class.mysql.php and put your database login info at the top.

3) Create a new directory called archive under your forum, like /forum/archive

4) Open the included .htaccess and change the Error 404 to your new archive path.

5) Open index.php and change the self-explanatory variables at the top of the file.

6) Upload all 3 files to your archive directory.

7) Submit /forum/archive/index.php to search engines and watch em crawl

DISCLAIMER: I don't use 2.x but I checked the schema and this should work fine.

NOTES: This uses ErrorDocument and query string parsing to get the variables needed. I do not have the time or energy to troubleshoot this if it does not work on your server. Sorry!

Pilot · #**222** 05-23-2002, 06:26 PM

Absolutely, it would be awful (and very easy) to forget to add a new private forum to the exclude list and find your private posts indexed by search engines, you might even get sued for breach of privacy and you can never get rid of the things once in the search engines. Really a big risk.

MUCH safer to list the forums that you want indexed and have the code ignore any others. No risk of mistakes then.

tpearl5 · #**223** 05-27-2002, 02:29 PM

I finially got everything indexed on google! YEY! The only problem I see is this:

http://www.google.com/search?num=100...=Google+Search

See all the 'No posts, please go back' ? I'm afraid google indexed too many of these pages and not enough of the actual threads. Is there a way to turn off the pages without topics?

DarkReaper · #**224** 05-28-2002, 09:19 PM

Quote:

Originally posted by DarkReaper
Is there a way to get it to know which forums are private automatically, and not display those? I've got too big of a forum to manually maintain all the private forum ids.

...

apfeifer · #**225** 06-01-2002, 03:38 AM

For those of you who were having posts not show up, you may have to change some code. The original has it skip all but one second of the last day of each of the 'weeks' to fix this, find:

Code:

	$date1 = "$month/$fw/$year";
	$date2 = "$month/$lw/$year";

	$ts1 = strtotime("$date1");
	$ts2 = strtotime("$date2");

	echo spacer(2)."Dates: $date1 to $date2<br>&nbsp;<br>";

	$query = "SELECT title,threadid,lastpost FROM thread WHERE lastpost > '$ts1' AND lastpost < '$ts2' AND forumid='$forumID' ORDER BY dateline ASC";

And replace it with:

Code:

	$date1 = "$month/$fw/$year";
	$date2 = "$month/$lw/$year";

	$ts1 = strtotime("$date1");
	$ts2 = strtotime("$date2");
	$ts3 = $ts2 + 86399;

	echo spacer(2)."Dates: $date1 to $date2<br>&nbsp;<br>";

	$query = "SELECT title,threadid,lastpost FROM thread WHERE lastpost > '$ts1' AND lastpost < '$ts3' AND forumid='$forumID' ORDER BY dateline ASC";

This may not be the problem everyone is having, but it is the problem I was having. Hope it helps someone!

apfeifer · #**226** 06-01-2002, 03:44 AM

Another thing, if you don't want it to display categories that can't contain threads find:

Code:

	$query = "SELECT title,forumid FROM forum$whereclause ORDER BY forumid ASC";

And change it to:

Code:

	$query = "SELECT title,forumid FROM forum$whereclause AND cancontainthreads='1' ORDER BY forumid ASC";

Just another little nit-picky thing I wanted to add.

Thomas P · #**227** 06-01-2002, 03:17 PM

Too bad, doesn't work for me, it simply reloads the page again and again...

my code:

PHP Code:


			
<?



/*



vbSpiderFriend v1.1b by ~shabang~





*/



$privateForums="|8|14|9|15|16|12|19|20|30|35|36|"; // Hidden forumids, enclosed by | pipes

$firstPost="11/05/2000"; // MM/DD/YYYY of your forum's first post

$spacer="&nbsp;&nbsp;"; // The characters or spaces to use as one indent

$forumURL=""; // Base URL of your forum

$refresh=0; // Change this to 1 if you want the archive to automatically

            // forward the user if they come in from a search engine.

            // This option is initially turned off because Google considers

            // this to be 'cloaking.'



$homeURL="http://www.mcseboard.de"; // The link URL for the top of the page

$homeLink="Zur?ck zu MCSEboard.de - Deutsches MCSE Forum zu Windows 2000 & XP"; // The link text for the top of the page



$keywords="(...some stuff...)";  // SET META INFORMATION HERE

$description="(...some stuff...)";             // The script will add to these fields

$pagetitle="MCSE Forum zu Windows 2000 und Windows XP - MCSEboard.de";         // with the info from the thread



include("class.mysql.php");



// NOTHING MORE TO EDIT BELOW /////////////////////////////////////////////////////////////





$baseURL=$forumURL."/archiv";

$dateSplit = split("/",$firstPost);

$firstMonth = preg_replace("/^0/","",$dateSplit[0]);

$firstYear = $dateSplit[2];

$currentYear=date("Y",time());

$currentMonth=preg_replace("/^0/","",date("m",time()));

$fullHomeLink= "<center><a href=\"$homeURL\"><b>$homeLink</b></a><br>&nbsp;<br>";



(...)

Any ideas?

http://www.mcseboard.de/archiv/

Thanks,
-Tom

diettalk · #**228** 06-01-2002, 05:58 PM

Try this line...

$forumURL=""; // Base URL of your forum

should be something like:

$forumURL="/forums"; // Base URL of your forum

Quote:

Originally posted by Thomas P
Too bad, doesn't work for me, it simply reloads the page again and again...

my code:

PHP Code:

<? /* vbSpiderFriend v1.1b by ~shabang~ */ $privateForums="|8|14|9|15|16|12|19|20|30|35|36|"; // Hidden forumids, enclosed by | pipes $firstPost="11/05/2000"; // MM/DD/YYYY of your forum's first post $spacer="  "; // The characters or spaces to use as one indent $forumURL=""; // Base URL of your forum $refresh=0; // Change this to 1 if you want the archive to automatically // forward the user if they come in from a search engine. // This option is initially turned off because Google considers // this to be 'cloaking.' $homeURL="http://www.mcseboard.de"; // The link URL for the top of the page $homeLink="Zur?ck zu MCSEboard.de - Deutsches MCSE Forum zu Windows 2000 & XP"; // The link text for the top of the page $keywords="(...some stuff...)"; // SET META INFORMATION HERE $description="(...some stuff...)"; // The script will add to these fields $pagetitle="MCSE Forum zu Windows 2000 und Windows XP - MCSEboard.de"; // with the info from the thread include("class.mysql.php"); // NOTHING MORE TO EDIT BELOW ///////////////////////////////////////////////////////////// $baseURL=$forumURL."/archiv"; $dateSplit = split("/",$firstPost); $firstMonth = preg_replace("/^0/","",$dateSplit[0]); $firstYear = $dateSplit[2]; $currentYear=date("Y",time()); $currentMonth=preg_replace("/^0/","",date("m",time())); $fullHomeLink= "<center><a href=\"$homeURL\"><b>$homeLink</b></a><br> <br>"; (...)

Any ideas?

http://www.mcseboard.de/archiv/

Thanks,
-Tom

Thomas P · #**229** 06-01-2002, 08:49 PM

Quote:

Originally posted by diettalk
Try this line...

$forumURL=""; // Base URL of your forum

should be something like:

$forumURL="/forums"; // Base URL of your forum

Thanks for helping, I do have vB installed on my Root, so I tried

$forumURL="http://www.mcseboard.de"; // Base URL of your forum

and

$forumURL="/"; // Base URL of your forum

Both had the same effect as the empty one, strange...

Thomas P · #**230** 06-02-2002, 10:48 AM

It works now

Yeah, cool and now it says: no posts go back :ermm:

Update: Ok, it works now sometimes correct(?)
Think I still have to tweak here and there...

Till · #**231** 06-09-2002, 06:45 PM

Hi,
I have an odd error. Whenever I click on a link from the index page (where it lists all the forums), I get a 404. (I took out the ErrorDocument, cause it would just reroute me to the archive's index and I was wondering if it was your script or a 404. Anyway ...)

I looked at your script and saw that you are reading everything from $REQUEST_URI, instead of using $PATH_INFO. As I would have guessed.

I would parse it like that:

PHP Code:


			
list($dummy,$forumID,$year,$month,$week,$threadID)=explode('/',get_env('PATH_INFO');

$dummy being "archive", the rest is self explainatory. (Equal to your variables.)

My server runs Apache 1.3.x with PHP4.x.x (installed as an Apache Module). No clue really how to fix it, since I am a bit off with your script (reading someone else' code can be a tough on indeed.). :-/

Any help is appreciated.

Thanks!

Till

P.S.

A "bug", that I might add.

I have categories and forums. So for example, "Site related" holds "News" and "Chat", yet they all get output on the same page although "Site related" is not allowed to have any threads, let alone posts.

If you have a look at the vbulletin database, table "forum", the is a field called "cancontainthreads", which' value is 0 if it's a category and 1, if it's an actual forum. (Kind of hard to explain, but I am sure you get the drift.) So my suggestion would be to select those first and kind of group by.

Maybe not really a bug, more a feature request.

X vBulletin 3.8.12 by vBS Debug Information
Page Generation 0.11722 seconds Memory Usage 2,332KB Queries Executed 27 (?)
More Information
Template Usage: (1)SHOWTHREAD (1)ad_footer_end (1)ad_footer_start (1)ad_header_end (1)ad_header_logo (1)ad_navbar_below (1)ad_showthread_beforeqr (4)bbcode_code (3)bbcode_php (3)bbcode_quote (1)footer (1)forumjump (1)forumrules (1)gobutton (1)header (1)headinclude (1)modsystem_post (1)navbar (6)navbar_link (120)option (1)pagenav (1)pagenav_curpage (4)pagenav_pagelink (2)pagenav_pagelinkrel (11)post_thanks_box (11)post_thanks_button (1)post_thanks_javascript (1)post_thanks_navbar_search (11)post_thanks_postbit_info (10)postbit (11)postbit_onlinestatus (11)postbit_wrapper (1)spacer_close (1)spacer_open (1)tagbit_wrapper Phrase Groups Available: global inlinemod postbit posting reputationlevel showthread	Included Files: ./showthread.php ./global.php ./includes/init.php ./includes/class_core.php ./includes/config.php ./includes/functions.php ./includes/class_hook.php ./includes/modsystem_functions.php ./includes/functions_bigthree.php ./includes/class_postbit.php ./includes/class_bbcode.php ./includes/functions_reputation.php ./includes/functions_post_thanks.php Hooks Called: init_startup init_startup_session_setup_start init_startup_session_setup_complete cache_permissions fetch_postinfo_query fetch_postinfo fetch_threadinfo_query fetch_threadinfo fetch_foruminfo style_fetch cache_templates global_start parse_templates global_setup_complete showthread_start showthread_getinfo forumjump showthread_post_start showthread_query_postids showthread_query bbcode_fetch_tags bbcode_create showthread_postbit_create postbit_factory postbit_display_start post_thanks_function_post_thanks_off_start post_thanks_function_post_thanks_off_end post_thanks_function_fetch_thanks_start post_thanks_function_fetch_thanks_end post_thanks_function_thanked_already_start post_thanks_function_thanked_already_end fetch_musername postbit_imicons bbcode_parse_start bbcode_parse_complete_precache bbcode_parse_complete postbit_display_complete post_thanks_function_can_thank_this_post_start pagenav_page pagenav_complete tag_fetchbit_complete forumrules navbits navbits_complete showthread_complete
Messages:

The Arcive of Official vBulletin Modifications Site.

It is not a VB3 engine, just a parsed copy!