Spider friendly URLs

For vB 2.0

This little hackette is a quick fix to allow search engine bots to spider your threads.

Although this will allow the bots to index every thread on your site, it will not make the threads 'search engine optimized'. They will see exactly what you see when you visit your site. It simply removes the CGI bits from the URL's which prevents most search engine bots from spidering more than one level deep.

If you want a hack that allows to fully customize how the thread will look to the search engine bot, you should look at Overgrows more complete hack here.

The advantage of this hack over Overgrows is that it does not require htaccess support which can have performance issues. This could also be seen as a disadvantage though as my hack requires that you have mod_rewrite enabled on your Apache Server, whereas Overgrows method should work with just about any web host out there.

Take yer pick

[Logikos]

Thanks, can this hack be used with just moding the .htaccess file only?

[NTLDR]

I'm pretty sure it can, but I've never used it.

[Logikos]

Anyone know how we would be able to use this with just the .htaccess file? I would love to have this hack on my boards, been seeing some great responds.

[iceman11111]

Help!!!

I have installed all of the additions to this great hack, but I have one major problem!

When I goto Update my counters, I get a an error, trying to "Update Thread Info"

I get this

Database error in vBulletin Control Panel 2.2.9:

Invalid SQL: UPDATE thread SET lastpost=1055471230,replycount=9,postusername='tla ne', postuserid='2575', lastposter='GYMBRAT',attach= WHERE threadid=1999
mysql error: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near 'WHERE threadid=1999' at line 1

mysql error number: 1064

Date: Wednesday 04th of February 2004 06:26:13 PM
Script: http://www.intense-training.com/foru...dmin/misc.php3
Referer: http://www.intense-training.com/foru...ction=chooser&

[eva2000]

Originally Posted by buro9

OK, this is for eva2000 and then I'll go through what I'm doing to keep all my links lovely

In your forum directory, set up your .htaccess to contain the following:

RewriteRule ^f([0-9]+)\/s$ forumdisplay.php?f=$1 [R=301]
RewriteRule ^t([0-9]+)\/s\.html$ forumdisplay.php?f=$1 [R=301]

Let me explain each part of those rules... well... one of them:

RewriteRule ARG1 ARG2 ARG3= If the URL requested matches the first arg, perform the second arg accord to the rules of the final arg.

ARG1 = Regular expression to match. In our case, ^f([0-9+])\/s$ where ^ = start of line, ([0-9]+ ) = 1 or more numbers, \/ = escaped forward slash (has to be escaped within a regexp), $ = end of line.

ARG2 = URL to request instead. In our case we also include $1 which is the first matching part of the regexp in ARG1 which is the number... the forumId.

ARG3 = Rule to dictate how to handle the ARG2 URL... [L] would bring it inline (leave the browser URL as it is and force the URL to the output stream without redirect)... I'm using [R=301] which is to send back a http 301 error which is "Moved Permanently"... I do this as I want Google and search engines to update all of their links over time and quite hitting the legacy ones.

So that's your solution eva Though do bug me if I screwed that up... I've only just finally got round to it... and the 301 error may require a fully qualified URL and not a relative one.



Anyhow... this is now to help the people who are NOT upgrading their VB2 but, are doing a fresh install of VB3 at a new URL and are archiving their VB2. I'm not sure how many of these there will be... but our forum quite enjoys the zest of occassional re-births... and it will help to ensure that our VB3 threads are truly threaded from day 1.

So... I've got this URL at the moment:

http://www.bowlie.com/forum/

And I'm moving that forum to:

http://www.bowlie.com/2003/

And putting VB3 on the /forum/ URL.

Yes I have lots of VB licenses

Anyhow... I want all t*.html links to go through to the archived forum... to keep all of the search engine results valid.

And I want all of the f*.html links to go through to the new forums on the new board.

The threads are easy to map:

	#
	# Legacy thread redirects
	#
RewriteRule ^t([0-9]+)\.html$ http://www.bowlie.com/forum/showthread.php?threadid=$1 [R=301]
RewriteRule ^t([0-9]+)-([0-9]+)-([0-9]+)\.html$ http://www.bowlie.com/forum/showthread.php?threadid=$1&perpage=$2&pagenumber=$3 [R=301]
RewriteRule ^t([0-9]+)-([0-9]+)--(.*)--([0-9]+)\.html$ http://www.bowlie.com/forum/showthread.php?threadid=$1&perpage=$2&highlight=$3&pagenumber=$4 [R=301]

But the forums are harder.

The forumid's on both the old and new sites are different... so what was forumid = 4 on VB2 is now forumid = 11 on VB3. So I need a map to track these and then set up all of the RewriteRules.

I've created a javascript array of the forumids... where the first value is the VB2 forumid, and the second value is the VB3 forumid.

Some forums won't exist on my VB3 site... so those links will go through to the archive too.

I've then created a simple bit of JavaScript which when run from Internet Explorer (doesn't work in Mozilla as that is standards compliant ) will produce my rewriterules for me:

<script type="text/javascript">
var forums = new Array(
  new Array('16','21'),
  new Array('17','22'),
  new Array('29',''),
  new Array('33','29'),
  new Array('4','11'),
  new Array('30',''),
  new Array('27','')
);

var html = "";

var oldForumUrl = "http://www.bowlie.com/2001/";
var newForumUrl = "http://www.bowlie.com/forum/";

for (var ii = 0; ii < forums.length; ii++) {
  if (forums[ii][1] == '') {
    // Link to archive
		html += "RewriteRule ^f" + forums[ii][0] + "\\.html$ " + oldForumUrl + "forumdisplay.php?f=" + forums[ii][0] + " [R=301]\n";
    html += "RewriteRule ^f" + forums[ii][0] + "--.*\\.html$ " + oldForumUrl + "forumdisplay.php?f=" + forums[ii][0] + " [R=301]\n";
  } else {
    // Link to new
		html += "RewriteRule ^f" + forums[ii][0] + "\\.html$ " + newForumUrl + "forumdisplay.php?f=" + forums[ii][1] + " [R=301]\n";
    html += "RewriteRule ^f" + forums[ii][0] + "--.*\\.html$ " + newForumUrl + "forumdisplay.php?f=" + forums[ii][1] + " [R=301]\n";
  }
}

document.write("<plaintext>" + html);

</script>

Obviously modify the URLs as applicable to you.

The output can then be pasted into your .htaccess and you will have:

Legacy forum URLs for forums that exist on your new VB going to VB3
Legacy forum URLs for forums that don't exist on your new VB going to VB2
Legacy thread URLs going to your VB2

Google would start spidering your new VB3 links pretty soon, including the new thread ones.

Now... if all of that went straight over your head... just ignore it and don't do any of it It's there to help other users that like me have installed this hack widely, indexed heavily by Google and are choosing to have a fresh VB3 and archive their VB2... that may be no-one else here... in which case... ignore this

thanks buro9, sorry for the late reply my forum's server was down due to hardware failure since Jan 31

[gmarik]

This one would be good for vB3 ...

[Haakon]

I agree, I really miss this hack. 300 extra guests are allways welcome you know , ANYONE successfully implemented this hack on vb3?

[DenzoForums]

This looks cool.. I would like to install tis for VB 3.0.1. when or if it's available.

Dave

[lierduh]

Please understand, there is no need for VB3 to have this hack.

The Archive system is designed to feed the robots. Besides the main forums are indexed by Googles etc due to the removal of session ID from the URL. The script detects spiders by its user agent and disables the session ID if found.

If anything, one might want to shorten the Archive's URL to potentially increase the ranking. eg.

/archive/index.php/f-35.html

to

/arc/f-35.html

This can be easily achieved by using rewrites.

[DenzoForums]

My archives are not working.. not vbulletin problem, but apparently the sever i am hosted on doesn;t support some feature for that to work. Is there anything else i can do?