Spider friendly URLs

For vB 2.0

This little hackette is a quick fix to allow search engine bots to spider your threads.

Although this will allow the bots to index every thread on your site, it will not make the threads 'search engine optimized'. They will see exactly what you see when you visit your site. It simply removes the CGI bits from the URL's which prevents most search engine bots from spidering more than one level deep.

If you want a hack that allows to fully customize how the thread will look to the search engine bot, you should look at Overgrows more complete hack here.

The advantage of this hack over Overgrows is that it does not require htaccess support which can have performance issues. This could also be seen as a disadvantage though as my hack requires that you have mod_rewrite enabled on your Apache Server, whereas Overgrows method should work with just about any web host out there.

Take yer pick

[eva2000]

got it to work now with this

RewriteEngine on
RewriteRule ^/f([0-9]+)/?$ /forumdisplay.php?forumid=$1 [L]
RewriteRule ^/f([0-9]+)/s?$ /forumdisplay.php?forumid=$1 [L]
RewriteRule ^/t([0-9]+)\.html$ /showthread.php?threadid=$1 [L]
RewriteRule ^/t([0-9]+)/s([^/]?)\.html$ /showthread.php?threadid=$1&s=$2 [L]
RewriteRule ^/s([^/\?]0-9)+/$ /index.php?s=$1 [L]

[SpiceGirl]

ok, one final time please? can someone post the EXACT steps? of course, only putting some code in the .htaccess (the rewriteEngine stuff) wont do jack. we need to edit our .php files as well, right?

so which ones to edit? where? how?

thanks in advance!

[fastforward]

Originally posted by SpiceGirl
ok, one final time please? can someone post the EXACT steps? of course, only putting some code in the .htaccess (the rewriteEngine stuff) wont do jack. we need to edit our .php files as well, right?

so which ones to edit? where? how?

thanks in advance!

Here's the steps:

1. Edit httpd.conf and use Eva2000's latest rules: ie.
   

   RewriteEngine on
   RewriteRule ^/f([0-9]+)/?$ /forumdisplay.php?forumid=$1 [L]
   RewriteRule ^/f([0-9]+)/s?$ /forumdisplay.php?forumid=$1 [L]
   RewriteRule ^/t([0-9]+)\.html$ /showthread.php?threadid=$1 [L]
   RewriteRule ^/t([0-9]+)/s([^/]?)\.html$ /showthread.php?threadid=$1&s=$2 [L]
   RewriteRule ^/s([^/\?]0-9)+/$ /index.php?s=$1 [L]

2. Add the following line to your 'Head Insert' template:

   <base href="$bburl">

3. Edit the following templates:
   forumdisplay_forumbit_level1_nopost
   forumdisplay_forumbit_level1_post
   forumdisplay_forumbit_level2_nopost
   forumdisplay_forumbit_level2_post
   forumhome_forumbit_level1_nopost
   forumhome_forumbit_level1_post
   forumhome_forumbit_level2_nopost
   forumhome_forumbit_level2_post
   

   Change the links to look like this:
   <a href="$bburl/f$forum[forumid]/s$session[sessionhash]">$forum[title]</a>

4. Edit the following template:
   forumdisplaybit
   

   Change the links to look like this:
   <a href="$bburl/t$thread[threadid]/s$session[sessionhash].html">$thread[title]</a>

If you have problems with missing images in Netscape, you should go though all templates and edit the links to images to include the full path, ie' $bburl/{images_folder}/a.gif. You can use the template search function for this.

This will spider the first page of all threads only. In most cases, this should be more than enough. It is possible to add support for multipage threads, but this requires editing the code and it will probably change from version to version of vB. If you really need this, there are posts in this thread that discuss it.

I personally use the basic method. It's given me 71,800 entries in Googe for dbforums.com and 47,500 for britishexpats.com.

[tpearl5]

The only problem I'm having is with online.php displaying unknown locations. I tried the code posted here, but got errors. I'm running 2.2.5. Anyone ?]

Take a look: http://degster.com/midi/board/

[nuno]

Paul, how do we fix the Unknown Location Fields showing up in WOL page?
TIA

[fastforward]

Originally posted by nuno
Paul, how do we fix the Unknown Location Fields showing up in WOL page?
TIA

easypeasy
In online.php : Change the 'default' block of the CASE statement on or around line 626 to look like this:

  default:
    if (preg_match ("/t([0-9]+)\.html/", $filename, $match)) {
       $userinfo[threadid] = $match[1];
       $userinfo[activity] = 'showthread';
       $threadids .= ",$userinfo[threadid]";
    } elseif (preg_match ("/f([0-9]+)/", $filename, $match)) {
       $forumid = $match[1];
       $forumids .= ",$forumid";
       $userinfo[activity] = 'forumdisplay';
       $userinfo[forumid] = $forumid;
    } elseif (preg_match ("/robots\.txt/", $filename)) {
       $userinfo[activity] = 'spider';
    } else {
      $userinfo[activity] = 'unknown';
    }
  }

[nuno]

Tada, working like a charm now :bunny:
Muchas gracias Paul

[Velocd]

I've attempted an installation of this hack, but am facing an annoying obstacle.

Where in the world is this httpd.conf file located? I have found .htaccess in my forums directory, which is a completely blank file, but not httpd.conf. Which am I suppose to modify?

[Velocd]

*gasp*

on..verge..of..death..need.......help

[fastforward]

Originally posted by Velocd
I've attempted an installation of this hack, but am facing an annoying obstacle.

Where in the world is this httpd.conf file located? I have found .htaccess in my forums directory, which is a completely blank file, but not httpd.conf. Which am I suppose to modify?

The location of httpd.conf is dependent on your installation of Apache. Probably the most common place for it is /etc/

If you are unfamiliar with editing this file... be careful; your web server depends on it.

[filburt1]

FYI, on OS X, it's in /etc/httpd/

[filburt1]

Okay, my .htaccess file must be at fault. My site is at /home/filburt1/public_html/webdesignforums/ ; it's an addon domain in CPanel. So http://www.webdesignforums.net/ really loads http://webdesignforums.turtletips.com/ , same as http://www.turtletips.com/webdesignforums/ .

I get 404 errors for every single link. My .htaccess is identical to eva's.

[Velocd]

Originally posted by fastforward

The location of httpd.conf is dependent on your installation of Apache. Probably the most common place for it is /etc/

If you are unfamiliar with editing this file... be careful; your web server depends on it.

Well crap, I can't seem to find it anywhere :sleep:
Not that I'm bitter about it

[Velocd]

*one day later*

Ok so maybe I'm "alittle" bitter about it...

I searched for the file httpd.conf and couldn't find it anywhere.

It looks like according to this thread there is another way to install this hack using .htaccess? Maybe..?

[Velocd]

*one day later*

So, I walked into this bar, and this guy asked me "Watcha looking for", and I said "httpd.conf, you know where it is?" And then he said...

oh wait..uh..that was random *confused*

Anyway, could somebody please help this poor guy out in his search for the holy httpd.conf..or other methods to installing this hack (see above thread)