Spider friendly URLs

For vB 2.0

This little hackette is a quick fix to allow search engine bots to spider your threads.

Although this will allow the bots to index every thread on your site, it will not make the threads 'search engine optimized'. They will see exactly what you see when you visit your site. It simply removes the CGI bits from the URL's which prevents most search engine bots from spidering more than one level deep.

If you want a hack that allows to fully customize how the thread will look to the search engine bot, you should look at Overgrows more complete hack here.

The advantage of this hack over Overgrows is that it does not require htaccess support which can have performance issues. This could also be seen as a disadvantage though as my hack requires that you have mod_rewrite enabled on your Apache Server, whereas Overgrows method should work with just about any web host out there.

Take yer pick

[eva2000]

Originally posted by fastforward

Yay...

Thanks mate.

I'll have to fix the docs and revert my templates tomorrow then.

thanks... one thing though with gzip and mod_gzip enabled i'm finding the shorter urls not being compressed on access compared to the real urls although it's being reported as gzipped at http://leknor.com/code/gziped.php

[fastforward]

Originally posted by eva2000
one thing though with gzip and mod_gzip enabled i'm finding the shorter urls not being compressed on access compared to the real urls although it's being reported as gzipped at http://leknor.com/code/gziped.php

uhh..oh! You know, I actually did wonder if that would be a problem but I checked with leknor and as you mentioned that says it is gzipped. What are you using to determine it's not?

I seem to remember that when I first implemented this it seemed slower to load. That's what prompted me to check with leknor.com. I hope I can fix this. non-gzipped pages are not acceptable

[eva2000]

Originally posted by fastforward

uhh..oh! You know, I actually did wonder if that would be a problem but I checked with leknor and as you mentioned that says it is gzipped. What are you using to determine it's not?

I seem to remember that when I first implemented this it seemed slower to load. That's what prompted me to check with leknor.com. I hope I can fix this. non-gzipped pages are not acceptable

simple browser/eye test

i loaded the shorter url page right click page properties

then loaded the real url and right click page properties and compared the 2 sizes

shorter url was 129kb and real url was 19kb

but both reported as 19kb when tested at leknor

[eva2000]

oh yeah another thing i;m using the Zend cache trial so with html pages they won't be cached right ?

[fastforward]

Originally posted by eva2000
simple browser/eye test

i loaded the shorter url page right click page properties

then loaded the real url and right click page properties and compared the 2 sizes

shorter url was 129kb and real url was 19kb

but both reported as 19kb when tested at leknor

I don't trust the browser method.

When I do that on threads I get 81K for the short ones and not available for on the originals. And for the forumhome I get 56K with the short url and 58K with the long url.

[fastforward]

Originally posted by eva2000
oh yeah another thing i;m using the Zend cache trial so with html pages they won't be cached right ?

Not sure. I don't think it will have any affect. Zend shouldn't know about the rewrite. The request is translated by Apache before it gets to the PHP engine or the Zend stuff.

I'm just guessing though, I don't really know.

[eva2000]

Originally posted by fastforward

I don't trust the browser method.

When I do that on threads I get 81K for the short ones and not available for on the originals. And for the forumhome I get 56K with the short url and 58K with the long url.

okay tested it on my sister's pc and dialup connection and yup it's compressed the page shows 129kb but loads in 5 - 10 secs on 56k modem

[fastforward]

Just to be double sure I enabled gzip logging in my Apache logs. You'll be pleased to know that it all works and everything is compressed as usual All the exclude and include filters work aswell, based on the original urls).

[fastforward]

Someone alerted me to a Netscape problem when using this hack.

I seems that Netscape ignores the < BASE > tag.

Subsequently, any urls that do not start with a slash or a fully qualified domain will fail.

The solution is to go through all the showthread, forumhome and forumdisplay templates and prefix all relative HREF's with the $ bburl variable.

This means the hack isn't quite so 'quick and easy' after all

If someone knows why Netscape is doing this please let us know.

[eva2000]

Originally posted by fastforward
Someone alerted me to a Netscape problem when using this hack.

I seems that Netscape ignores the < BASE > tag.

Subsequently, any urls that do not start with a slash or a fully qualified domain will fail.

The solution is to go through all the showthread, forumhome and forumdisplay templates and prefix all relative HREF's with the $ bburl variable.

This means the hack isn't quite so 'quick and easy' after all

If someone knows why Netscape is doing this please let us know.

really ? hmmmmm

[eva2000]

woah i had to edit nearly all of my templates

first important one

do a search template for the forum action part

<form action="

and you'll have to change them all to

<form action="/

otherwise, posting, replying, logging in and out and any forms won't work in netscape

only images i couldn't find the link to edit to add $bburl/ to are

$post[foldericon]

and the only link i can't seem to find to add $bburl/ to is in forumdisplaybit

$thread[postedby]

where is this ?

[fastforward]

I also had a problem with the bbcode smilies not showing up. I had to edit functions.php and add a slah before the href in this line:

$bbcode=str_replace(trim($smilie[smilietext]),"<img src=\"/$smilie[smiliepath]\" border=\"0\" alt=\"\">",$bbcode);

It's on or around line 362.

Not sure about the posted by link yet.

I wish I'd never thought of this hack now

[eva2000]

i fixed the smilies and posticon problem by adding a slash / in the edit smilies in the admin panel

also another problem

merging threads won't work with your shorter url

is it possible to show/display the real url in a small table on/under the thread or on the merge thread template

i.e. display something like please input the thread to merge in the format

http://animeboards.com/showthread.php?s=&threadid=XXXX

where XXXX corresponds to

http://animeboards.com/tXXXX/s.html

[fastforward]

Originally posted by eva2000
i fixed the smilies and posticon problem by adding a slash / in the edit smilies in the admin panel

also another problem

merging threads won't work with your shorter url

is it possible to show/display the real url in a small table on/under the thread or on the merge thread template

i.e. display something like please input the thread to merge in the format

http://animeboards.com/showthread.php?s=&threadid=XXXX

where XXXX corresponds to

http://animeboards.com/tXXXX/s.html

What we need is a little link that can be displayed on every page that whne clicked, will reload whatever page you happen to on using the original url. It should be possible with a seperate php script. However, I gotta wonder if it's really worth the hassle. I might just give it up and just use Overgrow's hack.

There must be an easier way than this. I'm going to spend a few hours reading the Apache docs. I can't believe it needs to be this hard.

[eva2000]

Originally posted by fastforward

What we need is a little link that can be displayed on every page that whne clicked, will reload whatever page you happen to on using the original url. It should be possible with a seperate php script. However, I gotta wonder if it's really worth the hassle. I might just give it up and just use Overgrow's hack.

There must be an easier way than this. I'm going to spend a few hours reading the Apache docs. I can't believe it needs to be this hard.

well for what it's worth.. google.com bot paid my forums and visit and i can see it running all the way through my forums especially with overgrow's hack but yours as well