Spider friendly URLs

For vB 2.0

This little hackette is a quick fix to allow search engine bots to spider your threads.

Although this will allow the bots to index every thread on your site, it will not make the threads 'search engine optimized'. They will see exactly what you see when you visit your site. It simply removes the CGI bits from the URL's which prevents most search engine bots from spidering more than one level deep.

If you want a hack that allows to fully customize how the thread will look to the search engine bot, you should look at Overgrows more complete hack here.

The advantage of this hack over Overgrows is that it does not require htaccess support which can have performance issues. This could also be seen as a disadvantage though as my hack requires that you have mod_rewrite enabled on your Apache Server, whereas Overgrows method should work with just about any web host out there.

Take yer pick

Also if you've set "Hide Private Forums" they won't even get the forums permission page as they won't even be able to see the forum!

Think of a web crawler as a Guest visitor.

[Byron]

I have the following in my .htaccess

RewriteEngine on
RewriteRule ^/f([0-9]+)/s([^/]+?)$ /forumdisplay.php?forumid=$1&s=$2 [L]
RewriteRule ^/t([0-9]+)/s([^/]+?)\.html$ /showthread.php?threadid=$1&s=$2 [L]
RewriteRule ^/s([^/])+?/$ /index.php?s=$1 [L]

Kept getting 500 errors or 404 errors.

Are there any other commands which I need to include in the .htaccess?

Thanks

[Byron]

anyone got this hack working via .htaccess?

[steven]

well thanks to auto, I got this working. Anyways I have my forums setup like www.mydomain.com/forums so the following is my .htaccess file

RewriteEngine on 
RewriteRule ^f([0-9]+)/?$ forumdisplay.php?forumid=$1&s=$2 [L] 
RewriteRule ^t([0-9]+)\.html$ showthread.php?threadid=$1 [L]
RewriteRule ^t([0-9]+)-([0-9]+)\.html$ showthread.php?threadid=$1&perpage=15&pagenumber=$2 [L]
RewriteRule ^f([0-9]+)-([0-9]+)/$ forumdisplay.php?forumid=$1&daysprune=1000&sortorder=&sortfield=lastpost&perpage=40&pagenumber=$2 [L]
RewriteRule ^[ft]([0-9]+)/s([^/]*)/(.+)$ /$3 [L]

[Byron]

Thanks steven.

Your .htaccess works. No more 500 or 404 errors now. But I noticed that your thread url is in the format of

http://domain.com/forums/t100.html

My thread url is in the format of

http://domain.com/forums/t100/s.html

and it leads to a 404 page as i am using fastforward's hack instructions for step 2 and 3.

May i know what did you use to replace (step 2 & 3)

<a href="$bburl/f$forum[forumid]/s$session[sessionhash]">$forum[title]</a>

<a href="$bburl/t$thread[threadid]/s$session[sessionhash].html">$thread[title]</a>



Thanks a lot.
Byron

[steven]

byron I PM'd you.

[Byron]

* Byron walking up to the podium. *

Firstly I would like to thank Fastforward for writing this and thanks to steven and auto for the .htaccess.

* Speech end. *


Feedback to those using the .htaccess method.

I have got it running by removing the "s$session[sessionhash]" part in step 2 & 3, using hard coded url for all instances of $bburl and using the the following in my .htaccess.

RewriteEngine on
RewriteRule ^f([0-9]+)/?$ forumdisplay.php?forumid=$1&s=$2 [L]
RewriteRule ^t([0-9]+)\.html$ showthread.php?threadid=$1 [L]
RewriteRule ^t([0-9]+)-([0-9]+)\.html$ showthread.php?threadid=$1&perpage=15&pagenumber=$ 2 [L]
RewriteRule ^f([0-9]+)-([0-9]+)/$ forumdisplay.php?forumid=$1&daysprune=1000&sortord er=&sortfield=lastpost&perpage=40&pagenumber=$2 [L]
RewriteRule ^[ft]([0-9]+)/s([^/]*)/(.+)$ /$3 [L]

Step 2 & 3

<a href="$bburl/f$forum[forumid]/">$forum[title]</a>

<a href="$bburl/t$thread[threadid].html">$thread[title]</a>

That's all.



Wait. That's not the end. Need to confirm.

To Fastforward and the rest : Did i miss out anything by using the above .htaccess and removing the "s$session[sessionhash]" in step 2 &3?


Thanks. Cheers to vB.
Byron

[cyrus]

i dont get it

i downloaded the install txt

what the hell is httpd.conf ?

and I dont get any of the other instructions either

can someone make a dummys guide of it for me ??

thank you

[Haakon]

I`ve implemented the hack temporarily through .htaccess , and it seems to works great. Especially seeing that PR* only drops one level from inedex to any thread. Currently it drops two from index to category, but this is unnecessary, and can be fixed by just deleting this slash:
<a href="$bburl/f$forum[forumid]/">$forum[title]</a>
<a href="$bburl/f$forum[forumid]">$forum[title]</a>

One thing I would really like to see implememnted into the hack, is the code for rewriting the path to the next pages of threads listed, like
"Pages (3): [ 1 2 3 > "
would get SE friendly. The current code is quite wast and it`s dynamic so I don`t know if it`s possible, is it?

Like
http://www.radiomute.com/forumdispla...5&pagenumber=2

could be transformed into http://www.radiomute.com/f261000252 ?

Then all threads would be crawled

*PR= Page Rank, used by google as the main meassure of page importance. The higher, the better!

[dost]

I read the whole thread from the beginning and I'm confused now. I really want to install this hack, but everyone is saying something different. What is the easiest way to implement this hack? I can do it via httpd.conf. I'd really appreciate it if someone could post the final instructions.

Thanks