Spider friendly URLs

For vB 2.0

This little hackette is a quick fix to allow search engine bots to spider your threads.

Although this will allow the bots to index every thread on your site, it will not make the threads 'search engine optimized'. They will see exactly what you see when you visit your site. It simply removes the CGI bits from the URL's which prevents most search engine bots from spidering more than one level deep.

If you want a hack that allows to fully customize how the thread will look to the search engine bot, you should look at Overgrows more complete hack here.

The advantage of this hack over Overgrows is that it does not require htaccess support which can have performance issues. This could also be seen as a disadvantage though as my hack requires that you have mod_rewrite enabled on your Apache Server, whereas Overgrows method should work with just about any web host out there.

Take yer pick

[fastforward]

Gimme till tomorrow night and I'll redo the hack based on the 2.02 code and post full instructions. I'll use the 'double rewrite' method that porfiry contributed to avoid changing a bazillion templates

ThomasP: The /f12 bits are created by editing the templates. All the rewrite does is change those short urls back to something that the vB php code knows about.

oops, my bad - I see.
Thanks for your help and take your time,
-Tom

[auto]

I finally got it to work by taking out the extra "/" from the first 2 lines, wooohooo!!!
Now I face smaller, yet another problem.

Take this page for example:
http://www.automotiveforums.com/vbulletin/t4667.html
The 2nd page of the thread will not be indexed by search engines

How can I use mod_rewrite to make the 2nd page be
http://www.automotiveforums.com/vbulletin/t4667-2.html
3rd page: http://www.automotiveforums.com/vbulletin/t4667-3.html... etc?

Thanks for any help.

[auto]

Well, I've done it!

http://www.automotiveforums.com/vbulletin/t4667.html

And click on page 2, 3, and 4

Thread [ << < 1 2 3 4 > >> ]

If there's anybody that wants the rewrite code for this I'll post it, it was easier than I thought, after twisting my head for 2 hours.

[fastforward]

Originally posted by auto
Well, I've done it!

http://www.automotiveforums.com/vbulletin/t4667.html

And click on page 2, 3, and 4

Thread [ << < 1 2 3 4 > >> ]

If there's anybody that wants the rewrite code for this I'll post it, it was easier than I thought, after twisting my head for 2 hours.

You didn't tell us you were doing it! I spent hours last night doing this Actually, 2.0.2 made it a little easier due to the change in the pagenav handling.

Mine appears to be working on IE, but I haven't tried every possible scenario and I haven't tested it on Netscape. It handles multipage threads, but I still need to add support for multi-page threadlistings. Once complete, you should be able to block spiders from all dynamic urls and still have them index ALL posts.

If anyone wants to help debug it, here it is:

----------------------------------------------

1) Edit /etc/httpd.conf
httpd.conf needs editing to enable and configure mod_rewrite. Here's all the relevent bits from my httpd.conf. It can go anywhere between the <VirtualHost> & </VirtualHost> tags.
RewriteEngine on
RewriteRule ^/f([0-9]+)/s([^/]+?)$ /forumdisplay.php?forumid=$1&s=$2 [L]
RewriteRule ^/t([0-9]+)/s([^/]+?)\.html$ /showthread.php?threadid=$1&s=$2 [L]
RewriteRule ^/s([^/])+?/$ /index.php?s=$1 [L]




2) Edit Templates
a. The following templates contain the forum links that need changing (one occurance in each)

-> forumdisplay_forumbit_level1_nopost
-> forumdisplay_forumbit_level1_post
-> forumdisplay_forumbit_level2_nopost
-> forumdisplay_forumbit_level2_post
-> forumhome_forumbit_level1_nopost
-> forumhome_forumbit_level1_post
-> forumhome_forumbit_level2_nopost
-> forumhome_forumbit_level2_post

The new link should be: <a href="$bburl/f$forum[forumid]/">$forum[title]</a>


b. The following template contains the thread link that needs changing:

-> forumdisplaybit

The new link should be: <a href="$bburl/t$thread[threadid].html">$thread[title]</a>


c. The following templates contain the pagenav bits:

-> forumdisplay_multipagenav_pagenumber

Replace the link within this templatewith:
$bburl/t$thread[threadid]-$acurpage.html

--------------------------------------------------------------

[auto]

This is what I added for the thread#-page#.html to work:

RewriteRule ^t([0-9]+)-([0-9]+)\.html$ showthread.php?threadid=$1&perpage=10&pagenumber=$ 2 [L]

Then I had to modify functions.php

You can find what to modify by finding the veriables that correspond to the page nav in the templates.

[Thomas P]

Has anyone managed to get that working via .htaccess?
If mod_rewrite is enabled it works as well via .htaccess - right?

At least that works on my account:

RewriteEngine on
RewriteBase /test/
RewriteRule   ^(.+)  http://www.mcseboard.de/$1  [R,L]

thx,
-Tom

[steven]

Originally posted by Thomas P
Has anyone managed to get that working via .htaccess?
If mod_rewrite is enabled it works as well via .htaccess - right?

At least that works on my account:

RewriteEngine on
RewriteBase /test/
RewriteRule   ^(.+)  http://www.mcseboard.de/$1  [R,L]

thx,
-Tom

I would also like to know if anyone has managed to do this via .htaccess?

[TheComputerGuy]

where can I find the htaccess that you speak of, I mean I looked all over my account and could not find it...

[Thomas P]

Just create one...