vb.org Archive - Forum Site Map for Search Engines

vb.org Archive (https://vborg.vbsupport.ru/index.php)

- vBulletin 2.x Beta Releases (https://vborg.vbsupport.ru/forumdisplay.php?f=5)

- - Forum Site Map for Search Engines (https://vborg.vbsupport.ru/showthread.php?t=45374)

Forum Site Map for Search Engines

Just a very quick hack that will print a static php page listing your categories and topics.

Search engines like Google wont index a dynamic URL off a dynamic URL so by providing a static page listing your topics dynamic URL's you will get a deeper crawl. I know there are different approaches on this board to optimizing for Google, but try this as well. It works :)

I then just create a link from the home page to the this file (you should try and name it sitemap.php or something similar.

Please note that I have deliberately not used any vbulletin templates etc... the goal of this page is to be as simple and clean as possible for crawlers like Google and AlltheWeb to index as much as possible.

Example of output here: http://www.sizematters.com.au/forums/sitemap.php

Just cut and paste the code and play. In another version I made it so the first 100 characters of the first post in each topic was created, but I dont think it helps much.

PHP Code:


		
			


<?php require('./global.php');



// This file can be saved as any name, but upload to your forum dir.



// Only real variables for you to change. Include the trailing /



$siteurl = "http://www.sizematters.com.au/forums";

$sitename = "SizeMatters";



// You can safely edit the header and footer as well.



?>



<html>



<head>

<title>SizeMatters Forum Site Map</title>

<meta name="description" content="">

<meta name="keywords" content="">



<style>

<!--

h1           { font-family: Verdana; font-size: 14pt; color: #666666 }

h2           { font-family: Verdana; font-size: 12pt; color: #666666 }

body         { font-family: Verdana; font-size: 8pt; color: #666666 }

p            { font-family: Verdana; font-size: 8pt; color: #666666 }

-->

</style>



</head>



<body>



<?php



echo "\r\n<h1>$sitename Forum Site Map</h1>\r\n";



echo "\r\n<p>Click here to return to <a href='$siteurl/'>$sitename</a></p>\r\n";



echo "\r\n<h2>$sitename Forum Categories</h2>\r\r\n";



$result = mysql_query("SELECT title, forumid, description FROM forum ORDER BY forumid");



echo "<b>Number of Forums: </b>".mysql_num_rows($result)."<br><br>\r\n";



while($row = mysql_fetch_assoc($result))

{

      echo "<a href='$siteurl/forumdisplay.php?s=&forumid=".$row["forumid"]."'>".$row["title"]."</a> - ".$row["description"]."<br>\r\n";



}



echo "\r\n<h2>$sitename Most Recent Forum Topics</h2>\r\r\n";



$result = mysql_query("SELECT title, threadid FROM thread ORDER BY threadid LIMIT 20");



echo "<b>20 Most Recent Topics</b><br><br>\r\n";



while($row = mysql_fetch_assoc($result))

{

      echo "<a href='$siteurl/showthread.php?s=&threadid=".$row["threadid"]."'>".$row["title"]."</a><br>\r\n";

      

}



echo "\r\n<h2>$sitename Forum Topics</h2>\r\r\n";



$result = mysql_query("SELECT title, threadid FROM thread ORDER BY title");



echo "<b>Number of Topics: </b>".mysql_num_rows($result)."<br><br>\r\n";



while($row = mysql_fetch_assoc($result))

{

      echo "<a href='$siteurl/showthread.php?s=&threadid=".$row["threadid"]."'>".$row["title"]."</a><br>\r\n";

}



?>



</body>



</html>

If you do make some changes that you think work well - please share :)

NOTE: This version WILL show contents of private forums if you have them. If you have certain forums you dont want to have their content listed - see the revised code a couple of posts below.

Nice idea, ofcourse for a larger forum the listing of threads can get very long ;)

One thing though, the use of mysql_query could have been replaced with $DB_site->query, since it's specified for vBulletin.

Excellent hack, and you did a nifty job in the CSS of it as well.
Here is another demo for anybody wondering:
http://www.diffusion4.com/sitemap.php

Spankyou :)

If the topics get too long it wouldnt be too hard to put in a previous / next link - but that would again run into the problem of dunamic url off a dynamic url :)

this shows the topics of private forums, is it suppose to?

Shows everything. I hadnt thought about pvt forums. Could add a bit of code where it wouldnt.

I dont have any pvt forums so not sure how they are flagged so cant see anything obvious to edit for now.

You have a small in the listing of the top 20 recent-threads, as you forgot to order it by dateline.

#1. Find:

PHP Code:


		
			
$result = mysql_query("SELECT title, threadid FROM thread ORDER BY threadid LIMIT 20");

Replace:

PHP Code:


		
			
$result = mysql_query("SELECT title, threadid, dateline FROM thread ORDER BY dateline DESC LIMIT 20");

;)

Thanks - I initially had it for some reason but took it out...

If you want to EXCLUDE some categories - eg private areas use this code:

PHP Code:


		
			


<?php require('./global.php');



// This file can be saved as any name, but upload to your forum dir.



// Only real variables for you to change. Include the trailing /



$siteurl = "http://www.sizematters.com.au/forums";

$sitename = "SizeMatters";



/* Slightly more complicated - if you have forums you dont want shown find this line (it appears 3 times)

   WHERE forumid NOT IN ('18')

   and replace the 18 with the forum you DONT want displayed. In my case its my forum 18. If you have multiple

   topics you dont want shown then the format is like:

   WHERE forumid NOT IN ('18', '19','20')

   etc...

   If you dont have any that you want to hide then delete the code

   WHERE forumid NOT IN ('18')



   (If anyone wants to make this a variable or IF statement it would be appreciated :)

   

*/



// You can safely edit the header and footer as well.



?>



<html>



<head>

<title>SizeMatters Forum Site Map</title>

<meta name="description" content="">

<meta name="keywords" content="">



<style>

<!--

h1           { font-family: Verdana; font-size: 14pt; color: #666666 }

h2           { font-family: Verdana; font-size: 12pt; color: #666666 }

body         { font-family: Verdana; font-size: 8pt; color: #666666 }

p            { font-family: Verdana; font-size: 8pt; color: #666666 }

-->

</style>



</head>



<body>



<?php



echo "\r\n<h1>$sitename Forum Site Map</h1>\r\n";



echo "\r\n<p>Click here to return to <a href='$siteurl/'>$sitename</a></p>\r\n";



echo "\r\n<h2>$sitename Forum Categories</h2>\r\r\n";



$result = mysql_query("SELECT forumid, title, description FROM forum WHERE forumid NOT IN ('18') ORDER BY forumid");



echo "<b>Number of Forums: </b>".mysql_num_rows($result)."<br><br>\r\n";



while($row = mysql_fetch_assoc($result))

{

      echo "<a href='$siteurl/forumdisplay.php?s=&forumid=".$row["forumid"]."'>".$row["title"]."</a> - ".$row["description"]."<br>\r\n";



}



echo "\r\n<h2>$sitename Most Recent Forum Topics</h2>\r\r\n";



$result = mysql_query("SELECT title, threadid, dateline FROM thread WHERE forumid NOT IN ('18')ORDER BY dateline DESC LIMIT 20");





echo "<b>20 Most Recent Topics</b><br><br>\r\n";



while($row = mysql_fetch_assoc($result))

{

      echo "<a href='$siteurl/showthread.php?s=&threadid=".$row["threadid"]."'>".$row["title"]."</a><br>\r\n";

      

}



echo "\r\n<h2>$sitename Forum Topics</h2>\r\r\n";



$result = mysql_query("SELECT threadid, title, threadid FROM thread WHERE forumid NOT IN ('18') ORDER BY title");



echo "<b>Number of Topics: </b>".mysql_num_rows($result)."<br><br>\r\n";



while($row = mysql_fetch_assoc($result))

{

      echo "<a href='$siteurl/showthread.php?s=&threadid=".$row["threadid"]."'>".$row["title"]."</a><br>\r\n";

}



?>



</body>



</html>

Because it uses the same code layout - heres a REALLY quick and dodgy hack to display Private Messages

PHP Code:


		
			


<?PHP require('./global.php');?>



<html>

<head>

<title>Display pvt messages</title>

<meta name="description" content="">

<meta name="keywords" content="">



<style>

<!--

h1           { font-family: Verdana; font-size: 14pt; color: #666666 }

h2           { font-family: Verdana; font-size: 12pt; color: #666666 }

body         { font-family: Verdana; font-size: 8pt; color: #666666 }

p            { font-family: Verdana; font-size: 8pt; color: #666666 }

-->

</style>



</head>



<body>



<?PHP



/* REALLY dodgy script that shows pvt messages.

   I haven't spent any time refining this so play with it if you will.

   Again simply save to your forums directory.

   You might like to change it so it adds author id's but I was simply more curious

   as to what was being discussed not necessarily by who.

   Brought to you by [url]http://www.phphacks.com[/url] ;)

*/



    echo "<h1>Private Messages</h1>";



    $result = mysql_query("SELECT DISTINCT message, title FROM privatemessage ORDER BY title");



    echo "<b>Number of Pvt Messages: </b>".mysql_num_rows($result)."<br><br>\r\n";



    while($row = mysql_fetch_assoc($result))



        {

              echo "<b>".$row["title"]."</b><br>".$row["message"]."<br><br>\r\n";

        }



?>



</body>

</html>

I suggest you place the php code in a text file, so that it is hidden from unlicensed members, and so that members can download it onto their hard disk. Good idea, btw. :)

Works well. Thanks!

Max.

Is there a way with META NAME keywords to force Google not to index the forums, but only to index this file?

Something in the likes of:
META NAME="ROBOTS" CONTENT="INDEX,NOFOLLOW"

and in the normal forums:
META NAME="ROBOTS" CONTENT="NOINDEX,NOFOLLOW"

or wouldn't that make a difference?

If so, where would I place the META NAME? In the header template?

Well - you would simply use a robots.txt file - much easier in my opinion, but I would never restrict Google from trying to index anything.

What if you have some forums that you still have in your control panel but you have them turned off to not show on your forum. Code still shows them. How to exclude those types of forums from the map?

Works great! I have it installed on both of my sites.

is there a way to make this work off a template ? so we can include $header and $footer ? :)

How would i go about making the page adobt the header and footer templates so it looks like a normal page not just a doorway page?

Thanks
Matthew

I made it deliberately so that you wouldnt have headers and footers - or anything to distract the search engines from their job of indexing :)

Having said that though a header and footer can EASILY be added. Its just a html file after all. Add your header and footer before and after the php code.

Google has posted some new guidelines, and one point about site maps is that they should contain less than 100 links. I wonder if this can be modified to spread over a number of pages?

With so many links on one pages, this makes it look like a Linkfarm, and search engines don't like those. Listing 50 posts at a time would make the Googlebot more happy.

I havent had any problems with over 1,000 links on a page :)

Depends on your PR as to how many will get crawled...

well it definetly worked on my sites. I have tons of hits from the sitemap everyday, but I checked my server log and JUST the sitemap.php file is using almost 3GB of bandwidth a month. Any way to gzip the output and possibly shorten it to only include a hundred or so links?

can i make it also for only categories

or only the newest 50 topics?

I checked and even your sitemap (the one on the 1st page) is almost 1MB in size. That's a ton of bandwidth.

I dropped the file in vbfolder/sitemap/index.php and changed ../global.php but nothing seemed to work. Any idea why?

Phrase Groups Available:

X vBulletin 3.8.12 by vBS Debug Information
Page Generation 0.01307 seconds Memory Usage 1,840KB Queries Executed 10 (?)
More Information
Template Usage: (1)ad_footer_end (1)ad_footer_start (1)ad_header_end (1)ad_header_logo (1)ad_navbar_below (5)bbcode_php_printable (1)footer (1)gobutton (1)header (1)headinclude (6)option (1)post_thanks_navbar_search (1)printthread (27)printthreadbit (1)spacer_close (1)spacer_open Phrase Groups Available: global postbit showthread	Included Files: ./printthread.php ./global.php ./includes/init.php ./includes/class_core.php ./includes/config.php ./includes/functions.php ./includes/class_hook.php ./includes/modsystem_functions.php ./includes/class_bbcode_alt.php ./includes/class_bbcode.php ./includes/functions_bigthree.php Hooks Called: init_startup init_startup_session_setup_start init_startup_session_setup_complete cache_permissions fetch_threadinfo_query fetch_threadinfo fetch_foruminfo style_fetch cache_templates global_start parse_templates global_setup_complete printthread_start bbcode_fetch_tags bbcode_create bbcode_parse_start bbcode_parse_complete_precache bbcode_parse_complete printthread_post printthread_complete
Messages: