vBulletin Google Site Map
 

There are a few vBulletin Google Site maps around, but none as comprehensive as this that I have seen. This vBulletin Google site map is based on Kalius's work on discussed on his site and at Digitalpoint

FYI, the vBulletin Google Site Map modification contains at least one potential remote file include exploit. You are hereby notified that you should uninstall this modification, including the removal of the vBulletin Google Site Map PHP files from your server. The author of the modification has been notified, and the vBulletin Google Site Map thread has been moved to the Modification Graveyard forum.

Best,
vBorg Staff

vBulletin Google site map
with ability to add extra urls
(c) by T2DMan - Michael Brandon

8Nov05+13hrsGMT - update. Added support for Yahoo Sitemap & bug fix
7Nov05+13hrsGMT - update. Thanks for your help.

I have attached 1 version with 2 different preset configs for your convenience. vBSEO and non vBSEO configs.

Configure in the vbgsitemap-config.php file.
$base=""; // "" if in root, $base="forum/"; //if in the subdirectory forum

It now works well with forum permissions, and the vBulletin's scheduled task system. I have personally installed the reviewed vBGSiteMap on a site with over 200,000 urls. Copes with the large number of urls with no problems.

The vBGSiteMap works well with vbSEO, and it lists all the old url's to allow Google to spider the old url's and so find the 301 redirects to the new, and so delete the old url's from its index.

Compared to the previous versions

• Sitemap files now written into their own directory - means this can be safely chmod 777 - thanks Sarah King
• config and class files separated out
• new help file written
• many corrections made to the code - amazing what sitting on a piece of code can do, and amazing what you can learn from personally installing on many different servers.
• Can now work on mysql versions from 3.23.58 to 5+
  

Specifications

• works with v3.03 - 3.5
• access to this file is password protected

the same password as the database via the vBulletin includes/config.php file

• creates an index file and multiple gzipped site maps to less than the maximum permitted by Google (I have in fact restricted it to 3k url's, rather than 50k as my browser takes a while to process the file when checking it)
• creates a gzipped file containing the urls for Yahoo Sitemap

• the site map files are created in the /vbgsitemap/files directory which needs to be chmod to 777

adds

• forums - with paginations of forums (excluding redirects and password protected forums)
• threads - with pagination of threads (excluding invisible threads=soft deleted, and threads inside password protected forums)
• archive forum pages with paginations
• archive versions of threads with paginations
• members

extra

• extra urls from other program types
• add a new url per line onto file.txt, and a gzipped file will be created for it, and added to the site index file.

How to install

• unzip the files
• copy the vbgsitemap folder to your root directory
• chmod the vbgsitemap/files directory to 777
• copy vbgsitemap.php to your root directory
• copy the includes/cron/vbgsitemap-cron.php file to your vbulletin includes/cron/ directory
• look over the configuration options on the /vbgsitemap/vbgsitemap-config.php file
  • $base="" or "forum/" - if your forum is the actual root, or in the forum directory etc.
  • $addthreadarchive=false; - whether you want the archive version of the threads to be added or not.
  • the rest should not need to be changed
  • change to $xtrafile="file.txt"; if you have additional urls that you want included from another package, get a list of those url's and add them to file.txt.
• Then access the /vbgsitemap/index.php file in your browser to get things happening.
• The username/password is the username/password of your database.
• Add the resulting one index file to Google sitemaps: http://www.yourdomain.com/vbgsitemap.php
• Add the other file to Yahoo sitemaps
  http://www.yourdomain.com/vbgsitemap/files/urllist.gz
• A cron job can be setup in the vbulletin scheduled tasks
  
  
  • ./includes/cron/vbgsitemap-cron.php
  • depending on the number of posts/new threads to your forum, set to run daily or weekly etc. The cron will automatically ping Google and return success/failure into the cron log.

I went and downloaded the file from the URL you posted. I edited the file for my settings, but it would run correctly until I manually defined the DB info. This of course hosed a couple other things, but it generated the file.

I'm sure I missed something in the edits, mayhap you could note them better in the file for those of us who didn't sleep so well lastnight?

Anyway, it did generate the file, so thanks for that.

I'll post back when I have some results.

PS - Using 3.0.7

argh too much stuff to read. do I just activate the plugin

Looks interesting.... may check it out later.

Shouldn't you attach the hack in this thread?

its still in beta stage

I am sorry but if you release a hack here, it must also be downloadable from here. Please upload the needed files.

Sorry - I initially added it. Not sure what happened. Anyway, back again.
Having a look now at the issues mentioned. Its still beta, but there should not be any issues... But then again, thats why its beta, I need to take your advice on board.

Per the check boxes this is not a plugin. It is an external program that you define parameters on the files then run via your browser.

I also noticed that it misformed the URL, most likely because I had problems with it, but it ended up with the site main URL twice, then the rest of the address.

Please check your setup - the system picks up your domain. All you need to specify is the directory with a trailing forward slash. Or blank if your forum is in the root directory.

this looks really cool, how well would it work with vbadvanced cmps 2.0 custom vba pages?

how exactly should i use this hack ?

How do I use

I have now added instructions on how to setup - please see this thread.

vBAdvanced

With custom vbadvanced pages, you can manually add pages to the file specified and the program will add them to the Google site map. Or you can amend the file to drag the vbadvanced urls from the database and add them to the site map as well.

If you do manage to hack the file for vBAdvanced, please post the code here and a sample of the xml file, and I will add the code to the program.

I downloaded the zip file, and the readme is blank.

This thread includes the installation instructions
https://vborg.vbsupport.ru/showthread.php?t=91389

In addition, you should be able to read through the start of the vbsitemap-multi.php file to see the config options.

The blank file.txt is to add additional url's manually.

im having problems with this

it created the files no problem but when i click on one of the sitemap pages (i have 106000 generated)

i get this error

The XML page cannot be displayed
Cannot view XML input using style sheet. Please correct the error and then click the Refresh button, or try again later.

--------------------------------------------------------------------------------

An invalid character was found in text content. Error processing resource 'http://www.world-of-digital.com/sitemap6.xml.gz'...

For some strange reason, when I look at it there is no problem.

I get the correct response "
This XML file does not appear to have any style information associated with it."

Your error says that it can't use the style sheet - so disable the style sheet and you should be right???

where do i disble the style sheet ?

is it the css stored as files i need to disable ?

if so im sure it would be better to work around this because the style sheet increases speed and bandwidth on my site by 40% with tursk mmchache installed!

but if your not talking about that option then pay no attention to my post m8

would love this working i jsut stumbled across the sitemap feature in google and done a few searches and nearly every result i have read has been positive !!!!

cheers for releasing this aswell

update:

i just disabled the stylesheet css stored as files option and still the same error (yes i done a ctrl F5 refresh aswell)

Sorry, you have got me slightly wrong there. You definately need CSS stored as files on your vBulletin site. So change that back fast!

The xml file you have, has nothing wrong with it - its the way that you are reading it that is the issue.

The issue is to do with your browser and how it views specifically the xml files (nothing to do with vBulletin per say). When I personally look at the file you mention, I am able to view it with no problems. I get the notation at the top
However, when you look at it, you get
How are you looking at the xml file??? - I personally use firefox, but even with explorer, I see no error. I personally am not using an xml reader, nor am I attempting to display the xml in any other format other than what it is.

Can you post some screenshots? I don't have a 3.5 Board running because it's still beta and I want to know "what's on the market" for it.

I installed it but the last modified date for forum dispay shows 1969

Out of curosity, I have a 'dead forums' catagory filled with temp forums for certain events and misc other things. Would it be a good idea to have the sitemap generate a file for this section or not?

Basically, can we just upload the 3 files to our root dir (/, since that is where my forum is located) and leave it? Do I need to run any plugin installers?

No installers to run although you may want to run this as a cron to autoupdate once in a while. You'll also need to chmod root or create some files that have write permissions on it.

I don't quite understand how the Google Site Map works. I know what it DOES, but not how it functions. For instance, how does uploading your files do anything at all? Nothing seems to get executed, etc. I suppose that's why you suggest I cron the files? Which file? How often? I chmodded root to 0777 since I'm not sure what else to chmod.

I can not get this thing to work, need detailed instructions please! What goes into the file.txt? And what exactly has to be modified in order to get this to work?

And did I understand this right, you want us to chmod our root dir to 777?

Okay, got it to create the files, if I try to view the xml file I get:

XML Parsing Error: no element found
Location: http://www.mr2board.com/sitemap_vb_index.xml
Line Number 14, Column 1:
^

Please?

OKay, as far as the error im getting I think it's because the xml file is missing the </sitemapindex> on line 14. Is this a glitch in your script? I already modified the script;

fwrite( $multihandle, "<sitemap>
<loc>{$home}{$xtrafilegz}</loc>
<lastmod>{$a}</lastmod>
</sitemap>
");

}
//end xtra file

fwrite( $multihandle, "<sitemap>
<loc>{$home}{$file}</loc>
<lastmod>{$a}</lastmod>
</sitemap>
</sitemapindex>
");

At line 196 I added </sitemapindex> Which seemed to cure the problem.

Eagle Creek: You can use this on vb 3.0.7 as well (I am). There are no screen shots really, what it does it is generates (and gzips) xml files of your forums threads in the google site map format. It will also generate an index file of the XML files generated which you submit to google. Google uses this site map to more efficiently spider your site.

Gary King: Sorry I wasnt clear enough. Type in www.yoursite.com/vbsitemap-multi.php and hit enter to run it. It will then generate the files. You'll need to add vbsitemap-multi.php to a cron job to have it updated at regular intervals.

Hmm; maybe I will try it out then.

Thx.

Works great! When I added it to my scheduled tasks, it just brought up the page, and I had to manually click enter. Any pointers (or pages I should look at) that would explain how to make this into a cron job?

Thanks! :)

I'd probably just wait until time2dine releases the next version which should be out in the not to distance future. It may have that as a feature.

Setting it up for a cron job... something I will have a look at. I am away for the next week though. Sorry.

Set up the running of the file as a cron job with the url in the following format. Substitute the DOMAIN.COM, USERNAME and PASSWORD with the actual domain, username and password.

http://www.DOMAIN.COM/vbsitemap-mult...WORD&ping=true

Try it on your browser first to see that it works, then have it as a cron.

I'm having a little trouble with this. When I run it, my xml files all end up looking something like:
I have multiple loadbalanced webservers, and therefore I am running this on one of our boxes (the www1.site.com one), but it's messing up all the urls in my files. I tried doing a little ereg_replace action in the scripts, but there's so many different urls I sort of gave up.

Any ideas?

Oh also, is there the ability to put this in a dir off of the site root? I'm not keen on having so many files hanging out in root..

I have installed and configured the files and I am trying to log in with my administrator account.
It won't work.
Constantly getting an error:
Incorrect username/password
And I've even typed it to notepad and copy/pasted it, just to make sure I am not typing it wrong.
Won't work for this hack. But no problems whatsoever logging into my forum with the same name and password.
What can I do about it?
Joe

Never Mind. I fixed it.
Joe