Hi, I've been using this for a little while and just realized that for the most part, only my member profiles are being reported. This is supposed to report all posts, threads, etc. right? Is it because most of my forum is private?

Date 2009-06-14 05:42
Processing time 4.59 s
Total URLs 1,586 (-19)
Forumdisplay URLs 1 (-1)
Showthread URLs 0 (-)
ShowPost URLs 0 (-)
Archive URLs 2 (-1)
Member Profile URLs 1,581 (-17)
Poll Results URLs 0 (-)
Blog Entries URLs 0 (-)
Blog Tag URLs 0 (-)
Album URLs 0 (-)
Social Group URLs 0 (-)
Tag URLs 1 (-)

Yeah, personally I have Showpost URLs turned off because else the number gets too large but it looks like you have Showthread URLs turned off as well, which is the most important one.

Thank you. Unfortunately those options are set to Yes, so I'm not sure why they aren't showing... Anyone else have any ideas?

The threads must be visible to guests, otherwise they are not inlcuded in sitemap (since search engines will not be able to view them anyway).

I run a wordpress blog on the main portion of my site and have the forum installed on mysite.com/forum/

So through Wordpress I use a plugin that does a Google sitemap generator:

http://wordpress.org/extend/plugins/...map-generator/

So my question is should I use the vBSEO sitemap plugin in addition to this wordpress one as well or should I just run the wordpress one perhaps?

I assume Google doesn't take multiple sitemaps...

Thoughts?

https://www.google.com/webmasters/tools/
Get your site verified there, then you can submit multiple sitemaps.

Can someone help me verify if this is working? I get a log file within my vbseo_sitemap/data directory stating google received the sitemap, etc etc.

I feel like I did everything correctly but in reading post #263 by Ogmuk, he states to Shinng that his link (http://www.sportsgatherings.com/foru...p_index.xml.gz) should show a sitemap. When I try to see a sitemap at that same location but with my domain, I do not. I have that file on my server in the data directory, but when I go to www.pomoforacing.com/forums/sitemap_index.xml.gz, I get a "site not found."

Is it just that the link he typed above was incorrect? If I go to http://www.pomoforacing.com/forums/v...p_index.xml.gz, I get a "forbidden" (it's there, just forbidden to view).

Thanks.


*edit* For now, nevermind... for some reason, I don't have access to upload the .htaccess file to my forums directory. I can add the /data/ one, but not the main one. Unsure why and cannot access server control panel until I get home from work.

I installed this mod, checked it out and due to the fact that it ONLY adds links to your forum specific stuff like posts and blogs I uninstalled it. I would love if it actually crawled the site and indexed all the links from your site, not just the VB specific ones. Great idea, but very limited control and scope of what gets added to the sitemap files so I had to pass on this one.

thx. nice works.

Hello,

the .htacces goes to the forumroot (/forum) not the web accessible root of the site (/), right?

Correct.

Thank you Ogmuk.

My data folder contains some 87 sitemap-x-.xml.gz now and googlebot just started downloading them. It has downloaded over 260 files though (duplicates). Is that normal?

Hello,

Our forum is stored on a subdomain: http://forum.mysite.com The forum is located inside its own directory: http://forum.mysite.com/vb

I installed this version like instructed by the manual. So, the .htacces goes to http://forum.mysite.com/vb and the whole vbseo_sitemap directory and contents to http://forum.mysite.com/vb. Folders are CHMOD 777. In the ACP VBSEO settings search engines are notified automatically.

I ran the sitemap script and sitemaps are generated and stored inside http://forum.mysite.com//vb/vbseo_sitemap/data/

According to the report list in the ACP, the search engines are notified (confusing again, because Yahoo has to be notified manually?). According to the sitemap download log, the files are downloaded by googlebot. They were downloaded more than once yesterday after having installed the hack (shouldn't that be just once?).

Today there still is not a single sitemap file in google webmasters tool.

I guess something is still wrong?

This also confuses me. In the ACP/VBSEO settings it reads:

By default the vBSEO Sitemap files will be located in your forum root. Use this field to specify a different location, for example, your site root: http://www.yourdomain.com

NOTE: You must edit the .htaccess file of the folder in which you want to locate your sitemap files. More information here. Leave this setting empty to use default vB url.

I thought the files are stored in http://forum.mysite.com//vb/vbseo_sitemap/data/ by default?

No. Why do you have all those duplicates? Updates to the sitemap should overwrite the old files, not create brand new ones.

:confused:

If you created a subdomain for your forum, why would you then create a subfilder in that subdomain to install it? It would seem far more logical to install the forum in the root. You're making things unnecessarily complicated.

Second, the instructions as to where to find the sitemap need to be in the root directory unless you otherwise specifiy where to find them using the sitemap declaration in a robot.txt file in the root directory. In your case, the root directory is forum.mysite.com/ not forum.mysite.com/vb/.

I believe that was only when Yahoo was using the text format. Now that they're using the XML format like everyone else, it's all automatic. I've never done a manual notification.

It depends on how often you run the sitemap generator. Check your cron files.

Any errors will be noted in your Webmaster Central account - check there to find out what's wrong.

They are. Ignore this part for now. You have enough to figure out already and you certainly don't want to complicate it any further by messing with that setting.

Hello djbaxter,

Thank you for replying.

It doesn't make duplicate files in the directory. The googlebot seems to download them several times (within an hour, so they are not updated ones).


Because I can't create subdomains myself. The host is doing that and I have only a few. So, I have to put several things on one subdomain to prevent a cluttering and clash of names.

Hmm, thus the .htaccess supplied with this hack needs to be in forum.mysite.com and not forum.mysite.com/vb? But how comes google seems to be downloading the sitemap?
And that means I have to change the .htaccess a bit from

RewriteEngine On
RewriteRule ^((urllist|sitemap).*\.(xml|txt)(\.gz)?)$ vbseo_sitemap/vbseo_getsitemap.php?sitemap=$1 [L]

to:

RewriteEngine On
RewriteRule ^((urllist|sitemap).*\.(xml|txt)(\.gz)?)$ vb/vbseo_sitemap/vbseo_getsitemap.php?sitemap=$1 [L]

Right?

Ok.

The default one, I recall it's once a day.

No errors are reported.

Ok, the problem is just with the location of the .htaccess I guess?

1. I don't understand how it's possible that Google is trying to download the sitemap, not shoiwing it as successful, and yet not reporting any errors... unless you already have a sitemap in the root for forum.mysite.com? Try deleting ALL existing sitemaps and the manually running the cron job again from your vBulletin Admin CP (after relocating the .htaccess file).

2. No, you don't change the rewrite rules/instructions. That is already set up in the vB ACP. Leave it as it is.

Hello djbaxter,

No, I don't have a sitemap in the root.

I recall having experimented with vbseo a year ago or so and I did edit the .htaccess in the root then (http://forums.mysite.com). That info is still there in the rewrite rules:

RewriteRule ^((urllist|sitemap).*\.(xml|txt)(\.gz)?)$ vb/vbseo_sitemap/vbseo_getsitemap.php?sitemap=$1 [L]

{{Aik, I'm starting to understand what went wrong this year: nothing was there.}}

So, I only have the default htaccess supplied by this hack in forums.mydomain.com/vb and I remove the line from the root .htaccess? But how does google then know it should look into forums.mydomain.com/vb? Through the robot.txt like you mentioned? What do I put there?

Nothing at all? Is the right url submitted to the searchengines when completing the creation through CRON or run?

Is this the edited version? or the version created when you installed vbseo_sitemap? Don't use the edited one.

No. I didn't say remove the lines from .htaccess in the root. I said remove any sitemap files currently in the root. You need the redirect lines in the root directory version of .htaccess because it will redirect Google spiders when they go there to access them. Leave the lines in an .htaccess file in the /vb/ folder as well (I don't know how you have things set up in Webmaster Central - having it in both places shouldn't hurt and it may help).

It should be unless you've messed up any of the settings. If worst comes to worst, uninstall and then reinstall vbseo_sitemap to get back to the proper settings and then follow the directions given to you during the installation or in the instructions for installation.

It's an edited line I appended to an existing .ht in http://forums.mydomain.com
I'm pretty sure vb/ needs to be added to the path, otherwise it points to the non-existing folder http://forums.mydomain.com/vbseo_sitemap It is http://forums.mydomain.com/vb/vbseo_sitemap ?



You mean in vb's ACP regarding VBSEO? The default values.

:confused:

How can it be both the "default values" and an "edited line"?

In any case, try what I've suggested so far. I don't understand a lot of what you're posting here, not the least of which is the statement that Webmaster Central is trying to download the sitemap, apparently not finding it, and yet not showing any errors. Either I'm missing something or you are - I don't see how that's possible.

If my suggestions don't work, you may need to wait for one of the vBSEO people to log in.

Hello djbaxter,

Sorry for the confusion. By default values I mean what I set in Vbulletins ACP/VBSEO settings.

The edited line applies to the .ht I already had before I even knew about VBSEO. As far as I know searchbots only look in the root of a domain or subdomain for a sitemap. So, it has to be told there that it's somewhere else.

According to the VBSEO control panel, the sitemaps are created (and they are) and google is notified. Google then manages to download the files (according to the same VBSEO control panel). But when I go to googles Webmaster Central for my site, nothing is there.

hello what are the these files for in the htaccess file

isapi_rewrite, lighttpd_rewrite, zeus_rewrite

do i keep them in the htaccess folder and put the whole folder in the root directory
or
do i take those files out of the htaccess folder and put them in the root directory

Also it says that this file is in my root directory sitemap_index.xml.gz but its not. when i generate
a site map i dont see this in my root directory when i click on the link to see my site map it says
page not found? any thoughts on this

thanks

please answer..

going to buy gold soon as i get the ansawer

Hello. I use VBSEO and installed the sitemap v2.5
I want to add a line to robots.txt (EG: "sitemap: http://www.example.com/sitemap.xml")

What file is the sitemap? I see "sitemap_1.xml.gz" and "sitemap_index.xml.gz" and few others... I assume one of these is the file I need to point to?

Thnx, Kev

Yes. That's what the mod_rewrite line and vbseo_getsitemap.php does. The bots will look in the root (usually); the mod_rewrite and vbseo_getsitemap.php redirect it to where the files are actually physically located. If you try to do the work of those two functions, you'll end up confusing the bots, yourself, and everyone else. Think simple. Let the add-on do the complicated work.

If you are using an Apache server with mod_rewrite enabled, ignore those files. They are for alternate server operating systems or configurations.

See my reply to TosaInu above.

No. Just point to sitemap.xml -- mod_rewrite and vbseo_getsitemap.php take care of the rest.

Yes but... there is no sitemap.xml file to point to.. I am supposed to point to a file that is not there??

Yes. That's what those mod_rewrite lines do. They intercept any attempts to retirve sitemap.xml and redirect them to the appropriate file name and location.

Originally Posted by Budweiser
Yes but... there is no sitemap.xml file to point to.. I am supposed to point to a file that is not there??
Yes. That's what those mod_rewrite lines do. They intercept any attempts to retirve sitemap.xml and redirect them to the appropriate file name and location.

I don't have mod rewrite and I still don't
see the site map and I still get a page that's not there
when I try to viewy site map

I still don't get it

If you don't have mod_rewrite enabled on your server, the standard solution contained in the installation instructions for the vbseo_sitemap add-on will not work.

In that case, try the following...

Add the following line to the robots.txt file in the root directory of your site or create a plain text file (ASCII/ANSI format) containing the line and upload it to the root of your site:


This assumes your forum is installed in a subfolder called "forum". If the subfolder name is something else, change the line to reflect both your actual domain and the actual subfolder name.

If your forum is installed in the root directory, the line would look like this:

If your forum is installed in the root of a subdomain, it would look something like this:

1. Note that in all cases you are pointing the spiders to the real location of the sitemaps index file, which tells the spider how many sitemap files to look for and their filenames. This index file is created automatically by the vbseo_sitemap cron job.
   
2. Note that what I'm suggesting in this post is to be used instead of the standard mod_rewrite solution indicated in the vbseo_sitemap installation insturctions.

Hello i check the server and i do have it
mod_rewrite is enabled :
LoadModule rewrite_module libexec/mod_rewrite.so

so what should i do now since i do have mod_rewrite

thanks

Follow the installation instructions for the add-on and add the lines to your .htaccess file.

i dont see any place in the insstructions that mentions the add on as you mention and the .htaccess files i in the root directorie of the forum

Ok its working my .htaccess was not correct thanks for your help..going to your website right now as were speaking to see whats the difference on the free version and paid version

It's not my website. I'm not associated weith vBSEO and there is no "paid version" of the vBSEO sitemap. It is a free offering.

vBSEO itself is a vBulletin SEO package and it's not free.

I am trying to verify that I have this installed correctly. I tried testing my mod_rewrite as described at bvseo.com (Forum Link) but it didnt work right. I will follow up on that issue on their website.

Here is a screen shot of my report details (the site isn't open yet, not many posts)

Note: ALL of the search engine pings failed. Does that mean it is not installed correctly?

When I click on Sitemap Files>Index File... There is a reference to this URL but it is not valid on my server.. Is this correct/incorrect? (http://www.usamusicians.com/forum/sitemap_1.xml.gz)

Also when I click on urllist.txt I get this message..."Sitemap file not found:"
Looks like a problem to me but who knows...:erm:

There is no urllist.txt - that is an oudated, now-defunct format previously used by Yahoo. The link you posted to your sitemap worked just fine for me.

Please read what I posted above. I think you are confusing vBSEO, a commercial vBulletin SEO package, with vBSEO_sitemap. You have installed the free vbseo_sitemap only. You have NOT installed the vBSEO package. They are separate products and there is a huge difference.

I am not confused. I purchased and installed VBSEO a few days ago and it was suggested to me by their staff to install the sitemap also.... so I did. I just want to make sure everything is working optimally because I haven't used VBSEO or a sitemap before. If it is not installed correctly or is just not working correctly then what's the sense... I'm sure you can understand where I am coming from...

So the fact that the search engine pings failed is not an issue?

and.. If urllist.txt is an oudated, now-defunct format previously used by Yahoo then why is it included in this updated version of the sitemap app? (This is just a nag/rhetorical question)

My apologies. But if you have a l;egal vBSEO license, your best bet is to submit a support ticket and/or post in the sitemap forum there. You'll likely get a much faster response. Chances are one of the support people will get your authorization for FTP access and sort it out for you.

As for urllist.txt, I don't know - you'll have to ask the developers that question.

I don't know what's going on with the pings. Do you have a Google Webmaster Central accounbt? If not, get one. Add your forum as a site and add a sitemap to that listing in Webmaster Central. If Google is having a problem retrieving your sitemap, they should indicate that via error messages.

thanks post :)

Has anyone written an add-on module for vbSEO that will index the "Links and Downloads" vb module? I see one exists for Downloads II... but I'm using LDM...

Mike

im getting a 404 page error when google is trying to find the robot.txt file. this program dossent have this so how do i make or get a robot.txt file

any thoughts on how to make this??