PDA

View Full Version : Vbulletin Sitemap Actually Conflicts With Canonical Urls Preventing Proper Indexing


mykkal
05-24-2015, 12:57 AM
Is there a way to fix the vbulletin sitemap so that it returns the value of the Canonical Url? There is a thread on vBulletin.com which talks about vBulletin's SEO. It's actually against the grain as far as google webmaster tools is concerned.

For those that use htaccess to rewrite anything other than the default URL listings Vbulletin will make a sitemap contrary to the canonical URL resulting in a looping override of the canonical URL. The sitemap will say "these urls exist" and Google sees them as a preference. It will then send a robot to verify the page exist resulting in exposure to the canonical url. this cause google to drop references to the canonical URL as soon as the next sitemap is submitted. That is nightly for me.

The problem lies in /includes/class_sitemap.php.

Here is a word from Google expressingly stating not to do what vB functionality currently does:
https://support.google.com/webmasters/answer/139066?hl=en

While we encourage you to use any of these methods, none of them are required. If you don't indicate a canonical URL, we'll identify what we think is the best version or URL.
Don't use the robots.txt file for canonicalization purposes.
Don't use the URL removal tool for canonicalization: it removes all versions of a URL from search.
Don't specify different URLs as canonical for the same page (e.g. one URL in a sitemap and a different URL for that same page using rel="canonical")

--------------- Added 1432502849 at 1432502849 ---------------

FYI i'm asking if there is a preexisting canonical reference other than using 'bburl'? I'm trying to edit code on line 797 of includes/class_sitemap.php to get vbulletin to pull the url rewrites we use as the url for the sitemap to write. We've got tens of thousands of linkbacks preexisting from the site being 7 years old. The rewrites we used were custom and vbulletin doesn't have an option that displays them that way. The canonical urls are already rewritten to reflect what our url rewrites in htacess are. If I can get this to work then we will not have anymore problems with Google Webmaster tools.

If you know of w way please let me know:

$content .= "\n\t" . '<loc>' . htmlspecialchars_uni($this->registry->options['bburl'] . '/xmlsitemap.php?fn=' . urlencode($sitemap['loc'])) . '</loc>';

kh99
05-25-2015, 08:07 PM
So are you saying that, when generating the sitemap, you'd like to do the reverse of what your htaccess rewrite rules do? I don't know how to do that, but I was wondering if I understood correctly.

mykkal
05-25-2015, 09:36 PM
So are you saying that, when generating the sitemap, you'd like to do the reverse of what your htaccess rewrite rules do? I don't know how to do that, but I was wondering if I understood correctly.

Hey there, this is an issue unique to those that use a custom url structure. I fixed it by getting a custom made sitemap. Basically this was a problem that .htaccess couldn't fix. The vbulletin sitemap was creating links in a http://Yoursite.com/showthread.php?t-122232 format. Google looks at any sitemap as a prefered format. It also looks at a "rel=canonical" tag as a prefered format. Like say... Google begins to index in one format, visits the page and begins to index in another format. Then when the sitemap is checked again the cycle starts all over.

Basically the two canceled each other out. Now that the sitemap matches the Canonical tag things will index much quicker.

Plus this sitemap pings google anyways so I don't have to wait weeks for indexing. New posts are showing up in the index within an hour or so.

RichieBoy67
05-25-2015, 11:07 PM
Can you share your work? :)

mykkal
05-25-2015, 11:28 PM
Can you share your work? :)

I'll private message you. Last time i tried to do that some were offended.

SaN-DeeP
05-26-2015, 03:55 PM
way to fix vbulletin sitemap so that it returns the value of the Canonical Url
not doable in sitemaps FYI..

mykkal
05-26-2015, 04:13 PM
not doable in sitemaps FYI..

its been done. just can't do it in vBulletin's native sitemap.

SaN-DeeP
05-26-2015, 05:01 PM
its been done. just can't do it in vBulletin's native sitemap.
Hello mykkal,
This is great to understand.. Sorry but.
I just did not read your first post completely, just few lines and answered.
Can you post a working solution to rest of community ?

mykkal
05-26-2015, 05:34 PM
Hello mykkal,
This is great to understand.. Sorry but.
I just did not read your first post completely, just few lines and answered.
Can you post a working solution to rest of community ?

I cannot. I fixed the issue by fixing the VBSEO sitemap code to work with 4.2.3. Everytime I've tried to share the fixes we've implemented over the last weeks I've been harassed by members or shunned by moderators.

I was going to PM the changes to a member but I changed my mind after the headache of last week.

I can, however post to you my sitemap links so you can see that custom htaccess redirects & sitemaps do work. Then you can endeavor to fix it for yourself. I want to share but I've decided the headache of last week isn't worth it.

to make the sitemap work fix your purchased version of vBSEO first...then fix the sitemap. It will render urls in whatever format your htaccess rewrite rules reflect.

SaN-DeeP
05-26-2015, 06:04 PM
Thanks mykkal for your immediate reply.. Appreciated...

I will strictly suggest you to remove those above links (for your own site privacy/seuecirty first).
2nd vbseo should not allow me to download those files (check your rules)..
3rd do not share those links in public (like these places).. I have a note of those URIs.. will try to spend some time with same.. as we are already in process of removing vbseo from all our domain/site (s).

mykkal
05-26-2015, 06:13 PM
hey there, those are sitemap files from robots.txt. Anyone can see them. They aren't dangerous or anything. And honestly, having them posted here is probably beneficial for me too. lol

If you do use vBSEO you need to identify the security flaws in it.

TheLastSuperman
05-27-2015, 05:49 PM
Way I see it is, we all should just agree to disagree! Sounds good to me and then everyone still stands on their own two feet :D. If I'm inclined to close another thread I shall be stealing your drink sir! You all just play nice and I will hug you for it next time I see you :cool:.

Edit: ^^ Thread cleaned up, off-topic posts removed including two of my own so hopefully no one is offended if so I didn't mean to offend you, see I'm not biased (well lol :p).

Also as a reminder to those searching and happening across this thread in the future... there were some other seo type modifications and such for 3.x and early on in 4.x series and while none of them were as robust as the then king of seo plugins for vB (vBSEO was the primary seo software for years) they can still benefit some users more than others - not everyone needs tons of options - there are some simple seo type plugins and changes you can make that are very beneficial - I'll try to compile a list of those mods here soon just busy as heck lately so someone slap me upside the back of the head with a reminder if I forget but you can search now and find them.