Mini Mods - HTTP Error Response Codes by BOP5 (Soft 404 Solution)

Brought to you by BirdOPrey5 / Qapla.com

This issue was brought to my attention by Sherrie at TAZ in this thread.

The issue is that when Google crawls a site and runs into the vBulletin No Permission message or other standard vBulletin error pages vBulletin gives the standard "200" HTTP response code which means everything is OK, even though there is no real content on these pages. Therefore Google considers these "soft errors" and they can damage overall SEO of your site if there are too many of them.

What this does is make vBulletin return standard HTTP "Error" responses- by default 403 (Not Authorized) for No Permission pages and 404 (Not Found) for general error messages. When Google sees these responses (as opposed to 200) it knows not to bother indexing the page and everyone is happy.

As of version 2.2 new options have been added to return custom response codes when the forum is closed or when the forum is busy. It will also send a retry header telling the crawler they can try again later. As of version 2.0 there is also a separate response code for search.php errors.

A limitation of this mod is it works best with PHP 5.4.0 or higher. As of version 2.0 If you are on an older version of PHP it may not work, but it should not cause any errors, so it is worth trying.

There are no instructions, once installed it is enabled by default. In Admin CP -> Settings -> Options -> HTTP Error Responses by BOP5 you can enable/disable the mod and choose the response codes you want for the different error types. You can use any standard HTTP response code available from this list at wikipedia.

The mod works fine on VB 3.x or 4.x including VB 3.8.8 (still in Beta) and VB 4.2.2.

To test that it is working use a free online status checker such as: http://httpstatus.io/
(Note the site will cache your result for a while so test with a unique URL each time.)

Before you install the mod any URL that will show a no-permission or other vBulletin error will show a status code of 200. After installing the same page will show a 403, 404, or other code that you have configured.

Added Boothby as a "co-author" as thanks for his contribution of code and ideas. :up:

------------------------------------------------------

Please "Mark as Installed" if you use this.
Donations always appreciated. :up:
[s]Nominate MOTM if you LOVE it! [/s] (You did, Thanks!)

[PattiOz]

Brilliant. Perfect. Works like a charm! My SEO thanks you.

[Boothby]

Hi Joe. Could you add the following codes for search-results?

404 Not Found for no result found

410 Gone for outdated search-ID

When a distinction is not possible then the 410 code should be presented.

Thx

[BirdOPrey5]

Originally Posted by Boothby

Hi Joe. Could you add the following codes for search-results?

404 Not Found for no result found

410 Gone for outdated search-ID

When a distinction is not possible then the 410 code should be presented.

Thx

Please "Mark as Installed"


Updated to version 1.1. I wasn't able to determine whether it was no results found or outdated id however, so I added a new setting that will be the response code for any search.php errors. The default response for this is now 410.

I also added code that will prevent the mod from breaking your site if you install it on PHP 5.4+ but later you or your web host downgrade back to PHP 5.3. (Which happened to me.)

This was a minor code change so I believe it worked but I was unable to test it- if someone could test and confirm it works I'd appreciate it.

[Boothby]

Thx Joe. The question came from a member of vb-germany.org who wanted a 404 status code for search. And your mod was the closest. And now it fits perfectly.

[BirdOPrey5]

Thanks. I would like to add however the better way would be for search.php to be off limits via a robots.txt file so Google doesn't try to index search links at all.

This is a generic VB robots.txt file for a forum in the /forums/ directory of a website-

User-agent: *
Disallow: /forums/admincp/
Disallow: /forums/clientscript/
Disallow: /forums/cpstyles/
Disallow: /forums/customavatars/
Disallow: /forums/customprofilepics/
Disallow: /forums/images/
Disallow: /forums/modcp/
Disallow: /forums/attachment.php
Disallow: /forums/editpost.php
Disallow: /forums/image.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/profile.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usernote.php

[Boothby]

Those who have < PHP5.4 can add the following at the beginning of the error_generic plug-in and test it. This is a non standardized method, so it may or may not work.

PHP Code:

if (!function_exists('http_response_code'))
{
    function http_response_code($newcode = NULL)
    {
        static $code = 200;
        if($newcode !== NULL)
        {
            header('X-PHP-Response-Code: '.$newcode, true, $newcode);
            if(!headers_sent())
                $code = $newcode;
        }       
        return $code;
    }
} 


source: http://www.liketly.com/forum/thread/...response-code/

Thanks to RolandCP for the hint.

[DemOnstar]

I have 96 warnings from webmaster tools.



Not entirely sure how to deal with them and if this is the correct procedure but installed to see what happens.

[BirdOPrey5]

Originally Posted by Boothby

Those who have < PHP5.4 can add the following at the beginning of the error_generic plug-in and test it. This is a non standardized method, so it may or may not work.

PHP Code:

if (!function_exists('http_response_code'))
{
    function http_response_code($newcode = NULL)
    {
        static $code = 200;
        if($newcode !== NULL)
        {
            header('X-PHP-Response-Code: '.$newcode, true, $newcode);
            if(!headers_sent())
                $code = $newcode;
        }       
        return $code;
    }
} 


source: http://www.liketly.com/forum/thread/...response-code/

Thanks to RolandCP for the hint.

Wow- I tested this on 3 forums (1 VB 3.8, 2 VB 4.x) and it worked fine on all 3. :up:

You should copy the code to the top of both plugins however.

Do you mind if I add the code to the plugin?

[BirdOPrey5]

Originally Posted by DemOnstar

I have 96 warnings from webmaster tools.



Not entirely sure how to deal with them and if this is the correct procedure but installed to see what happens.

You should ignore those- the robots.txt is doing what it is supposed to do.

[addamroy]

damn, my host's server (urljet) is using a prior version of php

I was all excited to install this lol