Mini Mods - HTTP Error Response Codes by BOP5 (Soft 404 Solution)

Brought to you by BirdOPrey5 / Qapla.com

This issue was brought to my attention by Sherrie at TAZ in this thread.

The issue is that when Google crawls a site and runs into the vBulletin No Permission message or other standard vBulletin error pages vBulletin gives the standard "200" HTTP response code which means everything is OK, even though there is no real content on these pages. Therefore Google considers these "soft errors" and they can damage overall SEO of your site if there are too many of them.

What this does is make vBulletin return standard HTTP "Error" responses- by default 403 (Not Authorized) for No Permission pages and 404 (Not Found) for general error messages. When Google sees these responses (as opposed to 200) it knows not to bother indexing the page and everyone is happy.

As of version 2.2 new options have been added to return custom response codes when the forum is closed or when the forum is busy. It will also send a retry header telling the crawler they can try again later. As of version 2.0 there is also a separate response code for search.php errors.

A limitation of this mod is it works best with PHP 5.4.0 or higher. As of version 2.0 If you are on an older version of PHP it may not work, but it should not cause any errors, so it is worth trying.

There are no instructions, once installed it is enabled by default. In Admin CP -> Settings -> Options -> HTTP Error Responses by BOP5 you can enable/disable the mod and choose the response codes you want for the different error types. You can use any standard HTTP response code available from this list at wikipedia.

The mod works fine on VB 3.x or 4.x including VB 3.8.8 (still in Beta) and VB 4.2.2.

To test that it is working use a free online status checker such as: http://httpstatus.io/
(Note the site will cache your result for a while so test with a unique URL each time.)

Before you install the mod any URL that will show a no-permission or other vBulletin error will show a status code of 200. After installing the same page will show a 403, 404, or other code that you have configured.

Added Boothby as a "co-author" as thanks for his contribution of code and ideas. :up:

------------------------------------------------------

Please "Mark as Installed" if you use this.
Donations always appreciated. :up:
[s]Nominate MOTM if you LOVE it! [/s] (You did, Thanks!)

[Boothby]

This is original from vB_Bootstrap::check_state()

Maybe we should use the whole part, esp. the pemission check?

PHP Code:

        // check that board is active - if not admin, then display error
        if (
            !defined('BYPASS_FORUM_DISABLED')
                AND
            !$vbulletin->options['bbactive']
                AND
            !in_array(THIS_SCRIPT, array('login', 'css', 'mobile'))
                AND
            !($vbulletin->userinfo['permissions']['adminpermissions'] & $vbulletin->bf_ugp_adminpermissions['cancontrolpanel'])
        )
        { 


[BirdOPrey5]

Originally Posted by Boothby

This is original from vB_Bootstrap::check_state()

Maybe we should use the whole part, esp. the pemission check?

PHP Code:

        // check that board is active - if not admin, then display error
        if (
            !defined('BYPASS_FORUM_DISABLED')
                AND
            !$vbulletin->options['bbactive']
                AND
            !in_array(THIS_SCRIPT, array('login', 'css', 'mobile'))
                AND
            !($vbulletin->userinfo['permissions']['adminpermissions'] & $vbulletin->bf_ugp_adminpermissions['cancontrolpanel'])
        )
        { 


I see- those exceptions are so a site Administrator can still log in even with the forum off. The purpose of this mod is to return codes so search engines are aware of the status of the page- a search engine would never be browsing the site as an administrator... Also simply changing the response code doesn't prevent the page from loading in a browser- so I am confident we don't need the permission check or the script check.

Going to have an update available later today. :up:

[BirdOPrey5]

Updated to version 2.2. Added options for board closed, and board too busy. Added Boothby as supporter/co-author.

[RichieBoy67]

Originally Posted by BirdOPrey5

Updated to version 2.2. Added options for board closed, and board too busy. Added Boothby as supporter/co-author.

Nice work BOP!

[Mecho]

i've got "Possible outages" message from Google :

While crawling your site, we have noticed an increase in the number of transient soft 404 errors around 2014-05-08 13:00 UTC (London, Dublin, Edinburgh). Your site may have experienced outages. These issues may have been resolved

the example of links that google sent is alive!! I've sent the exact message to you via pm.

Thanks

[BirdOPrey5]

"transient soft" means that the 404 code was never sent- and as the URLs you sent me were to regular threads and not pages that should usually trigger a vBulletin Error message...

I see they message was dated May 8th - Is it possible your site really had a small outage? I would upgrade to the latest version and monitor to see if you get more issues.

[Mecho]

no outage , and i've updated to the latest version today to see the result.

the thing is that traffic has been dropped in last few weeks. it can be from many different things but in this case i'm afraid that this plugin send the same wrong message for exists threads to search engines and indexed links goes down and traffic either.

that's a guess only. correct me if i'm wrong.

thanks

[Boothby]

Can it be that google can index from public hidden content on your forums?

[RichieBoy67]

I monitor traffic on a daily basis on my sites and clients sites and have this installed on 3 different forums now and I have not noticed any change at all. I am using php4.3 though right now so that could be the reason. It may take time to show any changes in error pages as well.

I plan on upgrading php next week.

Is there any way to test this to see if it works? I have tried but I cannot tell a difference.

[CAG CheechDogg]

Originally Posted by RichieBoy67

I monitor traffic on a daily basis on my sites and clients sites and have this installed on 3 different forums now and I have not noticed any change at all. I am using php4.3 though right now so that could be the reason. It may take time to show any changes in error pages as well.

I plan on upgrading php next week.

Is there any way to test this to see if it works? I have tried but I cannot tell a difference.

Use this link, http://httpstatus.io/ ...once you put your link to test, once the test is done hover over the 200 code green box and you should see the response code change ....

If I understand how this works, the test gives you a 200 in the little green box but you see the response code once you hover over it ....hope that helps and hope I am giving you correct information lol ...