Mini Mods - HTTP Error Response Codes by BOP5 (Soft 404 Solution)

Brought to you by BirdOPrey5 / Qapla.com

This issue was brought to my attention by Sherrie at TAZ in this thread.

The issue is that when Google crawls a site and runs into the vBulletin No Permission message or other standard vBulletin error pages vBulletin gives the standard "200" HTTP response code which means everything is OK, even though there is no real content on these pages. Therefore Google considers these "soft errors" and they can damage overall SEO of your site if there are too many of them.

What this does is make vBulletin return standard HTTP "Error" responses- by default 403 (Not Authorized) for No Permission pages and 404 (Not Found) for general error messages. When Google sees these responses (as opposed to 200) it knows not to bother indexing the page and everyone is happy.

As of version 2.2 new options have been added to return custom response codes when the forum is closed or when the forum is busy. It will also send a retry header telling the crawler they can try again later. As of version 2.0 there is also a separate response code for search.php errors.

A limitation of this mod is it works best with PHP 5.4.0 or higher. As of version 2.0 If you are on an older version of PHP it may not work, but it should not cause any errors, so it is worth trying.

There are no instructions, once installed it is enabled by default. In Admin CP -> Settings -> Options -> HTTP Error Responses by BOP5 you can enable/disable the mod and choose the response codes you want for the different error types. You can use any standard HTTP response code available from this list at wikipedia.

The mod works fine on VB 3.x or 4.x including VB 3.8.8 (still in Beta) and VB 4.2.2.

To test that it is working use a free online status checker such as: http://httpstatus.io/
(Note the site will cache your result for a while so test with a unique URL each time.)

Before you install the mod any URL that will show a no-permission or other vBulletin error will show a status code of 200. After installing the same page will show a 403, 404, or other code that you have configured.

Added Boothby as a "co-author" as thanks for his contribution of code and ideas. :up:

------------------------------------------------------

Please "Mark as Installed" if you use this.
Donations always appreciated. :up:
[s]Nominate MOTM if you LOVE it! [/s] (You did, Thanks!)

[hilaryl]

We have had this mod installed for a good while now. The only concerning thing is the number of pages we have reported in Search Console as Access Denied.

This is the purpose of the mod - so nothing is wrong.

I'm interested though, is 403 response better than 401 response? Would 401 still report as Access Denied in Search Console?

[BirdOPrey5]

403 is forbidden, Google might think forbidden is better than 401 but I can't say for any certainty.

What I read was that 401 is preferred when the entire site should not be accessible and 403 is prefered when it's a specific page that is not accessible.

[hilaryl]

Hi BOP5 - I've been trying to find a definitive answer too.

Surely there is a right or wrong, not just a preferred response. Obviously neither 401 or 403 are wrong, but there must be some standard. It's not as if forum pages that require a login are a new thing on web.

I will keep searching, and hopefully find something definitive.

Cheers

[Stratis]

Joe, a question. I have changed my forum to https, do I must change in plugins the code (http) to (https) ?

Thank you

[BirdOPrey5]

Nothing that I can see should have to change for https.

[BadgerDog]

Installed (twice) for testing....

We use vBulletin 4.1.3 pl8...

Works great, except we have a conflict with Paul's "Display Unread Posts" mod v4.1.7.

https://vborg.vbsupport.ru/showthread.php?t=231060

When the HTTP mod is installed, Paul's mod shows the unread post count (top right corner of web site) as it should, however, when one clicks on the post number to view them, vBulletin shows page saying there are no unread posts, even though there are. You can click the "What's New" tab and the new posts show fine there.

Tried to diagnose as follows:

1. Turned HTTP mod OFF (still a conflict)
2. Disabled HTTP mod (still a conflict)
3. Uninstalled HTTP mod and Paul's "Display Unread Posts" mod v4.1.7 began to work fine again.
4. Re-insalled HTTP mod and problem returned.
5. Finally uninstalled HTTP mod and left things alone.

Hope this helps someone else ..

Regards,
Doug

[djbaxter]

Originally Posted by BadgerDog

Installed (twice) for testing....

We use vBulletin 4.1.3 pl8...

Works great, except we have a conflict with Paul's "Display Unread Posts" mod v4.1.7.

https://vborg.vbsupport.ru/showthread.php?t=231060

When the HTTP mod is installed, Paul's mod shows the unread post count (top right corner of web site) as it should, however, when one clicks on the post number to view them, vBulletin shows page saying there are no unread posts, even though there are. You can click the "What's New" tab and the new posts show fine there.

Tried to diagnose as follows:

1. Turned HTTP mod OFF (still a conflict)
2. Disabled HTTP mod (still a conflict)
3. Uninstalled HTTP mod and Paul's "Display Unread Posts" mod v4.1.7 began to work fine again.
4. Re-insalled HTTP mod and problem returned.
5. Finally uninstalled HTTP mod and left things alone.

Hope this helps someone else ..

Regards,
Doug

Thank you. I have a similar problem with at least one forum and was unable to solve it. I thought it might have something to do with post counts not synching unread posts correctly. I am going to try your solution.

BOP5: If it is confirmed as a conflict between the two add-ons, can it be resolved?

[BirdOPrey5]

Originally Posted by BadgerDog

Installed (twice) for testing....

We use vBulletin 4.1.3 pl8...

Works great, except we have a conflict with Paul's "Display Unread Posts" mod v4.1.7.

https://vborg.vbsupport.ru/showthread.php?t=231060

When the HTTP mod is installed, Paul's mod shows the unread post count (top right corner of web site) as it should, however, when one clicks on the post number to view them, vBulletin shows page saying there are no unread posts, even though there are. You can click the "What's New" tab and the new posts show fine there.

Tried to diagnose as follows:

1. Turned HTTP mod OFF (still a conflict)
2. Disabled HTTP mod (still a conflict)
3. Uninstalled HTTP mod and Paul's "Display Unread Posts" mod v4.1.7 began to work fine again.
4. Re-insalled HTTP mod and problem returned.
5. Finally uninstalled HTTP mod and left things alone.

Hope this helps someone else ..

Regards,
Doug

I'm sorry but that makes no sense, if the mod is disabled from Product Manager it is basically the same as being uninstalled, there are no files to upload. If you need to remove the mod for Paul's mod to work I suspect any mod removed would fix it, it may be some sort of product cache issue.

In any case only the 4.2.0 version of Paul's mod is available for download at this time. Looking at the code for that I don't see any places they would conflict.

[BadgerDog]

Originally Posted by BirdOPrey5

I'm sorry but that makes no sense, if the mod is disabled from Product Manager it is basically the same as being uninstalled, there are no files to upload. If you need to remove the mod for Paul's mod to work I suspect any mod removed would fix it, it may be some sort of product cache issue.

In any case only the 4.2.0 version of Paul's mod is available for download at this time. Looking at the code for that I don't see any places they would conflict.

The sequence is repeatable and I've installed and uninstalled the HTTP mode 3 times to validate the steps and the same results occur every time. I have left the HTTP mod uninstalled and moved on, but thank you for your efforts.... much appreciated ...

I do use Sucuri's firewall protection system, which caches our site around the world, so perhaps it's having an affect only with this mod, although I don't know why.

https://sucuri.net/

Thanks again....

Regards,
Doug