Mini Mods - HTTP Error Response Codes by BOP5 (Soft 404 Solution)

Brought to you by BirdOPrey5 / Qapla.com

This issue was brought to my attention by Sherrie at TAZ in this thread.

The issue is that when Google crawls a site and runs into the vBulletin No Permission message or other standard vBulletin error pages vBulletin gives the standard "200" HTTP response code which means everything is OK, even though there is no real content on these pages. Therefore Google considers these "soft errors" and they can damage overall SEO of your site if there are too many of them.

What this does is make vBulletin return standard HTTP "Error" responses- by default 403 (Not Authorized) for No Permission pages and 404 (Not Found) for general error messages. When Google sees these responses (as opposed to 200) it knows not to bother indexing the page and everyone is happy.

As of version 2.2 new options have been added to return custom response codes when the forum is closed or when the forum is busy. It will also send a retry header telling the crawler they can try again later. As of version 2.0 there is also a separate response code for search.php errors.

A limitation of this mod is it works best with PHP 5.4.0 or higher. As of version 2.0 If you are on an older version of PHP it may not work, but it should not cause any errors, so it is worth trying.

There are no instructions, once installed it is enabled by default. In Admin CP -> Settings -> Options -> HTTP Error Responses by BOP5 you can enable/disable the mod and choose the response codes you want for the different error types. You can use any standard HTTP response code available from this list at wikipedia.

The mod works fine on VB 3.x or 4.x including VB 3.8.8 (still in Beta) and VB 4.2.2.

To test that it is working use a free online status checker such as: http://httpstatus.io/
(Note the site will cache your result for a while so test with a unique URL each time.)

Before you install the mod any URL that will show a no-permission or other vBulletin error will show a status code of 200. After installing the same page will show a 403, 404, or other code that you have configured.

Added Boothby as a "co-author" as thanks for his contribution of code and ideas. :up:

------------------------------------------------------

Please "Mark as Installed" if you use this.
Donations always appreciated. :up:
[s]Nominate MOTM if you LOVE it! [/s] (You did, Thanks!)

[ZeroHour]

We had a problem using this mod where pages were cached on the error code.
Adding

header("Cache-Control: no-cache, must-revalidate");

Before any BOP header() line which meant error pages are not cached.

[BirdOPrey5]

Thanks, I will add that to the next version. :up:

[Stratis]

Where to copy this part of code?

[Stratis]

A question please...

Every time i see in webmasters three errors 403 and the url like this.
attachment.php?attachmentid=2178&d=xxxxxx
attachment.php?attachmentid=4659&d=xxxxxx
attachment.php?attachmentid=1721&d=xxxxxx

It says Access Denied, as very little i understand this is what supposed to be. I always make the action that they are fixed and after one day same thing happens.

Why does google read these over and over, if it cannot access them, even in robots.txt i have Disallow: /attachment.php, even if i delete this Disallow, same thing happens, i waited 4 days for robots to take effect with out the Disallow, nothing changed.

Do i must No Permission HTTP Code change from 403 to something else?

Any help will be grateful, thanks

[BirdOPrey5]

I'm really not sure why Google does the things it does. I would think robots.txt should be preventing access completely. It seems like a question to ask on the Google support site. Sorry.

[ZeroHour]

Originally Posted by BirdOPrey5

I'm really not sure why Google does the things it does. I would think robots.txt should be preventing access completely. It seems like a question to ask on the Google support site. Sorry.

Google dont index robots.txt stuff but they will still follow links to areas covered by it, pretty sure thats what I read a few years ago.

[Stratis]

Ok thanks both for answers, i see the way to ask google.

[ekempter]

Getting 404 errors on tags.php?tag=<forum> for users who are not logged on. I'm guessing that this is because I have search blocked to the public. Is there a work around?

[SWSUSA]

Doesn't work.

[BirdOPrey5]

Originally Posted by ekempter

Getting 404 errors on tags.php?tag=<forum> for users who are not logged on. I'm guessing that this is because I have search blocked to the public. Is there a work around?

Not that I am aware. Are you saying this mod is causing the errors? If tag searched is blocked to the public a 404 seems like the appropriate response.

Originally Posted by SWSUSA

Doesn't work.

You'll need to be more specific if you want help, it's obviously working for most people.