HTTP Error Response Codes by BOP5 (Soft 404 Solution)

[BirdOPrey5]

Brought to you by BirdOPrey5 / Qapla.com (http://www.qapla.com/mods/)

This issue was brought to my attention by Sherrie at TAZ in this thread (http://www.theadminzone.com/forums/showthread.php?t=107470).

The issue is that when Google crawls a site and runs into the vBulletin No Permission message or other standard vBulletin error pages vBulletin gives the standard "200" HTTP response code which means everything is OK, even though there is no real content on these pages. Therefore Google considers these "soft errors" and they can damage overall SEO of your site if there are too many of them.

What this does is make vBulletin return standard HTTP "Error" responses- by default 403 (Not Authorized) for No Permission pages and 404 (Not Found) for general error messages. When Google sees these responses (as opposed to 200) it knows not to bother indexing the page and everyone is happy.

As of version 2.2 new options have been added to return custom response codes when the forum is closed or when the forum is busy. It will also send a retry header telling the crawler they can try again later. As of version 2.0 there is also a separate response code for search.php errors.

A limitation of this mod is it works best with PHP 5.4.0 or higher. As of version 2.0 If you are on an older version of PHP it may not work, but it should not cause any errors, so it is worth trying.

There are no instructions, once installed it is enabled by default. In Admin CP -> Settings -> Options -> HTTP Error Responses by BOP5 you can enable/disable the mod and choose the response codes you want for the different error types. You can use any standard HTTP response code available from this list at wikipedia (http://en.wikipedia.org/wiki/List_of_HTTP_status_codes).

The mod works fine on VB 3.x or 4.x including VB 3.8.8 (still in Beta) and VB 4.2.2.

To test that it is working use a free online status checker such as: http://httpstatus.io/
(Note the site will cache your result for a while so test with a unique URL each time.)

Before you install the mod any URL that will show a no-permission or other vBulletin error will show a status code of 200. After installing the same page will show a 403, 404, or other code that you have configured.

Added Boothby as a "co-author" as thanks for his contribution of code and ideas. :up:

------------------------------------------------------

Please "Mark as Installed" if you use this. :)
Donations always appreciated. :up:
Nominate MOTM if you LOVE it! ;) (You did, Thanks!)

[BirdOPrey5]

Reserved.

[DemOnstar]

Good lad BOP5, on the ball as ever....

[Justinphx]

Excellent. Thanks and works like a charm.

[EddieDean]

This so sad. Today I was looking to solve those error and find your thread, but my forum is in Banahosting and the PHP version on the server is 5.3.28... There is a way I can solve this without migrate to other hosting?

[ozzy47]

See if your host will update the PHP, if not then you would need to either find a different host, or not use this particular mod. :(

[BirdOPrey5]

This so sad. Today I was looking to solve those error and find your thread, but my forum is in Banahosting and the PHP version on the server is 5.3.28... There is a way I can solve this without migrate to other hosting?

Sorry, been looking for ways to do it in older PHP versions but the only work-around uses the header() function, the problem is that function must be called before any output is created by the script and the hooks for the error messages aren't called until after script produces some output- so the header function calls are basically ignored. I'm not seeing any alternatives.

[Thangvip9x]

I am using vbulletin 4.2.2 but version PHP 5.3, i very want use this mod, you can modify this product

[ozzy47]

That was answered in the post above yours. :)

[ramesh_umk3]

So if i want to use this i have to spare my VBSEO right ?

[BirdOPrey5]

So if i want to use this i have to spare my VBSEO right ?

I don't know how it would work with VBSEO or not, I don't have VBSEO so I can't and don't test against it.

[djbaxter]

No reason why it shouldn't work with vBSEO.

[EddieDean]

I don't know how it would work with VBSEO or not, I don't have VBSEO so I can't and don't test against it.

I have VBSEO and its working like a charm.

[ForceHSS]

And no way to make it work with lower php version no way at all

if i edit the min php on the xml
<dependency dependencytype="php" minversion="5.4.0" maxversion="" />
Should it not work

Update got it working for a lower php

[Naijasite]

i just installed the mod it still show a status code of 200. the status did not change to 404

[BirdOPrey5]

I just tried a random bad URL from the site in your signature

http://www.questlot.com/forums/1998

On the website - http://httpstatus.io

And it is showing a 404 response code indicating the mod is working. What URL are you trying that you are getting a 200?

Also- Please "Mark as Installed"

[djbaxter]

Just received this in an email from from Google Webmaster Tools:

http://forum.psychlinks.ca/: Increase in authorization permission errors
Google detected a significant increase in the number of URLs we were blocked from crawling due to authorization permission errors.


I'd say that's pretty good evidence that this add-on is working perfectly. :)

[puertoblack2003]

nice mod. should i install this along side dbseo?

[BirdOPrey5]

nice mod. should i install this along side dbseo?

Would probably just suggest DBSEO add the feature if they haven't already, but I see no reason it should be a problem.

[ramesh_umk3]

Are your sure brother Bird

DragonByte SEO v1.0.10

Changes to Existing Features:

"No Permission" error screens now send a 403 Access Restricted header
"Go To Post" URLs now redirect to the normal thread URL, improving the canonical URL functionality


Bug Fixes And Issue Fixes:

"Go To Post" URLs now redirect to the normal thread URL
Custom Rewrite Rules would in some circumstances not work as intended
Using the Forum Path system without a forum ID in the path format would create invalid rewritten URLs


Using your mod along with latest dragonbyte wont get any conflicts right ?

Regards,

[BirdOPrey5]

I'm not sure about using it with the dragon byte mod, no- as I don't have DBSEO installed and never intend to install it- so I will never be testing my mods against it- just like I never tested them against VBSEO. It is most certainly use at your own risk.

I don't believe there will be a conflict. I also don't know when it will be added to DBSEO- but I was in communication with them and was told they do intend to add the feature. I suppose plans can always change but it is such an easy feature I doubt they would change their mind.

[krimznheidiz]

There is an error show " The following dependencies were not met:
This product is not compatible with version 5.3.28 of PHP. (Compatible starting with 5.4.0)"

[BirdOPrey5]

There is an error show " The following dependencies were not met:
This product is not compatible with version 5.3.28 of PHP. (Compatible starting with 5.4.0)"

That means that your server is running PHP 5.3.28 but I made it clear you need at least PHP 5.4.0 in the description of this mod. It will not work on your server.

[karnevil]

Hi BOP5 and all,

Question for you if I may

I regularly check google webmaster tools for crawl errors such as this, on our forum www.legalbeagles.info/forums/ we have a VIP area http://www.legalbeagles.info/forums/forumdisplay.php?35-VIP-MEMBERS-FORUM which google obviously does not have crawl access to.

If a member posts(publicly and crawlable) a link to a thread/post contained in the VIP area google crawls that post, tries to follow the link and comes up with the no permissions page.

This is then logged as a soft 404 error in webmaster tools. What is the best way to either return a different code that will not affect the crawl errors or prevent it from returning an error?

Would this mod fix this?

[BirdOPrey5]

Hi BOP5 and all,

Question for you if I may

I regularly check google webmaster tools for crawl errors such as this, on our forum www.legalbeagles.info/forums/ we have a VIP area http://www.legalbeagles.info/forums/forumdisplay.php?35-VIP-MEMBERS-FORUM which google obviously does not have crawl access to.

If a member posts(publicly and crawlable) a link to a thread/post contained in the VIP area google crawls that post, tries to follow the link and comes up with the no permissions page.

This is then logged as a soft 404 error in webmaster tools. What is the best way to either return a different code that will not affect the crawl errors or prevent it from returning an error?

Would this mod fix this?

This is exactly the mod to fix this. Under the default setup Google would instead get a 403 (not authorized) code on such pages.

[karnevil]

Excellent thank you,

We are running vB 4.2.1, not sure of the php version as yet, how would I check this?

[karnevil]

Found it, PHP Version 5.2.10

So reading back this mod will not work for us :(

[Alibass]

Found it, PHP Version 5.2.10

So reading back this mod will not work for us :(

Check with your hosting company, they may have a simple line code to add to your .htaccess to access 5.4

[karnevil]

Will do thanks.

Just looking through the templates to see where nopermission generates a 404 error to see if this can be modified

[BirdOPrey5]

Check with your hosting company, they may have a simple line code to add to your .htaccess to access 5.4

If you do upgrade to PHP 5.4.x you must upgrade to VB 4.2.2 as well. You might as well upgrade to VB 4.2.2 first.

Will do thanks.

Just looking through the templates to see where nopermission generates a 404 error to see if this can be modified

Fairly sure it isn't in the templates. If it is expressly set at all it would be in the PHP code.

[karnevil]

After looking through which template was returning the 404 error

STANDARD_ERROR

I replaced the 404 Not Found text to read 403 Forbidden

I checked that a genuine 404 page still came up as such so I know it hasn't affected that.

Not ideal but will see what results webmaster crawl errors comes back with over the next couple of weeks and let you know.

[BirdOPrey5]

I think there may be some confusion... Standard Error doesn't return a 404 error... If it did this mod wouldn't be necessary- the response code on standard error is 200 (OK) regardless of the actual text displayed in the browser- the text in the browser doesn't determine the response code Google gets. The response code isn't a visible element.

Am I right you have VBSEO installed? I don't think there is any other means vBulletin would display a 404 error message?

[Mickie D]

Thank you :)

Perfect Mod, should be standard built into vbulletin!!

Mick

[Mecho]

Awesome

it works without any issue on 4.2.2 PL1 - Vbseo installed.

First error number was 200
after installing this, it is 403

thanks BirdOPrey5

[synseal]

Hi and thanks for this!

In my webmasters tools I get a load of errors like this all related to tags

https://vborg.vbsupport.ru/external/2014/04/15.jpg (http://s429.photobucket.com/user/HiTeck1979/media/23-04-201417-27-31.jpg.html)

Shouldn't this modification fix this with the 403 no permission message?

Crawlers are getting to these pages (Tags) and because I don't allow guests to use the search function they are getting these type of errors and marking it as a soft 404.

https://vborg.vbsupport.ru/external/2014/04/16.jpg (http://s429.photobucket.com/user/HiTeck1979/media/23-04-201417-35-30.jpg.html)

https://vborg.vbsupport.ru/external/2014/04/17.jpg (http://s429.photobucket.com/user/HiTeck1979/media/23-04-201417-36-47.jpg.html)

Any idea TIA. :cool:

[BirdOPrey5]

I can confirm http://www.techkings.org/tags/worked.html is now getting a REAL 404 response code. So those soft errors should go away soon.

[synseal]

I can confirm http://www.techkings.org/tags/worked.html is now getting a REAL 404 response code. So those soft errors should go away soon.

These are errors I am still getting after installing this modification.

I cleared all the errors in webmasters tools (and I had loads) and these tag errors have come back after installing this, see the "detected dates" above in the image :up:

[BirdOPrey5]

if you put any of your "tag" URLs into this site- http://httpstatus.io - it confirms a 404 is being returned. It isn't 403 because by default the mod sends 403 for "No Permission" errors only and 404 for all others.

The dates on the image are all several or more days ago so I assume you won't see any more past today assuming the mod remains active.

However it is pretty clear you should add your entire /tags/ path to your robots.txt file, that would be the right way to prevent the crawl issues.

[Mecho]

installed and result is great. check soft.png (attached)

just one question , after installing this Access Denied urls increased (access.png) . All these urls are for the threads which has been moved to Trash and those redirect to Trashcan url. is it ok or bad for SEO?

[BirdOPrey5]

From what I've read (I'm not claiming to be an expert) a 403/404 won't hurt overall SEO although the actual page causing the error will eventually be de-indexed. (Which is exactly what we want to happen.)

[PattiOz]

Brilliant. Perfect. Works like a charm! My SEO thanks you.

[Boothby]

Hi Joe. Could you add the following codes for search-results?

404 Not Found for no result found

410 Gone for outdated search-ID

When a distinction is not possible then the 410 code should be presented.

Thx

[BirdOPrey5]

Hi Joe. Could you add the following codes for search-results?

404 Not Found for no result found

410 Gone for outdated search-ID

When a distinction is not possible then the 410 code should be presented.

Thx
Please "Mark as Installed"


Updated to version 1.1. I wasn't able to determine whether it was no results found or outdated id however, so I added a new setting that will be the response code for any search.php errors. The default response for this is now 410.

I also added code that will prevent the mod from breaking your site if you install it on PHP 5.4+ but later you or your web host downgrade back to PHP 5.3. (Which happened to me.)

This was a minor code change so I believe it worked but I was unable to test it- if someone could test and confirm it works I'd appreciate it.

[Boothby]

Thx Joe. The question came from a member of vb-germany.org who wanted a 404 status code for search. And your mod was the closest. And now it fits perfectly. ;)

[BirdOPrey5]

Thanks. I would like to add however the better way would be for search.php to be off limits via a robots.txt file so Google doesn't try to index search links at all.

This is a generic VB robots.txt file for a forum in the /forums/ directory of a website-


User-agent: *
Disallow: /forums/admincp/
Disallow: /forums/clientscript/
Disallow: /forums/cpstyles/
Disallow: /forums/customavatars/
Disallow: /forums/customprofilepics/
Disallow: /forums/images/
Disallow: /forums/modcp/
Disallow: /forums/attachment.php
Disallow: /forums/editpost.php
Disallow: /forums/image.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/profile.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usernote.php

[Boothby]

Those who have < PHP5.4 can add the following at the beginning of the error_generic plug-in and test it. This is a non standardized method, so it may or may not work.

if (!function_exists('http_response_code'))
{
function http_response_code($newcode = NULL)
{
static $code = 200;
if($newcode !== NULL)
{
header('X-PHP-Response-Code: '.$newcode, true, $newcode);
if(!headers_sent())
$code = $newcode;
}
return $code;
}
}

source: http://www.liketly.com/forum/thread/15416/php-how-to-send-http-response-code/

Thanks to RolandCP for the hint.

[DemOnstar]

I have 96 warnings from webmaster tools.

https://vborg.vbsupport.ru/attachment.php?attachmentid=149177&stc=1&d=1400664625

Not entirely sure how to deal with them and if this is the correct procedure but installed to see what happens.

[BirdOPrey5]

Those who have < PHP5.4 can add the following at the beginning of the error_generic plug-in and test it. This is a non standardized method, so it may or may not work.

if (!function_exists('http_response_code'))
{
function http_response_code($newcode = NULL)
{
static $code = 200;
if($newcode !== NULL)
{
header('X-PHP-Response-Code: '.$newcode, true, $newcode);
if(!headers_sent())
$code = $newcode;
}
return $code;
}
}

source: http://www.liketly.com/forum/thread/15416/php-how-to-send-http-response-code/

Thanks to RolandCP for the hint.

Wow- I tested this on 3 forums (1 VB 3.8, 2 VB 4.x) and it worked fine on all 3. :up:

You should copy the code to the top of both plugins however.

Do you mind if I add the code to the plugin?

[BirdOPrey5]

I have 96 warnings from webmaster tools.

https://vborg.vbsupport.ru/attachment.php?attachmentid=149177&stc=1&d=1400664625

Not entirely sure how to deal with them and if this is the correct procedure but installed to see what happens.

You should ignore those- the robots.txt is doing what it is supposed to do.

[addamroy]

damn, my host's server (urljet) is using a prior version of php :(

I was all excited to install this lol

[DemOnstar]

I tested the urls using the site that you provided and do still show status 200.

You should ignore those- the robots.txt is doing what it is supposed to do.

Cheers Joe, as you say...

[BirdOPrey5]

damn, my host's server (urljet) is using a prior version of php :(

I was all excited to install this lol

See post 46 b& 48, I have URLjet and it is working with those changes.

[BirdOPrey5]

I tested the urls using the site that you provided and do still show status 200.


What URL did you test?

[CAG CheechDogg]

Wow- I tested this on 3 forums (1 VB 3.8, 2 VB 4.x) and it worked fine on all 3. :up:

You should copy the code to the top of both plugins however.

Do you mind if I add the code to the plugin?

What does this exactly do BOP5? I installed this on my forums using 5.4 "then" reverted my PHP back to 5.3 since my website not my forums wont work with 5.4 ...I have an old version of Joomla as my CMS which is nicely bridged with my VB forums so using 5.4 is not even an option so I have to stick with 5.3..

[Boothby]

Do you mind if I add the code to the plugin?

Joe, feel free to use this code. I see no problems not to do so. Look at the small source-link. I think this code is too generic so that everyone can use it. :)

[BirdOPrey5]

Joe, feel free to use this code. I see no problems not to do so. Look at the small source-link. I think this code is too generic so that everyone can use it. :)

Great, thanks! Mod has been updated to version 2.0 to include this code. No longer limited to PHP 5.4.0 or higher.

What does this exactly do BOP5? I installed this on my forums using 5.4 "then" reverted my PHP back to 5.3 since my website not my forums wont work with 5.4 ...I have an old version of Joomla as my CMS which is nicely bridged with my VB forums so using 5.4 is not even an option so I have to stick with 5.3..

Suggest you upgrade to version 2.0. If it can work, it will work. It should not cause any errors on your site even if the code doesn't work for some reason.

The new code attempts to change the response code if it can (which isn't guaranteed on old PHP versions) but it won't bug out if it can't it just won't do anything.

[addamroy]

Saweeet.

Installed.

[CAG CheechDogg]

Great, thanks! Mod has been updated to version 2.0 to include this code. No longer limited to PHP 5.4.0 or higher.

Suggest you upgrade to version 2.0. If it can work, it will work. It should not cause any errors on your site even if the code doesn't work for some reason.

The new code attempts to change the response code if it can (which isn't guaranteed on old PHP versions) but it won't bug out if it can't it just won't do anything.

Good stuff BOP5 and thanks Boothby for the addition !

[Boothby]

I looked in the whole code and I think that this small piece can give false results, when working with the custom pre-5.4 http_response_code() function:

... http_response_code() != $vbo['bop5he_noperm_err'] ...

Then I have 2 suggestions:

First I found a way to look for outdated searchids. Second maybe we should send a 503 code for forum closed and server too busy?

// hook: search_before_process, only for >vB4.0
// outdated searchids
if(is_object($this) AND $this->results === null)
{
define('ERR_RESPONSE_CODE', 410);
}




// ERR_RESPONSE_CODE somewhere defined in the whole wide world?
if(defined('ERR_RESPONSE_CODE'))
{
http_response_header(ERR_RESPONSE_CODE);
}
// server to busy
else if($vbulletin->loadcache['loadavg'] > $vbulletin->options['loadlimit'])
{
http_response_header(503);
header('Retry-After: 60'); // 60 seconds is ok, isn't it?
}
// forum closed
else if(
!$vbulletin->options['bbactive']
AND
!in_array(THIS_SCRIPT, array('login', 'css', 'mobile'))
)
{
http_response_header(503);
header('Retry-After: 3600'); // 1 hour, put this in a new option?
}
// and your original code
else if (THIS_SCRIPT == 'search')
{
http_response_code($vbo['bop5he_ser_err']);
}
else
{
http_response_code($vbo['bop5he_gen_err']);
}



untested :D

[BirdOPrey5]

Boothby - These are some great ideas- thanks- I will incorporate most, if not all, into the next version. I will list you as a co-author. :)

What is the logic behind this line-

!in_array(THIS_SCRIPT, array('login', 'css', 'mobile'


Why should we check what script it is on if the board is turned off?

[Boothby]

This is original from vB_Bootstrap::check_state()

Maybe we should use the whole part, esp. the pemission check?

// check that board is active - if not admin, then display error
if (
!defined('BYPASS_FORUM_DISABLED')
AND
!$vbulletin->options['bbactive']
AND
!in_array(THIS_SCRIPT, array('login', 'css', 'mobile'))
AND
!($vbulletin->userinfo['permissions']['adminpermissions'] & $vbulletin->bf_ugp_adminpermissions['cancontrolpanel'])
)
{

[BirdOPrey5]

This is original from vB_Bootstrap::check_state()

Maybe we should use the whole part, esp. the pemission check?

// check that board is active - if not admin, then display error
if (
!defined('BYPASS_FORUM_DISABLED')
AND
!$vbulletin->options['bbactive']
AND
!in_array(THIS_SCRIPT, array('login', 'css', 'mobile'))
AND
!($vbulletin->userinfo['permissions']['adminpermissions'] & $vbulletin->bf_ugp_adminpermissions['cancontrolpanel'])
)
{

I see- those exceptions are so a site Administrator can still log in even with the forum off. The purpose of this mod is to return codes so search engines are aware of the status of the page- a search engine would never be browsing the site as an administrator... Also simply changing the response code doesn't prevent the page from loading in a browser- so I am confident we don't need the permission check or the script check.

Going to have an update available later today. :up:

[BirdOPrey5]

Updated to version 2.2. Added options for board closed, and board too busy. Added Boothby as supporter/co-author.

[RichieBoy67]

Updated to version 2.2. Added options for board closed, and board too busy. Added Boothby as supporter/co-author.
Nice work BOP!

[Mecho]

i've got "Possible outages" message from Google :

While crawling your site, we have noticed an increase in the number of transient soft 404 errors around 2014-05-08 13:00 UTC (London, Dublin, Edinburgh). Your site may have experienced outages. These issues may have been resolved

the example of links that google sent is alive!! I've sent the exact message to you via pm.

Thanks

[BirdOPrey5]

"transient soft" means that the 404 code was never sent- and as the URLs you sent me were to regular threads and not pages that should usually trigger a vBulletin Error message...

I see they message was dated May 8th - Is it possible your site really had a small outage? I would upgrade to the latest version and monitor to see if you get more issues.

[Mecho]

no outage , and i've updated to the latest version today to see the result.

the thing is that traffic has been dropped in last few weeks. it can be from many different things but in this case i'm afraid that this plugin send the same wrong message for exists threads to search engines and indexed links goes down and traffic either.

that's a guess only. correct me if i'm wrong.

thanks

[Boothby]

Can it be that google can index from public hidden content on your forums?

[RichieBoy67]

I monitor traffic on a daily basis on my sites and clients sites and have this installed on 3 different forums now and I have not noticed any change at all. I am using php4.3 though right now so that could be the reason. It may take time to show any changes in error pages as well.

I plan on upgrading php next week.

Is there any way to test this to see if it works? I have tried but I cannot tell a difference.

[CAG CheechDogg]

I monitor traffic on a daily basis on my sites and clients sites and have this installed on 3 different forums now and I have not noticed any change at all. I am using php4.3 though right now so that could be the reason. It may take time to show any changes in error pages as well.

I plan on upgrading php next week.

Is there any way to test this to see if it works? I have tried but I cannot tell a difference.

Use this link, http://httpstatus.io/ ...once you put your link to test, once the test is done hover over the 200 code green box and you should see the response code change ....

If I understand how this works, the test gives you a 200 in the little green box but you see the response code once you hover over it ....hope that helps and hope I am giving you correct information lol ...

[RichieBoy67]

Use this link, http://httpstatus.io/ ...once you put your link to test, once the test is done hover over the 200 code green box and you should see the response code change ....

If I understand how this works, the test gives you a 200 in the little green box but you see the response code once you hover over it ....hope that helps and hope I am giving you correct information lol ...

Thanks bro, awesome!

Just to verify here I am doing this correctly, One test would be to put a non existent path into the form and it should show a 404 error instead of the Vbulletin no permission page?

I think I may be a little confused as to what this does. This does not change error pages but rather response header code so those pages are no longer crawled therefor reducing soft errors and the like?

[CAG CheechDogg]

Thanks bro, awesome!

Just to verify here I am doing this correctly, One test would be to put a non existent path into the form and it should show a 404 error instead of the Vbulletin no permission page?

I think I may be a little confused as to what this does. This does not change error pages but rather response header code so those pages are no longer crawled therefor reducing soft errors and the like?

Yes correct, if I understand it correctly that's how its supposed to work ...

[BirdOPrey5]

Thanks bro, awesome!

Just to verify here I am doing this correctly, One test would be to put a non existent path into the form and it should show a 404 error instead of the Vbulletin no permission page?

I think I may be a little confused as to what this does. This does not change error pages but rather response header code so those pages are no longer crawled therefor reducing soft errors and the like?

Not exactly...

This mod kicks in when vBulletin shows a vBulletin generated error message.

One way to test it is to test a thread URL such as-

http://www.qapla.com/mods/showthread.php/1203

(you will see that URL works and response code will be 200)

then change the thread number to something much higher, that you don't have a valid thread for like-

http://www.qapla.com/mods/showthread.php/999999999999

Now that should return a 404 response code... If this mod wasn't enabled then that URL would return a 200 response code. The 404 proves the mod is working.

Be aware when testing on your own forum the exact URL may be cached by the status checker... so if you test before the mod is installed be sure to use a different URL (even if it is just a little different) when you test again after installing the mod.

If you used a URL that doesn't exist at all like
http://www.qapla.com/mods/shugugaga

That returns a 404 response because your server is producing a 404 page, vBulletin isn't running at all in this case, it has nothing to do with this mod.

[BirdOPrey5]

no outage , and i've updated to the latest version today to see the result.

the thing is that traffic has been dropped in last few weeks. it can be from many different things but in this case i'm afraid that this plugin send the same wrong message for exists threads to search engines and indexed links goes down and traffic either.

that's a guess only. correct me if i'm wrong.

thanks

I really don't believe this mod could have caused the Google issues- the hook it runs on simply isn't even executed unless vBulletin is already showing an error message. The error message must be called first, it can't call the error message itself.

[puertoblack2003]

I really don't believe this mod could have caused the Google issues- the hook it runs on simply isn't even executed unless vBulletin is already showing an error message. The error message must be called first, it can't call the error message itself.

This mod has nothing to do with it. I had that issue way before installing this.I had an occasion server hiccup when sometimes it'll go off line for a couple of minutes. i'm guessing during the time google is indexing. If i remember correctly google has a feature when that happens to mark as fix.. can remember though.

[tbworld]

Nice work @BirdoPrey, and thanks @Boothby! :)

[AusPhotography]

Thanks, working perfectly

[BirdOPrey5]

Updated to version 2.1.1. Fixed error on for VB 3.x users on legacy PHP versions (below 5.4.0)

[joeychgo]

Joe, see this thread...

http://www.vbulletin.com/forum/forum/vbulletin-4/vbulletin-4-questions-problems-and-troubleshooting/4042091-unusual-css-problems

[BirdOPrey5]

Joe, see this thread...

http://www.vbulletin.com/forum/forum/vbulletin-4/vbulletin-4-questions-problems-and-troubleshooting/4042091-unusual-css-problems

What version of PHP is on your server?

What exact version of this plugin was installed?

[joeychgo]

php ---- 5.4.20

Plugin Version 1.0.0

I just upgraded to the current version, same issues.

[BirdOPrey5]

Wow... 1.0.0 is a super-simple script... there is really nothing that can go wrong. A lot of people seem to be using it without this issue. Is it possible there is something on your site that somehow overrides the data sent when a 404 or similar error message is generated?

I really can't conceive how this mod could be causing the error because, especially version 1.0.0, you can see the only plugins are on error_ hooks which mean the code isn't even ran until vBulletin is already set to produce an error. If vBulletin is in error mode for example then the css isn't going to be returned anyway on a call to css.php for example.

Perhaps some sort of caching system that reads a 404 error and caches a 404 error page for a short time rather than returning then passing the data to vBulletin and waiting for a real page to be generated?

I'm kind of grasping at straws here... Are there any PHP errors in the error logs when this happens?

[joeychgo]

Nope, no errors in the logs....

[Alibass]

Nope, no errors in the logs....
I just went to your website and get this error ising IE, FF and Chrome
Invalid Page URL. If this is an error and the page should exist, please contact the system administrator and tell them how you got this message. Have you changed anything in your settings/site name/url/contact details or do you possibly have an old cookie

[joeychgo]

haven't changed anything

where did you see that

[AusPhotography]

PHP 5.4.29, Apache 2.4.9, MySQL 5.6.16, vB4.2.2pl1 -- all goodness here.

[joeychgo]

PHP 5.4.29, Apache 2.4.9, MySQL 5.6.16, vB4.2.2pl1 -- all goodness here.

No its not...

I get this when I click LOG IN on your site

[BirdOPrey5]

No its not...

I get this when I click LOG IN on your site

I don't... Tried Chrome and IE... What version of IE are you using? Looks old?

[BirdOPrey5]

Nope, no errors in the logs....

I see you have vB Optimise and DBSEO... My guess is a conflict with one of these.

It is my understand DBSEO has added this feature (or will soon) - double check it's not trying to do the same thing already.

However now you have issues with the photography site a browser issue is possible too.

[joeychgo]

I don't have either..... vB Optimise or DBSEO

Browser issue is possible, but I only have trouble with sites that have this mod installed. It also is happening on 2 separate computers, so its likely not a browser issue past a common setting.

[Alibass]

I see you have vB Optimise and DBSEO... My guess is a conflict with one of these.

It is my understand DBSEO has added this feature (or will soon) - double check it's not trying to do the same thing already.

However now you have issues with the photography site a browser issue is possible too.

haven't changed anything

where did you see that
I got this link http://www.forcbodiesonly.com/mopar-forum/forum.php from another thread you posted in and when I went there yesterday I got that error, but today all is working with that link. Perhaps you or someone else was working on your site at the time of my visit.

[Alibass]

I see you have vB Optimise and DBSEO... My guess is a conflict with one of these.

It is my understand DBSEO has added this feature (or will soon) - double check it's not trying to do the same thing already.

However now you have issues with the photography site a browser issue is possible too.
BOP5

DBSEO has this feature and I am using it and your mod along with DBOptomise. I am not having any conflicts.

[BirdOPrey5]

I don't have either..... vB Optimise or DBSEO

Browser issue is possible, but I only have trouble with sites that have this mod installed. It also is happening on 2 separate computers, so its likely not a browser issue past a common setting.

My mistake, I thought Alibass's signature link was yours.

[Boothby]

When the error happens again, can you open the Developer Tools Network Tab (http://blogs.msdn.com/b/ie/archive/2010/04/22/ie9-developer-tools-network-tab.aspx) and make a screenshot of it?

[joeychgo]

When the error happens again, can you open the Developer Tools Network Tab (http://blogs.msdn.com/b/ie/archive/2010/04/22/ie9-developer-tools-network-tab.aspx) and make a screenshot of it?

Sure

[AusPhotography]

No its not...
I get this when I click LOG IN on your site

Hmm strange. I can't replicate that. I get a normal rejection screen.

Also: A link to content hidden from guests... http://www.ausphotography.net.au/forum/showthread.php?132199-ginkgo-tree-in-Launceston

I don't have either..... vB Optimise or DBSEO
We don't either. We do run DBOptomise.
We also have xCache.

[DemOnstar]

Not sure about this Joe.
I noticed a 60% drop in visitors since install.

A user informed me that they were unable to access the site and so I went along to see for myself.
They are getting error 500.

I normally have no problem accessing the site from where I stay but, I went along to the users place and I couldn't access and got the same 500 error. (misconfiguration on the server).

I disabled the mod and the error went away.

Tested in IE, Firefox, Chrome.

EDIT: the error went away for me but she still had the same error.
She is using a Samsung Galaxy device with Chrome as her browser.

She has not ever been able to access the site.

Weird one as I was in the same bar, I got the error, disabled the mod from another bar and gained access but she still cannot.

[BirdOPrey5]

Not sure about this Joe.
I noticed a 60% drop in visitors since install.

A user informed me that they were unable to access the site and so I went along to see for myself.
They are getting error 500.

I normally have no problem accessing the site from where I stay but, I went along to the users place and I couldn't access and got the same 500 error. (misconfiguration on the server).

I disabled the mod and the error went away.

Tested in IE, Firefox, Chrome.

EDIT: the error went away for me but she still had the same error.
She is using a Samsung Galaxy device with Chrome as her browser.

She has not ever been able to access the site.

Weird one as I was in the same bar, I got the error, disabled the mod from another bar and gained access but she still cannot.

A 500 is a server error... usually not from PHP. The good thing is that a 500 error should be logged- You should be able to get more info from the error log.

What is your PHP version?

[Boothby]

Sure
You should then reload the page to see the network traffic and the headers sent. ;)

[DemOnstar]

Apache version 2.2.27
PHP version 5.2.17
MySQL version 5.5.37-log

Not sure what is happening really. All a bit random. I get access from one place but not from another.

[BirdOPrey5]

Apache version 2.2.27
PHP version 5.2.17
MySQL version 5.5.37-log

Not sure what is happening really. All a bit random. I get access from one place but not from another.

Do you have cloudflare? They were having problems yesterday- unsure if they were solved- I'm guessing they were at some point.

[DemOnstar]

I don't have cloudflare.

It is wrong to blame this mod for this error as I am not entirely sure what the problem is yet.

[ezak]

not working for me , or I don't understand well
testing thread that is hidden from guest or it has been deleted
like this
http://www.gem-flash.com/vb/showthread.php?t=1
http://httpstatus.io/ said 404
also google webmaster said not found

I have DBoptimize, DBSEO, XCache all enabled

[BirdOPrey5]

not working for me , or I don't understand well
testing thread that is hidden from guest or it has been deleted
like this
http://www.gem-flash.com/vb/showthread.php?t=1
http://httpstatus.io/ said 404
also google webmaster said not found

I have DBoptimize, DBSEO, XCache all enabled

404 means it is working, at leas partially. If it wasn't working it would show 200.

The exact code it should show in this case is configurable in settings.

[ezak]

404 means it is working, at leas partially. If it wasn't working it would show 200.

The exact code it should show in this case is configurable in settings.

so to be useful from the mod
change all status to 200 ?

its realy working when change Generic Error HTTP Code to 200 , its wokring and send 200 status
what should I do then , change all to 200 ?
your site is sending 404 in your wrong thread

[BirdOPrey5]

If you disable the mod Google will get 200.

The purpose of this mod is to change 200 to 404 (or some other number.)

When google gets 200 but can'tr read the thread it's considered a "soft 404" error which is bad for SEO.

When google gets a 404 it knows it is a bad page and stops looking for it- it is less bad for SEO.

[ezak]

thanks for your reply
I want to know something about another error
vb/showthread.php?t=222053
this thread is hidden from guest
and from HTTP status got error 403

is that normal ? and good for seo ?

[BirdOPrey5]

403 is again from this mod and yes it is better Google sees a 403 than a 200 on a page they cannot read the real content of.

[Black Snow]

Could you add a rewrite rule to your .htaccess file that directs all 200 pages to 403 pages instead of insalling this mod? Was curious about what difference it would make.

[BirdOPrey5]

Could you add a rewrite rule to your .htaccess file that directs all 200 pages to 403 pages instead of insalling this mod? Was curious about what difference it would make.

All "200" pages would mean every page on your site- that would be worse than not doing anything at all.

[geovadim]

Great mod. Thanks.

[hilaryl]

Our PHP version is currently 5.2.11 - what can we do to effectively get the same result as this mod?

[BirdOPrey5]

It may work on PHP 5.2 but it works best on PHP 5.4 or higher. You should try it, I don't think there is any major difference between 5.3 and 5.2 and I know it works OK on 5.3.

[hilaryl]

It may work on PHP 5.2 but it works best on PHP 5.4 or higher. You should try it, I don't think there is any major difference between 5.3 and 5.2 and I know it works OK on 5.3.

Thanks.

I was also wondering - I only wanted to change the response header for the No Permission pages, and just leave the other pages for now.

Is there a way I can do this? With the plugin, or just by creating my own plugin for those particular pages?

Cheers

[BirdOPrey5]

Thanks.

I was also wondering - I only wanted to change the response header for the No Permission pages, and just leave the other pages for now.

Is there a way I can do this? With the plugin, or just by creating my own plugin for those particular pages?

Cheers

Easiest way would be to set the options you don't want to change to 200 in the mod settings. 200 is what they would be normally. Keep only the No Permission pages as 403/404 whatever you choose.

[hilaryl]

Just thought I would update to say that I have successfully implemented this plugin in vBulletin 4.1.9 - running PHP 5.2.11.

All response headers are returning as per the settings.

Cheers!

[hilaryl]

Hi BOP5,

I've added the plugin to our site and all is going well.

I've found that in addition to pages that require a user login to access particular areas, we also have pages where the administrator sets a specific password just for that forum.

With the plugin installed the pages header response code is returning a 404, where as it would be better if this was a 403 response, like the other login required pages.

Is that possible?

We only have a small amount of these pages, so it's not a big deal - just thought I would ask.

Thanks,
hilaryl

[Stratis]

Tested in PHP 5.2.17

Date:Fri, 15 May 2015 18:20:28
GMTServer:Apache
X-Powered-By:PleskLin
Cache-Control:private
Pragma:private
X-UA-Compatible:IE=Edge
X-PHP-Response-Code:404

I will see after some days with google, thank you very much, I appreciate this.

[BirdOPrey5]

I'm sure it's not impossible but it isn't something I can implement quickly. VB just uses the generic error call to display the request for the password so the response code will be whatever you have configured for generic errors, 404 by default.

To be honest very few people use forum passwords so I don't think I'll be adding functionality just for forum passwords.

[hilaryl]

I'm sure it's not impossible but it isn't something I can implement quickly. VB just uses the generic error call to display the request for the password so the response code will be whatever you have configured for generic errors, 404 by default.

To be honest very few people use forum passwords so I don't think I'll be adding functionality just for forum passwords.

Hi BOP5,

That's cool - I wasn't sure how that page was generated, but that makes sense as to why it's returning the 404.

Thanks for your quick reply.

Cheers

[ZeroHour]

We had a problem using this mod where pages were cached on the error code.
Adding
header("Cache-Control: no-cache, must-revalidate");
Before any BOP header() line which meant error pages are not cached.

[BirdOPrey5]

Thanks, I will add that to the next version. :up:

[Stratis]

Where to copy this part of code? :)

[Stratis]

A question please...

Every time i see in webmasters three errors 403 and the url like this.
attachment.php?attachmentid=2178&d=xxxxxx
attachment.php?attachmentid=4659&d=xxxxxx
attachment.php?attachmentid=1721&d=xxxxxx

It says Access Denied, as very little i understand this is what supposed to be. I always make the action that they are fixed and after one day same thing happens.

Why does google read these over and over, if it cannot access them, even in robots.txt i have Disallow: /attachment.php, even if i delete this Disallow, same thing happens, i waited 4 days for robots to take effect with out the Disallow, nothing changed.

Do i must No Permission HTTP Code change from 403 to something else?

Any help will be grateful, thanks

[BirdOPrey5]

I'm really not sure why Google does the things it does. I would think robots.txt should be preventing access completely. It seems like a question to ask on the Google support site. Sorry.

[ZeroHour]

I'm really not sure why Google does the things it does. I would think robots.txt should be preventing access completely. It seems like a question to ask on the Google support site. Sorry.

Google dont index robots.txt stuff but they will still follow links to areas covered by it, pretty sure thats what I read a few years ago.

[Stratis]

Ok thanks both for answers, i see the way to ask google.

[ekempter]

Getting 404 errors on tags.php?tag=<forum> for users who are not logged on. I'm guessing that this is because I have search blocked to the public. Is there a work around?

[SWSUSA]

Doesn't work.

[BirdOPrey5]

Getting 404 errors on tags.php?tag=<forum> for users who are not logged on. I'm guessing that this is because I have search blocked to the public. Is there a work around?

Not that I am aware. Are you saying this mod is causing the errors? If tag searched is blocked to the public a 404 seems like the appropriate response.

Doesn't work.

You'll need to be more specific if you want help, it's obviously working for most people.

[hilaryl]

We have had this mod installed for a good while now. The only concerning thing is the number of pages we have reported in Search Console as Access Denied.

This is the purpose of the mod - so nothing is wrong.

I'm interested though, is 403 response better than 401 response? Would 401 still report as Access Denied in Search Console?

[BirdOPrey5]

403 is forbidden, Google might think forbidden is better than 401 but I can't say for any certainty.

What I read was that 401 is preferred when the entire site should not be accessible and 403 is prefered when it's a specific page that is not accessible.

[hilaryl]

Hi BOP5 - I've been trying to find a definitive answer too.

Surely there is a right or wrong, not just a preferred response. Obviously neither 401 or 403 are wrong, but there must be some standard. It's not as if forum pages that require a login are a new thing on web.

I will keep searching, and hopefully find something definitive.

Cheers

[Stratis]

Joe, a question. I have changed my forum to https, do I must change in plugins the code (http) to (https) ?

Thank you

[BirdOPrey5]

Nothing that I can see should have to change for https.

[BadgerDog]

Installed (twice) for testing....

We use vBulletin 4.1.3 pl8...

Works great, except we have a conflict with Paul's "Display Unread Posts" mod v4.1.7.

https://vborg.vbsupport.ru/showthread.php?t=231060

When the HTTP mod is installed, Paul's mod shows the unread post count (top right corner of web site) as it should, however, when one clicks on the post number to view them, vBulletin shows page saying there are no unread posts, even though there are. You can click the "What's New" tab and the new posts show fine there.

Tried to diagnose as follows:

1. Turned HTTP mod OFF (still a conflict)
2. Disabled HTTP mod (still a conflict)
3. Uninstalled HTTP mod and Paul's "Display Unread Posts" mod v4.1.7 began to work fine again.
4. Re-insalled HTTP mod and problem returned.
5. Finally uninstalled HTTP mod and left things alone.

Hope this helps someone else .. :)

Regards,
Doug

[djbaxter]

Installed (twice) for testing....

We use vBulletin 4.1.3 pl8...

Works great, except we have a conflict with Paul's "Display Unread Posts" mod v4.1.7.

https://vborg.vbsupport.ru/showthread.php?t=231060

When the HTTP mod is installed, Paul's mod shows the unread post count (top right corner of web site) as it should, however, when one clicks on the post number to view them, vBulletin shows page saying there are no unread posts, even though there are. You can click the "What's New" tab and the new posts show fine there.

Tried to diagnose as follows:

1. Turned HTTP mod OFF (still a conflict)
2. Disabled HTTP mod (still a conflict)
3. Uninstalled HTTP mod and Paul's "Display Unread Posts" mod v4.1.7 began to work fine again.
4. Re-insalled HTTP mod and problem returned.
5. Finally uninstalled HTTP mod and left things alone.

Hope this helps someone else .. :)

Regards,
Doug

Thank you. I have a similar problem with at least one forum and was unable to solve it. I thought it might have something to do with post counts not synching unread posts correctly. I am going to try your solution.

BOP5: If it is confirmed as a conflict between the two add-ons, can it be resolved?

[BirdOPrey5]

Installed (twice) for testing....

We use vBulletin 4.1.3 pl8...

Works great, except we have a conflict with Paul's "Display Unread Posts" mod v4.1.7.

https://vborg.vbsupport.ru/showthread.php?t=231060

When the HTTP mod is installed, Paul's mod shows the unread post count (top right corner of web site) as it should, however, when one clicks on the post number to view them, vBulletin shows page saying there are no unread posts, even though there are. You can click the "What's New" tab and the new posts show fine there.

Tried to diagnose as follows:

1. Turned HTTP mod OFF (still a conflict)
2. Disabled HTTP mod (still a conflict)
3. Uninstalled HTTP mod and Paul's "Display Unread Posts" mod v4.1.7 began to work fine again.
4. Re-insalled HTTP mod and problem returned.
5. Finally uninstalled HTTP mod and left things alone.

Hope this helps someone else .. :)

Regards,
Doug

I'm sorry but that makes no sense, if the mod is disabled from Product Manager it is basically the same as being uninstalled, there are no files to upload. If you need to remove the mod for Paul's mod to work I suspect any mod removed would fix it, it may be some sort of product cache issue.

In any case only the 4.2.0 version of Paul's mod is available for download at this time. Looking at the code for that I don't see any places they would conflict.

[BadgerDog]

I'm sorry but that makes no sense, if the mod is disabled from Product Manager it is basically the same as being uninstalled, there are no files to upload. If you need to remove the mod for Paul's mod to work I suspect any mod removed would fix it, it may be some sort of product cache issue.

In any case only the 4.2.0 version of Paul's mod is available for download at this time. Looking at the code for that I don't see any places they would conflict.

The sequence is repeatable and I've installed and uninstalled the HTTP mode 3 times to validate the steps and the same results occur every time. I have left the HTTP mod uninstalled and moved on, but thank you for your efforts.... much appreciated ...

I do use Sucuri's firewall protection system, which caches our site around the world, so perhaps it's having an affect only with this mod, although I don't know why.

https://sucuri.net/

Thanks again.... :)

Regards,
Doug