Weird Problem ... apache? database connection?

This is a thread about what has been going on recently with me

http://www.vbulletin.com/forum/showthread.php?t=207240

Last changes:
11/12/06 installed media wiki
11/11/06 upgraded to vB 3.6.3
11/11/06 upgraded DB Box to FC6 (mysql 5.0.22)
11/11/06 upgraded DB Box RAID firmware
11/11/06 upgraded DB Box mother board bios
11/10/06 mysql restarted (oom-killer)
11/9/06 DB box crashed (RAID / ACCRAID IRQ hard crash)
11/7/06 installed APC Master Switch (remote reboot) restarted both boxes
11/5/06 DB box crashed (RAID / ACCRAID IRQ hard crash)
11/6/06 mysql restarted (oom-killer)
11/3/06 upgraded web box to FC6 (apache 2.2.3/php 5.1.8)
11/1/06 DB box crashed (RAID / ACCRAID IRQ hard crash)
10/28/06 DB box went live

This morning something new happened....three times now it just blanked out and couldnt see the DB. Mysql has been running now for almost 69 hours. Oom-killer has not restarted it. The loads on the web server on average are 0.5 to 1.0 ... starting this morning it bumps up a little to 4 once in awhile. Twice the load has hit 40's and I get a few seconds worth of database error emails. Once I got 50 or so db error messages but the load did not go up. Thru out this the DB box load has not gotten above 0.25.

mysql_connect() [<a href='function.mysql-connect'>function.mysql-connect</a>]: Can't connect to MySQL server on '172.27.0.1' (4) /var/www/html/forums/includes/class_core.php on line 273
MySQL Error : 
Error Number : 
Date : Monday, November 13th 2006 @ 10:55:18 AM
Script : http://*****.com/forums/
Referrer : 
IP Address : 139.62.32.200
Username : 
Classname : vB_Database

I am fairly confused as to why this is happening and I am not really thrilled that it happened 3 times in the past 3 hours. It was working perfectly since I left the DC almost 48 hours ago.... the only things I changed is the vBulletin version and added media wiki.

Occassionally the web box goes up to 10 or so for the load .... I have never seen it do that before.... almost like someone is trying to bombard apache every 45 min or so.

[DevilYellow]

<a href="http://www.vbulletin.com/forum/showthread.php?t=210840" target="_blank">http://www.vbulletin.com/forum/showthread.php?t=210840</a>

Seems like something to do with moderator permissions is being called for each thread of the search results. See post 33.

While there are a number of large boards running 3.6 I really hope that my php memory requirements (and to some extent the reason for my load is high) have something to do with this.

Oh, and an update on my situation:
DB Box running 100% strong for 30 days now.
Web Box - no matter what it will overload once in awhile and usually a big over load once every few days. New memory in the box has yet to yield any error.

Have yet to try anything else - but besides RAID configuration and OS/LAMP versions there is nothing else too try.

[orban]

web box..does it swap? and try lighttpd, works so good for me.

[DevilYellow]

Not getting into the swap. I have 4GB in it - going to 6GB soon.

I just dont understand why apache is running this terrible.

How easy is it install, config, and test lighttpd with apache still installed?

Can they use the same directory? I am hoping it would be as easy as install, stop apache, start lighttpd and thats it.

[orban]

Yes I did that. You have to have a PHP with both Apache support + fgci tho before you can switch (unless you run Apache with fgci)

[DevilYellow]

Not sure, how would I check that option for PHP?

[orban]

phpinfo() will tell how it was compiled

[jason|xoxide]

Originally Posted by DevilYellow

How easy is it install, config, and test lighttpd with apache still installed?

You might want to try Litespeed instead, it's closer to a drop-in replacement for httpd than lighttpd.

[DevilYellow]

litespeed seems pretty cool. I had problems if I had it use the current apache config. It wouldn’t let me change anything - so if I try it again I wont use that option.

I didn’t want to shut down apache to start it just incase there was a problem. I was trying to get it to go on a different port. I may try it again soon. As long as the board would function then it would be great - just need to make sure it logged the same way for webalizer.

What is really starting to piss me off is that apache is still unresponsive 1-4 times a day. Nothing at all in the error_log ... the access_log does not look funny. There is nothing else I can find. I have disabled plugins, tried every combination for the httpd.conf, tried various memory levels for PHP, fixed an error in my hosts file, tried my best to re-create it..... it just doesn’t have any rhyme or reason. Usually once in the morning when people start getting on again - but its not load sensitive. I could have the busiest night and it wouldn’t miss a beat. So why is it having problems at half the load?

I refuse to think its just the fact that its "fedora" - and it scares me that if I do anything with RHEL or CentOS because they are all close I could have the same problem.

With out errors its hard to diagnose

Oh, and before upgrading to 3.6.4 the same site was 100% rock solid running 3.5.1 on FC3 with Apache 2.0.52 and PHP 4.3.9

We may have resolution - http://www.vbulletin.com/forum/showp...8&postcount=28