vbulletin is killing my server! :(

[Areku]

HI there,

I'm getting mad...

Vbulletin seems to be working fine. Right now I'm running 2.3.0 and its a bit hacked but they're quite "standard" hacks, like quick reply, pm popup box, vbarcade and member photo.

Server is a dedicated dual Xeon with 1Gb Ram. Im running latest php and mysql versions.

I've seen traffic of an average of 500 simultanous users and microstats keep displaying a server load average of 0.85...

However, sometimes, all of a sudden, something makes it go mad and server reaches loads of 170.x and up, bringing apache and mysql down...

HOW can I identify the problem? First I thought it was vbportal, but i replaced it with vbhome lite and problem persists...

Help PLEASE!

[Zachery]

HI there,

I'm getting mad...

Vbulletin seems to be working fine. Right now I'm running 2.3.0 and its a bit hacked but they're quite "standard" hacks, like quick reply, pm popup box, vbarcade and member photo.

Server is a dedicated dual Xeon with 1Gb Ram. Im running latest php and mysql versions.

I've seen traffic of an average of 500 simultanous users and microstats keep displaying a server load average of 0.85...

However, sometimes, all of a sudden, something makes it go mad and server reaches loads of 170.x and up, bringing apache and mysql down...


HOW can I identify the problem? First I thought it was vbportal, but i replaced it with vbhome lite and problem persists...

Help PLEASE!
well i can suggest switching to vbulletin 3.0 as i know it has decreased server load in many instances,

you should go to vbulletin.com and talk to eva2000 and he will help you tweak the mysql settings :)

[Areku]

But this is not an issue of "vb 2.xx being overloader than vb 3.xx"...

When it works, it works fine, keeping server low (0.8x), but there's something making it hit the 170.xx...

[Link14716]

How many games are in your arcade?

[Areku]

14

[Zachery]

But this is not an issue of "vb 2.xx being overloader than vb 3.xx"...

When it works, it works fine, keeping server low (0.8x), but there's something making it hit the 170.xx...
overall vB3 is less server intensive than vB2 i still suggest talking to eva2000 in the server configuration area thought :)

[Link14716]

Even though 14 arcade games aren't a lot, a major problem with the arcade is the number of queries skyrocket when you add more games. I believe that it might be causing the chaos.

[Areku]

I think server is tweaked VBoriented, and I'll talk to eva2000, but the problem is clearly temporary and it occurrs may be once or twice a day, being days with no problems at all...

I don't think arcade is the problem either, mine is not a game-oriented forum, and I think arcade is the least traffiqued area of my site...

[Zachery]

I think server is tweaked VBoriented, and I'll talk to eva2000, but the problem is clearly temporary and it occurrs may be once or twice a day, being days with no problems at all...

I don't think arcade is the problem either, mine is not a game-oriented forum, and I think arcade is the least traffiqued area of my site...
do you have mysql 4.0.17 with querry cacheing enabled?

[Areku]

For example, right now, 517 online users, 11 playing the arcade. Server load: 1.40

[Areku]

Faranth were do I check that?

MySQL is v3.23.58
PHP v4.3.3

[Zachery]

Faranth were do I check that?

MySQL is v3.23.58
PHP v4.3.3
im purdy sure its only aviable in the 4.x serries

[Areku]

Isn't there any sort of "server load vs scripts" logger?

Ie. before load gets higher than xx, i get a log of what caused it?

[Areku]

faranth,

would a mysql upgrade + enabling cache save us?

[Link14716]

I'm still putting my money on the arcade hack being the culprit.

[Zachery]

faranth,

would a mysql upgrade + enabling cache save us?
it might, but im not mysql expert :) thats why its always best to ask eva :)

[NTLDR]

Yep, query caching is only available in MySQL 4.x, I'd recommend upgrading to that for starters :) I'd also recommend installing MMCache, Erwin wrote a tutorial over at vB.com for it.

[Areku]

It seems I can't upgrade as server works with Ensim... any other tips?

[Erwin]

What does MRTG (or whatever you use to monitor stats) say regarding when the load is highest?

[eva2000]

HI there,

I'm getting mad...

Vbulletin seems to be working fine. Right now I'm running 2.3.0 and its a bit hacked but they're quite "standard" hacks, like quick reply, pm popup box, vbarcade and member photo.

Server is a dedicated dual Xeon with 1Gb Ram. Im running latest php and mysql versions.

I've seen traffic of an average of 500 simultanous users and microstats keep displaying a server load average of 0.85...

However, sometimes, all of a sudden, something makes it go mad and server reaches loads of 170.x and up, bringing apache and mysql down...


HOW can I identify the problem? First I thought it was vbportal, but i replaced it with vbhome lite and problem persists...

Help PLEASE!
go to server config forum at vbulletin.com and post a thread with this required info http://www.vbulletin.com/forum/showthread.php?t=70117 :)

[Areku]

eva2000, done.

Erwin, which kind of info exactly?

I've been told the server goes down due to a LOOP caused by a PHP/MYSQL call...

Does any1 know of any hack that would cause this/Has caused this in the past with VB 2.x series?

[Zachery]

id guess its squalls call on the arcade hack

[Areku]

Ok i'll try to disable this and see if server keeps alive...

[g-force2k2]

Use vbulletin's debug mode to display the query load how many are being made and how long each takes. That way you can get a closer fix on what the problem is.

Regards,
g-force2k2

[Areku]

I can now say it's not due to VBPRO ARCADE.... Server went down twice after turning the arcade off...

I already have microstats enabled and I monitor both the queries and server load from there...

I've seen VB working with PRO ARCADE enabled with 500 users online and server load be 0.85, so I DON'T THINK it's VB itself, but some sort of loop (as adviced by the server administration dept.) that it's causing the server overload (once it starts going up, load won't stop until apache and mysql go down).

Problem is mainly identifying the source of the loop...
:(

[Erwin]

I can now say it's not due to VBPRO ARCADE.... Server went down twice after turning the arcade off...

I already have microstats enabled and I monitor both the queries and server load from there...

I've seen VB working with PRO ARCADE enabled with 500 users online and server load be 0.85, so I DON'T THINK it's VB itself, but some sort of loop (as adviced by the server administration dept.) that it's causing the server overload (once it starts going up, load won't stop until apache and mysql go down).

Problem is mainly identifying the source of the loop...
:(
It might not be vB - it might be the way your server is set up, or a cron job. Does this happen at the same time each day?

[Areku]

It's a standard Rackshack (now EV1) dual xeon server, quite "normal" afai understand...

And no, it's not a cron job.

[Erwin]

It's a standard Rackshack (now EV1) dual xeon server, quite "normal" afai understand...

And no, it's not a cron job.
I have the same server. :) I have an ev1 dual xeon with 4Mb RAM. How busy is your server? My loads were fine on vB2. Do you need more RAM?

[Areku]

4Mb Ram? wow! ;)

I just have 1Gb RAM...

But I'm afraid whatever the bug is, it would consume the whole thing no matter how much resources I add, as it seems the load is only stopped once services (apache/sql) fail.....

And the server is NORMALLY not that busy, which means that in normal conditions, it averages 0.8x to 1.x load averages... Far from the 4.x maximum permitted...

[Brad]

4Mb Ram? wow! ;)

I just have 1Gb RAM...

But I'm afraid whatever the bug is, it would consume the whole thing no matter how much resources I add, as it seems the load is only stopped once services (apache/sql) fail.....

And the server is NORMALLY not that busy, which means that in normal conditions, it averages 0.8x to 1.x load averages... Far from the 4.x maximum permitted...
What other hacks do you have installed?, you need to find this loop and kill it/optimize it.

[cirisme]

It seems I can't upgrade as server works with Ensim... any other tips? I had no trouble with using MySQl 4 + Turck MMCache with Ensim :)

It sounds like possibly a cron script that hangs. It might also be a form of a dos attack... we had this a while back and it took 3 weeks to find out why we had daily outages for no apparent reason.

When this happens, what does top say?

[eva2000]

It's a standard Rackshack (now EV1) dual xeon server, quite "normal" afai understand...

And no, it's not a cron job.
nah ya problem is your apache/mysql configuration settings.. as stated at vbulletin.com support forums :)