VB bringing down a whole dedicated server?

[nando99]

We run an active VB forum, we're hosted on a hivelocity.com dedicated server... The server has been crashing lately and the deeper we dig into the problem, the more it looks like its the Vbulletin script... now, I don't know if its a bug, a plugin, a messed up server configuration or what...

Here are some error messages I get

Device Manager has determined that Intel Xeon Sandy Bridge E3-1230 3.2 (9HD3)
has failed test Ping requests on address 199.xxx.xxx.xx.
Packet loss was 100%.

Mar  3 11:36:00 server kernel: Killed process 14045, UID 507, (php)
total-vm:231396kB, anon-rss:6076kB, file-rss:27836kB
Mar  3 11:36:00 server kernel: [<ffffffff81158b1a>] ?
alloc_pages_vma+0x9a/0x150
Mar  3 11:36:00 server kernel: [<ffffffff8113f08e>] ?
remove_vma+0x6e/0x90
Mar  3 11:38:36 server kernel: [ pid ]  uid  tgid total_vm      rss cpu
oom_adj oom_score_adj name
Mar  3 11:38:36 server kernel: Killed process 15334, UID 507, (php)
total-vm:228120kB, anon-rss:9008kB, file-rss:49472kB
Mar  3 11:38:36 server kernel: [<ffffffff81158b1a>] ?
alloc_pages_vma+0x9a/0x150

tail /var/log/messages
Mar  4 18:34:57 server suhosin[24138]: ALERT - script tried to increase
memory_limit to 134217728 bytes which is above the allowed value (attacker
'66.249.71.10', file
'/home/yeouschc/public_html/community/includes/class_xml.php', line 37)
Mar  4 18:34:58 server suhosin[24141]: ALERT - script tried to increase
memory_limit to 134217728 bytes which is above the allowed value (attacker
'76.109.252.163', file
'/home/yeouschc/public_html/community/includes/class_xml.php', line 37)
Mar  4 18:34:58 server suhosin[24143]: ALERT - configured request variable name
length limit exceeded - dropped variable
'40515-I-finally-pulled-off-my-silenced-spas-12-MOAB-54-15-rushing-specialist'
(attacker '66.249.71.10', file
'/home/yeouschc/public_html/community/showthread.php')
Mar  4 18:34:58 server suhosin[24143]: ALERT - script tried to increase
memory_limit to 134217728 bytes which is above the allowed value (attacker
'66.249.71.10', file
'/home/yeouschc/public_html/community/includes/class_xml.php', line 37)
Mar  4 18:34:59 server suhosin[24145]: ALERT - script tried to increase
memory_limit to 134217728 bytes which is above the allowed value (attacker
'66.249.71.10', file
'/home/yeouschc/public_html/community/includes/class_xml.php', line 37)
Mar  4 18:35:01 server suhosin[24164]: ALERT - script tried to increase
memory_limit to 134217728 bytes which is above the allowed value (attacker
'98.116.66.199', file
'/home/yeouschc/public_html/community/includes/class_xml.php', line 37)
Mar  4 18:35:01 server suhosin[24166]: ALERT - script tried to increase
memory_limit to 134217728 bytes which is above the allowed value (attacker
'91.95.248.2', file
'/home/yeouschc/public_html/community/includes/class_xml.php', line 37)
Mar  4 18:35:02 server suhosin[24168]: ALERT - script tried to increase
memory_limit to 134217728 bytes which is above the allowed value (attacker
'129.82.65.243', file
'/home/yeouschc/public_html/community/includes/class_xml.php', line 37)
Mar  4 18:35:02 server suhosin[24170]: ALERT - script tried to increase
memory_limit to 134217728 bytes which is above the allowed value (attacker
'71.142.131.78', file
'/home/yeouschc/public_html/community/includes/class_xml.php', line 37)
Mar  4 18:35:02 server suhosin[24172]: ALERT - script tried to increase
memory_limit to 134217728 bytes which is above the allowed value (attacker
'66.75.63.204', file
'/home/yeouschc/public_html/community/includes/class_xml.php', line 37)
[root@server ~]#

Is it my rss feeds? I'm at a total lost... Anyone have any idea? Thanks a million in advance!

[DivisionByZero]

disable php-suhosin, increase your php memory limit to 256M, and upgrade to the latest PHP.

[setishock]

The word attacker followed by an IP sure got my attention. I washed a couple of them through some IP lookup sites and got they are from Sweden. Could be as simple as too many connections at one time and the system is crashing. Check your traffic logs.

[nando99]

wouldnt disabling php-suhosin decrease the php security?

[whitedd]

Originally Posted by nando99

wouldnt disabling php-suhosin decrease the php security?

no

...use mod-security....

[nando99]

Originally Posted by whitedd

no

...use mod-security....

do u use any specific modsecurity rules?

[DivisionByZero]

Originally Posted by nando99

wouldnt disabling php-suhosin decrease the php security?

Not unless you're hosting some real shoddy code. The only way PHP can go rogue is through a script that is parsed by the PHP interpreter.

[nando99]

well, i've increased the limit to 256, updated to the latest version of php and disabled php-suhosin... i'm also using mod-security with the default configuration plus this additional code for ddos prevention.. thoughts?

SecRuleEngine On

SecAuditEngine RelevantOnly
SecAuditLogType Serial
SecAuditLog logs/mod_security.log

# a folder where mod_security will store data variables
SecDataDir logs/mod_security-data

# ignore requests from localhost or some other IP
SecRule REMOTE_ADDR "^127\.0\.0\.1$" "phase:1,nolog,allow"

# for all non static urls count requests per second per ip
# (increase var requests by one, expires in 1 second)
SecRule REQUEST_BASENAME "!(\.avi$|\.bmp$|\.css$|\.doc$|\.flv$|\.gif$|\
                            \.htm$|\.html$|\.ico$|\.jpg$|\.js$|\.mp3$|\
                            \.mpeg$|\.pdf$|\.png$|\.pps$|\.ppt$|\.swf$|\
                            \.txt$|\.wmv$|\.xls$|\.xml$|\.zip$)"\
                            "phase:1,nolog,pass,initcol:ip=%{REMOTE_ADDR},setvar:ip.requests=+1,expirevar:ip.requests=1"

# if there where more than 5 requests per second for this IP
# set var block to 1 (expires in 5 seconds) and increase var blocks by one (expires in an hour)
SecRule ip:requests "@eq 5" "phase:1,pass,nolog,setvar:ip.block=1,expirevar:ip.block=5,setvar:ip.blocks=+1,expirevar:ip.blocks=3600"

# if user was blocked more than 5 times (var blocks>5), log and return http 403
SecRule ip:blocks "@ge 5" "phase:1,deny,log,logdata:'req/sec: %{ip.requests}, blocks: %{ip.blocks}',status:403"

# if user is blocked (var block=1), log and return http 403
SecRule ip:block "@eq 1" "phase:1,deny,log,logdata:'req/sec: %{ip.requests}, blocks: %{ip.blocks}',status:403"

# 403 is some static page or message
ErrorDocument 403 "<center><h2>Go away..."