Mysql problem, any help is appreciated.

[Deimos]

I'm not sure where to look, but intermittently, mysql either crashes or stops working
The only time I know when something is wrong is when I start getting emails about it being down

Is there a paticular log file I need to check to find out what happened?

This is from the hostname.err log file

Version: '5.0.27-standard' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Community Edition - Standard (GPL)
071203 17:59:38 [ERROR] Can't create thread to kill server

Number of processes running now: 0
071203 18:05:04 mysqld restarted
071203 18:05:04 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: using the same InnoDB data or log files.
071203 18:05:48 [Note] /usr/sbin/mysqld: Normal shutdown

InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
071203 18:05:50 InnoDB: Starting shutdown...
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
InnoDB: Unable to lock ./ibdata1, error: 11
InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
071203 18:05:55 InnoDB: Shutdown completed; log sequence number 0 43063192
071203 18:05:55 [Note] /usr/sbin/mysqld: Shutdown complete

071203 18:05:56 InnoDB: Started; log sequence number 0 43063192
071203 18:05:56 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.0.27-standard' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Community Edition - Standard (GPL)
071203 18:05:56 [Note] /usr/sbin/mysqld: Normal shutdown

071203 18:05:56 InnoDB: Starting shutdown...

Number of processes running now: 0
071203 18:05:56 mysqld restarted
071203 18:05:56 InnoDB: Started; log sequence number 0 43063192
071203 18:05:56 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.0.27-standard' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Community Edition - Standard (GPL)
A mysqld process already exists at Mon Dec 3 18:05:56 EST 2007

[Paul M]

What do the error e-mails actually say ?

[Deimos]

The first email I get, is this

Database error in vBulletin :

mysql_connect() [<a href='function.mysql-connect'>function.mysql-connect</a>]: Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (11)
/home/xxxx/public_html/includes/class_core.php on line 274

MySQL Error :
Error Number :
Date : Monday, December 3rd 2007 @ 06:04:15 PM
Script : http://www.uoforums.com/
Referrer :
IP Address : 87.194.xxx.xx
Username :
Classname : vB_Database

[Marco van Herwaarden]

This error means either:
1. The info in your config.php file is wrong (in which case your forums wouldn't work at all), or
2. MySQL crashed, it's not running or it can't find the socket. You need to contact your host about this. Here is more info on this error:
http://dev.mysql.com/doc/mysql/en/Can_not_connect_to_server.html

[Eikinskjaldi]

This is likely a memory problem Either you are actually running out of memory, or you are hitting a system limit such as open files. If you are running on a shared server you should look to changing hosts.

run ulimit -a to find out how many open files you are allowed
run the mysql commands show status and show variables to see if you are hitting any limits.

[Deimos]

I'm running on a VPS with 768mb of reserved system memory

This is the result of ulimit


core file size (blocks, -c) 1000000
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
pending signals (-i) 1024
max locked memory (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files (-n) 4096
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 14335
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

[Eikinskjaldi]

Your system looks fine. What does mysql report? what are the values for open files in show status and show variables?

Also, since you are on a vps you should be able to look at the bean counters, which will live under /proc somewhere. These will show an instant snap shot of your system resources.