记一次mysql的异常

配置腾讯云1G1核的云服务器。

service mysqld start怎么都起不来，

然后手动执行mysql启动命令 /usr/sbin/mysqld --user=mysql

标准输出如下：

[root@VM_0_8_centos lib]# /usr/sbin/mysqld --user=mysql
2019-08-20 21:16:36 0 [Note] /usr/sbin/mysqld (mysqld 5.6.42) starting as process 19322 ...
2019-08-20 21:16:36 19322 [Note] Plugin 'FEDERATED' is disabled.
2019-08-20 21:16:36 19322 [Note] InnoDB: Using atomics to ref count buffer pool pages
2019-08-20 21:16:36 19322 [Note] InnoDB: The InnoDB memory heap is disabled
2019-08-20 21:16:36 19322 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-08-20 21:16:36 19322 [Note] InnoDB: Memory barrier is not used
2019-08-20 21:16:36 19322 [Note] InnoDB: Compressed tables use zlib 1.2.11
2019-08-20 21:16:36 19322 [Note] InnoDB: Using Linux native AIO
2019-08-20 21:16:36 19322 [Note] InnoDB: Using CPU crc32 instructions
2019-08-20 21:16:36 19322 [Note] InnoDB: Initializing buffer pool, size = 50.0M
2019-08-20 21:16:36 19322 [Note] InnoDB: Completed initialization of buffer pool
2019-08-20 21:16:36 19322 [Note] InnoDB: Highest supported file format is Barracuda.
2019-08-20 21:16:36 19322 [Note] InnoDB: The log sequence numbers 103542583 and 103542583 in ibdata files do not match the log sequence number 103543430 in the ib_logfiles!
2019-08-20 21:16:36 19322 [Note] InnoDB: Database was not shutdown normally!
2019-08-20 21:16:36 19322 [Note] InnoDB: Starting crash recovery.
2019-08-20 21:16:36 19322 [Note] InnoDB: Reading tablespace information from the .ibd files...
2019-08-20 21:16:36 19322 [Note] InnoDB: Restoring possible half-written data pages
2019-08-20 21:16:36 19322 [Note] InnoDB: from the doublewrite buffer...
2019-08-20 21:16:37 19322 [Note] InnoDB: 128 rollback segment(s) are active.
2019-08-20 21:16:37 19322 [Note] InnoDB: Waiting for purge to start
2019-08-20 21:16:37 19322 [Note] InnoDB: 5.6.42 started; log sequence number 103543430
2019-08-20 21:16:37 19322 [Note] Server hostname (bind-address): '*'; port: 3306
2019-08-20 21:16:37 19322 [Note] IPv6 is available.
2019-08-20 21:16:37 19322 [Note]   - '::' resolves to '::';
2019-08-20 21:16:37 19322 [Note] Server socket created on IP: '::'.
2019-08-20 21:16:37 19322 [Note] Event Scheduler: Loaded 0 events
2019-08-20 21:16:37 19322 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.6.42'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL)

halt在这儿了，啥事儿没有，但是只要我请求一个我wordpress的网页接口，就会多一个『已杀死』：

[root@VM_0_8_centos lib]# /usr/sbin/mysqld --user=mysql
2019-08-20 21:16:36 0 [Note] /usr/sbin/mysqld (mysqld 5.6.42) starting as process 19322 ...
2019-08-20 21:16:36 19322 [Note] Plugin 'FEDERATED' is disabled.
2019-08-20 21:16:36 19322 [Note] InnoDB: Using atomics to ref count buffer pool pages
2019-08-20 21:16:36 19322 [Note] InnoDB: The InnoDB memory heap is disabled
2019-08-20 21:16:36 19322 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-08-20 21:16:36 19322 [Note] InnoDB: Memory barrier is not used
2019-08-20 21:16:36 19322 [Note] InnoDB: Compressed tables use zlib 1.2.11
2019-08-20 21:16:36 19322 [Note] InnoDB: Using Linux native AIO
2019-08-20 21:16:36 19322 [Note] InnoDB: Using CPU crc32 instructions
2019-08-20 21:16:36 19322 [Note] InnoDB: Initializing buffer pool, size = 50.0M
2019-08-20 21:16:36 19322 [Note] InnoDB: Completed initialization of buffer pool
2019-08-20 21:16:36 19322 [Note] InnoDB: Highest supported file format is Barracuda.
2019-08-20 21:16:36 19322 [Note] InnoDB: The log sequence numbers 103542583 and 103542583 in ibdata files do not match the log sequence number 103543430 in the ib_logfiles!
2019-08-20 21:16:36 19322 [Note] InnoDB: Database was not shutdown normally!
2019-08-20 21:16:36 19322 [Note] InnoDB: Starting crash recovery.
2019-08-20 21:16:36 19322 [Note] InnoDB: Reading tablespace information from the .ibd files...
2019-08-20 21:16:36 19322 [Note] InnoDB: Restoring possible half-written data pages
2019-08-20 21:16:36 19322 [Note] InnoDB: from the doublewrite buffer...
2019-08-20 21:16:37 19322 [Note] InnoDB: 128 rollback segment(s) are active.
2019-08-20 21:16:37 19322 [Note] InnoDB: Waiting for purge to start
2019-08-20 21:16:37 19322 [Note] InnoDB: 5.6.42 started; log sequence number 103543430
2019-08-20 21:16:37 19322 [Note] Server hostname (bind-address): '*'; port: 3306
2019-08-20 21:16:37 19322 [Note] IPv6 is available.
2019-08-20 21:16:37 19322 [Note]   - '::' resolves to '::';
2019-08-20 21:16:37 19322 [Note] Server socket created on IP: '::'.
2019-08-20 21:16:37 19322 [Note] Event Scheduler: Loaded 0 events
2019-08-20 21:16:37 19322 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.6.42'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL)
已杀死

为什么会这样？一开始以为是因为这个：

2019-08-20 21:16:36 19322 [Note] InnoDB: The log sequence numbers 103542583 and 103542583 in ibdata files do not match the log sequence number 103543430 in the ib_logfiles!

以为是因为ibdata和ib_logfiles的lsn序号不一致导致的问题，后来提工单工程师说这个不会导致mysql服务不可用。

然后搜到这篇文章最下面，建议看下/var/log/messages有什么：

[root@VM_0_8_centos scripts]# tailf /var/log/messages

Aug 20 21:16:47 VM_0_8_centos kernel: [19283]     0 19283    40493      173      38        0             0 top
Aug 20 21:16:47 VM_0_8_centos kernel: [19322]    27 19322   315195   112859     274        0             0 mysqld
Aug 20 21:16:47 VM_0_8_centos kernel: [19349]     0 19349    30342      332      65        0             0 sshd
Aug 20 21:16:47 VM_0_8_centos kernel: [19365]     0 19365    28286       95      11        0             0 sh
Aug 20 21:16:47 VM_0_8_centos kernel: Out of memory: Kill process 19322 (mysqld) score 445 or sacrifice child
Aug 20 21:16:47 VM_0_8_centos kernel: Killed process 19322 (mysqld) total-vm:1260780kB, anon-rss:451436kB, file-rss:0kB, shmem-rss:0kB
Aug 20 21:17:02 VM_0_8_centos systemd: Started Session 11858 of user root.
Aug 20 21:17:02 VM_0_8_centos systemd: Starting Session 11858 of user root.
Aug 20 21:18:01 VM_0_8_centos systemd: Started Session 11859 of user root.
Aug 20 21:18:01 VM_0_8_centos systemd: Starting Session 11859 of user root.

看到有一行out of memory和mysqld，难道是内存不够用了？？？

于是我把php的关于进程数的参数调低些如下：

pm.max_children = 10   #以前是50

pm.start_servers = 5

pm.min_spare_servers = 5

pm.max_spare_servers = 10  #以前是35

然后打开三个窗口，分别监控：

一个监控系统日志文件：tailf /var/log/messages

一个监控系统内存变化：watch -n 1 "free -h"

一个启动mysql服务：/usr/sbin/mysqld --user=mysql

果然再请求wordpress网页后，mysql也不会『已杀死』了。

问题解决了，但是内存还是太小了，可用的只剩100多M：

              total        used        free	 shared  buff/cache   available
Mem:           991M        657M         75M         35M        258M        130M
Swap:            0B          0B          0B

还是升级设备吧。