mysql实例无法启动故障修复

一、问题描述

    环境：Linux CentOS 6.5，mysql 5.6.16-log

    Linux服务器异常断电，mysql非正常退出，导致mysql数据和mysql日志出现不一致的情况，mysql启动失败。失败的现象：

1）mysql启动失败，提示无法更新pid文件。

2）查看mysql error日志，出现如下字样：

InnoDB: Error: page xxx log sequence number xxx

InnoDB: is in the future! Current system log sequence number xxx，其中 xxx为一串数字。

二、解决办法

需要强制恢复，启动成功后立即进行数据恢复。

三、解决步骤

1、查找mysql的配置文件my.cnf，/etc/my.cnf有最优先的级别，如果该目录下没有该文件，则从安装目录下寻找。

2、修改my.cnf，在[mysqld]下添加：

innodb_force_recory=1

修复模式的级别从1到6，具体的含义建议查官网资料（文章尾部有节选补充），设置一个级别尝试启动mysql，一直到能启动为止，实际我一直设置到6，mysql才能启动，从而证实是断电造成数据与日志不一至导致的问题。

3、数据导出

mysql启动状态下，备份数据文件

1）/mysql/data/ibdata1，ib_logfile0，ib_logfile1备份

2）mysql -uroot -p --all-databases > all-databases.sql，可以导出所有的数据文件。

3）/mysql/data/目录下，留下mysql，performance_schema，auto.cnf三个文件或文件夹，其余全部干掉。

4）重启数据库，这时发现之前创建的数据库都没有了，ibdata1，ib_logfile0，ib_logfile1这三个文件重新刷新了。

5）将导出的all.databases.sql文件重新导入

mysql -uroot -p < all-databases.sql

6）再重启一次数据库

四、资料补充

官网资料对innodb_force_recory的解释：

https://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html

摘抄了一部分：

As a safety measure, InnoDB prevents INSERT, UPDATE, or DELETE operations when innodb_force_recovery is greater than 0. As of MySQL 5.6.15, an innodb_force_recovery setting of 4 or greater places InnoDB in read-only mode.
1 (SRV_FORCE_IGNORE_CORRUPT)
Lets the server run even if it detects a corrupt page. Tries to make SELECT * FROM tbl_name jump over corrupt index records and pages, which helps in dumping tables.
2 (SRV_FORCE_NO_BACKGROUND)
Prevents the master thread and any purge threads from running. If a crash would occur during the purge operation, this recovery value prevents it.
3 (SRV_FORCE_NO_TRX_UNDO)
Does not run transaction rollbacks after crash recovery.
4 (SRV_FORCE_NO_IBUF_MERGE)
Prevents insert buffer merge operations. If they would cause a crash, does not do them. Does not calculate table statistics. This value can permanently corrupt data files. After using this value, be prepared to drop and recreate all secondary indexes. As of MySQL 5.6.15, sets InnoDB to read-only.
5 (SRV_FORCE_NO_UNDO_LOG_SCAN)
Does not look at undo logs when starting the database: InnoDB treats even incomplete transactions as committed. This value can permanently corrupt data files. As of MySQL 5.6.15, sets InnoDB to read-only.
6 (SRV_FORCE_NO_LOG_REDO)
Does not do the redo log roll-forward in connection with recovery. This value can permanently corrupt data files. Leaves database pages in an obsolete state, which in turn may introduce more corruption into B-trees and other database structures. As of MySQL 5.6.15, sets InnoDB to read-only.
You can SELECT from tables to dump them. With an innodb_force_recovery value of 3 or less you can DROP or CREATE tables. As of MySQL 5.6.27, DROP TABLE is also supported with an innodb_force_recovery value greater than 3.
If you know that a given table is causing a crash on rollback, you can drop it. If you encounter a runaway rollback caused by a failing mass import or ALTER TABLE, you can kill the mysqld process and set innodb_force_recovery to 3 to bring the database up without the rollback, and then DROP the table that is causing the runaway rollback.
If corruption within the table data prevents you from dumping the entire table contents, a query with an ORDER BY primary_key DESC clause might be able to dump the portion of the table after the corrupted part.
If a high innodb_force_recovery value is required to start InnoDB, there may be corrupted data structures that could cause complex queries (queries containing WHERE, ORDER BY, or other clauses) to fail. In this case, you may only be able to run basic SELECT * FROM t queries.

官网上说了，都到设置innodb_force_recory的地步了，级别从1到6逐个试，试到能启动mysql为止，启动后的第一件事是导出数据，然后删了数据库日志，将innodb_force_recory还原成0后，启动再还原数据，能救多少是多少。

tips：

1、用命令导出数据文件时，要注意会不会出现异常中断的现象，可以简单地将导出文件与ibdata1进行文件大小比对，一般导出文件会比ibdata1大。

2、/mysql/data/目录下的文件其实是各个数据库表结构的说明文件，如果不删除，导入数据文件时会提示表存在无法导入。删除文件时记得要备份。

3、innodb_force_recory设置大于0时，数据库访问权限会变成只读，查询SQL可以正常执行，DML执行会失败。