MySQL数据恢复工具之MyFlash
文章目录
- 概述
- 一.MyFlash安装
- 1.1 在线安装
- 1.2 离线安装
- 二.MyFlash参数
- 2.1 参数介绍
- 2.2 语法举例
- 三.MyFlash测试案例
- 3.1 数据准备
- 3.2 测试恢复DML
- 3.3 测试DDL恢复
- 参考:
概述
MyFlash是美团开发的一个回滚DML操作的工具，该工具是开源的，github地址为：https://github.com/Meituan-Dianping/MyFlash。通过该工具，可以实现MySQL数据库的闪回。
优点
1.如果binlog日志保留，可以闪回到任意时间
2.可以针对实例、数据库、表及指定的DML语句进行回滚
3.性能优于binlog2sql、mysqlbinlog
限制
1.binlog格式必须为row,且binlog_row_image=full
2.仅支持5.6与5.7
3.只能回滚DML（增、删、改）
一.MyFlash安装
1.1 在线安装
cd /usr/local/src
yum -y install git
yum install gcc* pkg-config glib2 libgnomeui-devel -y
git clone https://github.com/Meituan-Dianping/MyFlash.git
cd MyFlash/
gcc -w `pkg-config --cflags --libs glib-2.0` source/binlogParseGlib.c -o binary/flashback
安装记录
[root@10-31-1-119 src]# git clone https://github.com/Meituan-Dianping/MyFlash.git
正克隆到 'MyFlash'...
remote: Enumerating objects: 200, done.
remote: Total 200 (delta 0), reused 0 (delta 0), pack-reused 200
接收对象中: 100% (200/200), 2.60 MiB | 30.00 KiB/s, done.
处理 delta 中: 100% (85/85), done.
[root@10-31-1-119 src]# cd MyFlash/
[root@10-31-1-119 MyFlash]# gcc -w `pkg-config --cflags --libs glib-2.0` source/binlogParseGlib.c -o binary/flashback
1.2 离线安装
unzip MyFlash-master.zip
mv MyFlash-master /usr/local/MyFlash/
gcc -w pkg-config --cflags --libs glib-2.0 source/binlogParseGlib.c -o binary/flashback
二.MyFlash参数
2.1 参数介绍
[root@10-31-1-119 MyFlash]# cd binary/
[root@10-31-1-119 binary]# ./flashback --help
Usage:
flashback [OPTION?]
Help Options:
-h, --help Show help options
Application Options:
--databaseNames databaseName to apply. if multiple, seperate by comma(,)
--tableNames tableName to apply. if multiple, seperate by comma(,)
--start-position start position
--stop-position stop position
--start-datetime start time (format %Y-%m-%d %H:%M:%S)
--stop-datetime stop time (format %Y-%m-%d %H:%M:%S)
--sqlTypes sql type to filter . support INSERT, UPDATE ,DELETE. if multiple, seperate by comma(,)
--maxSplitSize max file size after split, the uint is M
--binlogFileNames binlog files to process. if multiple, seperate by comma(,)
--outBinlogFileNameBase output binlog file name base
--logLevel log level, available option is debug,warning,error
--include-gtids gtids to process
--exclude-gtids gtids to skip
参数
--databaseNames : 需要闪回的数据库名称，如果有多个数据库，用逗号”，”隔开。
--tableNames ： 要闪回的表名称，如果有多个表，用逗号”,”隔开。
--start-position ：闪回的起始位置，如不指定，从文件开始处回滚。
--stop-position : 闪回的终止位置，如不指定，回滚到文件结尾。
--start-datetime ： 闪回的开始时间。
--stop-datetime : 闪回的终止时间。
--sqlTypes ： 指定需要回滚的sql类型，支持INSERT、UPDATE、DELETE，多个类型使用逗号”,”分开。
--maxSplitSize ： 对文件进行固定尺寸的切割，以防止单次应用binlog尺寸较大，对线上造成压力。
--binlogFileNames : 指定需要回滚的binlog文件，美团文档说目前只支持单个binlog文件，经测试已经支持多个binlog文件同时闪回。
--outBinlogFileNameBase ：指定输出的binlog文件前缀，如不指定，则默认为binlog_output_base.flashback。
logLevel ： 仅供开发者使用，默认级别为error级别。在生产环境中不要修改这个级别，否则输出过多。
include-gtids ： 指定需要回滚的gtid,支持gtid的单个和范围两种形式。
exclude-gtids ： 指定不需要回滚的gtid，用法同include-gtids。
2.2 语法举例
1） 回滚整个文件
./flashback --binlogFileNames=haha.000041
mysqlbinlog binlog_output_base.flashback | mysql -h -u -p
2）回滚该文件中的所有insert语句
./flashback --sqlTypes='INSERT' --binlogFileNames=haha.000041
mysqlbinlog binlog_output_base.flashback | mysql -h -u -p
3）回滚大文件
回滚
./flashback --binlogFileNames=haha.000042
切割大文件
./flashback --maxSplitSize=1 --binlogFileNames=binlog_output_base.flashback
应用
mysqlbinlog binlog_output_base.flashback.000001 | mysql -h -u -p
...
mysqlbinlog binlog_output_base.flashback. | mysql -h -u -p
三.MyFlash测试案例
MyFlash闪回，一定要找准时间点，这个是比较麻烦的事情
3.1 数据准备
mysql> select current_timestamp();
+---------------------+
| current_timestamp() |
+---------------------+
| 2020-08-27 14:59:44 |
+---------------------+
1 row in set (0.00 sec)
mysql> create database test1 character set utf8;
Query OK, 1 row affected (0.00 sec)
mysql> create table t1(id int,name varchar(100));
ERROR 1046 (3D000): No database selected
mysql>
mysql> use test1;
Database changed
mysql>
mysql> create table t1(id int,name varchar(100));
Query OK, 0 rows affected (0.01 sec)
mysql> insert into t1 values (1,'a'),(2,'b'),(3,'c'),(4,'d'),(5,'f');
Query OK, 5 rows affected (0.01 sec)
Records: 5 Duplicates: 0 Warnings: 0
mysql> update t1 set name='test' where id >3;
Query OK, 2 rows affected (0.00 sec)
Rows matched: 2 Changed: 2 Warnings: 0
mysql> delete from t1 where id = 1;
Query OK, 1 row affected (0.00 sec)
mysql>
mysql> select * from t1;
+------+------+
| id | name |
+------+------+
| 2 | b |
| 3 | c |
| 4 | test |
| 5 | test |
+------+------+
4 rows in set (0.00 sec)
mysql> select current_timestamp();
+---------------------+
| current_timestamp() |
+---------------------+
| 2020-08-27 15:00:36 |
+---------------------+
1 row in set (0.00 sec)
3.2 测试恢复DML
需求是恢复到insert 5条数据之后的状态
-- 恢复之前 flush logs 换一个binlog
flush logs;
-- 通过mysqlbinlog查看 位置
mysqlbinlog -vv 10-31-1-119-bin.000008 | less
-- 不指定时间和位置，直接将整个binlog里面的update和delete语句的回滚脚本生成
./flashback --binlogFileNames=/var/lib/mysql/10-31-1-119-bin.000007 --databaseNames=test1 --tableNames=t1 --sqlTypes='UPDATE','DELETE' --outBinlogFileNameBase=test_tb
-- 查看闪回文件
mysqlbinlog --no-defaults --base64-output=decode-rows -vv test_tb.flashback
-- 执行恢复
mysqlbinlog --no-defaults binlog_output_base.flashback |mysql -uroot -pyourpass
测试记录
[root@10-31-1-119 binary]# ./flashback --binlogFileNames=/var/lib/mysql/10-31-1-119-bin.000007 --databaseNames=test1 --tableNames=t1 --sqlTypes='UPDATE','DELETE' --outBinlogFileNameBase=test_tb
[root@10-31-1-119 binary]# mysqlbinlog --no-defaults --base64-output=decode-rows -vv test_tb.flashback
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at 4
#200826 16:51:22 server id 11 end_log_pos 123 CRC32 0x7a848881 Start: binlog v 4, server v 5.7.31-log created 200826 16:51:22
# at 123
#200827 14:56:25 server id 11 end_log_pos 172 CRC32 0x50b79220 Table_map: `test1`.`t1` mapped to number 222
# at 172
#200827 14:56:25 server id 11 end_log_pos 215 CRC32 0xc0b0e68d Write_rows: table id 222 flags: STMT_END_F
### INSERT INTO `test1`.`t1`
### SET
### @1=1 /* INT meta=0 nullable=1 is_null=0 */
### @2='a' /* VARSTRING(300) meta=300 nullable=1 is_null=0 */
# at 215
#200826 16:51:27 server id 11 end_log_pos 246 CRC32 0x6474cbaf Xid = 3022960
COMMIT/*!*/;
# at 246
#200827 14:56:18 server id 11 end_log_pos 295 CRC32 0x5eeff83d Table_map: `test1`.`t1` mapped to number 222
# at 295
#200827 14:56:18 server id 11 end_log_pos 369 CRC32 0x18b0acc3 Update_rows: table id 222 flags: STMT_END_F
### UPDATE `test1`.`t1`
### WHERE
### @1=4 /* INT meta=0 nullable=1 is_null=0 */
### @2='test' /* VARSTRING(300) meta=300 nullable=1 is_null=0 */
### SET
### @1=4 /* INT meta=0 nullable=1 is_null=0 */
### @2='d' /* VARSTRING(300) meta=300 nullable=1 is_null=0 */
### UPDATE `test1`.`t1`
### WHERE
### @1=5 /* INT meta=0 nullable=1 is_null=0 */
### @2='test' /* VARSTRING(300) meta=300 nullable=1 is_null=0 */
### SET
### @1=5 /* INT meta=0 nullable=1 is_null=0 */
### @2='f' /* VARSTRING(300) meta=300 nullable=1 is_null=0 */
# at 369
#200826 16:51:27 server id 11 end_log_pos 400 CRC32 0x6474cbaf Xid = 3022960
COMMIT/*!*/;
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# End of log file
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
[root@10-31-1-119 binary]#
[root@10-31-1-119 binary]# mysqlbinlog --no-defaults test_tb.flashback |mysql -uroot -pabc123
mysql: [Warning] Using a password on the command line interface can be insecure.
[root@10-31-1-119 binary]#
[root@10-31-1-119 binary]#
[root@10-31-1-119 binary]#
[root@10-31-1-119 binary]#
-- 可以看到数据进行了闪回
[root@10-31-1-119 binary]# mysql -uroot -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 5174
Server version: 5.7.31-log MySQL Community Server (GPL)
Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> select * from test1.t1;
+------+------+
| id | name |
+------+------+
| 2 | b |
| 3 | c |
| 4 | d |
| 5 | f |
| 1 | a |
+------+------+
5 rows in set (0.00 sec)
mysql>
3.3 测试DDL恢复
MyFlash是不支持DDL恢复的，我们来测试下
数据准备，直接drop掉test.t3表
mysql> flush logs;
Query OK, 0 rows affected (0.01 sec)
mysql> show master status;
+------------------------+----------+--------------+------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------------+----------+--------------+------------------+-------------------+
| 10-31-1-119-bin.000010 | 154 | | | |
+------------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)
mysql> use test;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> drop table t3;
Query OK, 0 rows affected (0.01 sec)
mysql>
mysql> flush logs;
Query OK, 0 rows affected (0.00 sec)
mysql> show master status;
+------------------------+----------+--------------+------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------------+----------+--------------+------------------+-------------------+
| 10-31-1-119-bin.000011 | 154 | | | |
+------------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)
[root@10-31-1-119 binary]# ./flashback --binlogFileNames=/var/lib/mysql/10-31-1-119-bin.000010 --databaseNames=test --tableNames=t3 --outBinlogFileNameBase=test_tb3
[root@10-31-1-119 binary]# mysqlbinlog --no-defaults --base64-output=decode-rows -vv test_tb3.flashback
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at 4
#200827 16:45:43 server id 11 end_log_pos 123 CRC32 0x1ea17bef Start: binlog v 4, server v 5.7.31-log created 200827 16:45:43
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# End of log file
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
[root@10-31-1-119 binary]#
参考:
1.https://www.cnblogs.com/lijiaman/p/12770415.html
2.http://blog.itpub.net/26736162/viewspace-2674196/