MySQL 5.5.X升级至5.7.21遇到的坑
MySQL 5.5.X升级至5.7.21遇到的坑(一)
发现问题
将一个测试环境的5.5.x升级到5.7.21后,打算将5.7.21作为从库,开始同步主库数据(binlog_format=statement),结果刚一start slave就报错
Coordinator stopped because there were error(s) in the worker(s). The most recent failure being: Worker 1 failed executing transaction 'ANONYMOUS' at master log mysql-bin.000004, end_log_pos 812. See error log and/or performance_schema.replication_applier_status_by_worker table for more details about this failure or others, if any.查看error log
2018-02-10T19:52:52.347979+08:00 3 [Warning] Slave I/O for channel '': Notifying master by SET @master_binlog_checksum= @@global.binlog_checksum failed with er
ror: Unknown system variable 'binlog_checksum', Error_code: 1193
2018-02-10T19:52:52.348080+08:00 3 [Warning] Slave I/O for channel '': Unknown system variable 'SERVER_UUID' on master. A probable cause is that the variable i
s not supported on the master (version: 5.5.59-log), even though it is on the slave (version: 5.7.21-log), Error_code: 1193
2018-02-10T19:52:52.445947+08:00 5 [ERROR] Slave SQL for channel '': Worker 1 failed executing transaction 'ANONYMOUS' at master log , end_log_pos 2651; Error
'You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'GET, POST, FILE, CLASS,
METHOD ) VALUES ( 'maokaixin', 1518159802, 3395959414, '' at line 1' on query. Default database: 'fandb'. Query: 'INSERT INTO postlog ( USERNAME, TIME, IP, GE
T, POST, FILE, CLASS, METHOD ) VALUES ( 'maokaixin', 1518159802, 3395959414, 'gameid:0;', '', '', 'api', 'ajaxGetServers' )', Error_code: 1064
2018-02-10T19:52:52.446217+08:00 4 [Warning] Slave SQL for channel '': ... The slave coordinator and worker threads are stopped, possibly leaving data in incon
sistent state. A restart should restore consistency automatically, although using non-transactional storage for data or info tables or DDL queries could lead t
o problems. In such cases you have to examine your data (see documentation for details). Error_code: 1756
2018-02-10T19:52:52.446235+08:00 4 [Note] Slave SQL thread for channel '' exiting, replication stopped in log 'mysql-bin.000003' at position 2313
2018-02-10T19:53:15.708058+08:00 7 [Note] Slave SQL thread for channel '' initialized, starting replication in log 'mysql-bin.000003' at position 2313, relay l
og './mysql-relay.000009' position: 304找出原因
头两个Warning是由于主库没有binlog_checksum参数，也没有SERVER_UUID参数(看来从库开始同步时要先去主库查询这两个参数)
接着的ERROR报的错误竟然是error in your SQL syntax语法错误.
解析binlog后查到SQL语句为
INSERT INTO postlog ( USERNAME, TIME, IP, GET, POST, FILE, CLASS, METHOD ) VALUES ( 'maokaixin', 1518159802, 3395959414, 'gameid:0;', '', '', 'api', 'ajaxGetServers' );我依次在5.5和5.7的环境执行这个SQL发现该SQL在5.5可以正常执行，而在5.7执行就会报错语法错误
于是一点一点分析这个SQL哪有问题，说实话肉眼真没看出来
只好在5.7一点一点执行
INSERT INTO postlog ( USERNAME, TIME, IP) VALUES ( 'maokaixin', 1518159802, 3395959414);没问题可以执行
INSERT INTO postlog ( USERNAME, TIME, IP, GET) VALUES ( 'maokaixin', 1518159802, 3395959414, 'gameid:0;');报错了！,那显然就是GET有问题。 怀疑是保留字？
查看官方文档,发现还真是
https://dev.mysql.com/doc/refman/5.7/en/keywords.html
新的问题
这个保留字的问题还真蛋疼，说明开发不规范呗，只能让开发改了.
目前由于是测试，想要先跳过这个表，于是使用5.7的新特性动态更改replication filter
change replication filter REPLICATE_WILD_IGNORE_TABLE=('mysql.%','information_schema.%','performance_schema.%','union_log%.%','test.%','fandb.postlog');然后跳过这个错误
结果还是报错
奇怪了，我一度怀疑自己的复制过滤规则有问题。然后又搭建了一套5.7和一套5.5测试同样的复制过滤规则都管用
于是怀疑
1. 5.5做主5.7做从本身有问题，复制过滤规则不起作用
2. 复制过滤规则起作用，但是有别的问题
5.5主库
root@localhost 19:54: [fandb]> show create table postlog\G
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 5
Current database: fandb
*************************** 1. row ***************************
Table: postlog
Create Table: CREATE TABLE `postlog` (
`id` int(11) NOT NULL AUTO_INCREMENT COMMENT '自增ID',
`username` char(50) NOT NULL DEFAULT '' COMMENT '管理员账号',
`time` int(11) NOT NULL DEFAULT '0' COMMENT '时间',
`ip` int(11) NOT NULL DEFAULT '0' COMMENT 'IP',
`get` text NOT NULL COMMENT 'GET',
`post` text NOT NULL COMMENT 'POST',
`file` text NOT NULL COMMENT 'FILE',
`class` char(50) NOT NULL DEFAULT '' COMMENT 'CLASS',
`method` char(50) NOT NULL DEFAULT '' COMMENT 'METHOD',
PRIMARY KEY (`id`),
KEY `username` (`username`),
KEY `c_m` (`class`,`method`),
KEY `time` (`time`)
) ENGINE=InnoDB AUTO_INCREMENT=11 DEFAULT CHARSET=utf8 COMMENT='提交日志表'
1 row in set (0.00 sec)
root@localhost 19:59: [fandb]> select * from postlog;
Empty set (0.00 sec)5.7.21从库
root@localhost 19:58: [fandb]> select * from postlog;
Empty set (0.00 sec)
root@localhost 19:59: [fandb]> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 10.105.118.10
Master_User: repl
Master_Port: 3308
Connect_Retry: 60
Master_Log_File: mysql-bin.000004
Read_Master_Log_Pos: 1028
Relay_Log_File: mysql-relay.000010
Relay_Log_Pos: 1217
Relay_Master_Log_File: mysql-bin.000004
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table: mysql.%,information_schema.%,performance_schema.%,union_log%.%,test.%,fandb.postlog
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 1028
Relay_Log_Space: 2296
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 323308
Master_UUID:
Master_Info_File: mysql.slave_master_info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set: 3b6f4462-0190-11e8-ac03-525400a44d53:1-37445291
Auto_Position: 0
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)5.5主库以兼容的方式插入
root@localhost 19:59: [fandb]> INSERT INTO postlog ( USERNAME, TIME, IP, `GET`, POST, FILE, CLASS, METHOD ) VALUES ( 'maokaixin', 1518159802, 3395959414, 'gameid:0;', '', '', 'api', 'ajaxGetServers' );
Query OK, 1 row affected, 1 warning (0.00 sec)
root@localhost 20:00: [fandb]> select * from postlog;
+----+-----------+------------+------------+-----------+------+------+-------+----------------+
| id | username | time | ip | get | post | file | class | method |
+----+-----------+------------+------------+-----------+------+------+-------+----------------+
| 11 | maokaixin | 1518159802 | 2147483647 | gameid:0; | | | api | ajaxGetServers |
+----+-----------+------------+------------+-----------+------+------+-------+----------------+
1 row in set (0.00 sec)5.7.21从库查看
没有插入数据,复制规律规则生效
root@localhost 19:59: [fandb]> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 10.105.118.10
Master_User: repl
Master_Port: 3308
Connect_Retry: 60
Master_Log_File: mysql-bin.000004
Read_Master_Log_Pos: 1395
Relay_Log_File: mysql-relay.000010
Relay_Log_Pos: 1584
Relay_Master_Log_File: mysql-bin.000004
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table: mysql.%,information_schema.%,performance_schema.%,union_log%.%,test.%,fandb.postlog
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 1395
Relay_Log_Space: 2663
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 323308
Master_UUID:
Master_Info_File: mysql.slave_master_info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set: 3b6f4462-0190-11e8-ac03-525400a44d53:1-37445291
Auto_Position: 0
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)
root@localhost 20:00: [fandb]> select * from postlog;
Empty set (0.00 sec)5.5主库以非兼容方式插入
root@localhost 20:01: [fandb]> INSERT INTO postlog ( USERNAME, TIME, IP, GET, POST, FILE, CLASS, METHOD ) VALUES ( 'maokaixin', 1518159802, 3395959414, 'gameid:0;', '', '', 'api', 'ajaxGetServers' );
Query OK, 1 row affected, 1 warning (0.00 sec)
root@localhost 20:02: [fandb]> select * from postlog;
+----+-----------+------------+------------+-----------+------+------+-------+----------------+
| id | username | time | ip | get | post | file | class | method |
+----+-----------+------------+------------+-----------+------+------+-------+----------------+
| 11 | maokaixin | 1518159802 | 2147483647 | gameid:0; | | | api | ajaxGetServers |
| 13 | maokaixin | 1518159802 | 2147483647 | gameid:0; | | | api | ajaxGetServers |
+----+-----------+------------+------------+-----------+------+------+-------+----------------+
2 rows in set (0.00 sec)5.7.21从库复制sql thread报错
root@localhost 20:02: [fandb]> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 10.105.118.10
Master_User: repl
Master_Port: 3308
Connect_Retry: 60
Master_Log_File: mysql-bin.000004
Read_Master_Log_Pos: 1760
Relay_Log_File: mysql-relay.000010
Relay_Log_Pos: 1584
Relay_Master_Log_File: mysql-bin.000004
Slave_IO_Running: Yes
Slave_SQL_Running: No
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table: mysql.%,information_schema.%,performance_schema.%,union_log%.%,test.%,fandb.postlog
Last_Errno: 1064
Last_Error: Coordinator stopped because there were error(s) in the worker(s). The most recent failure being: Worker 1 failed executing transaction 'ANONYMOUS' at master log mysql-bin.000004, end_log_pos 1733. See error log and/or performance_schema.replication_applier_status_by_worker table for more details about this failure or others, if any.
Skip_Counter: 0
Exec_Master_Log_Pos: 1395
Relay_Log_Space: 3028
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 1064
Last_SQL_Error: Coordinator stopped because there were error(s) in the worker(s). The most recent failure being: Worker 1 failed executing transaction 'ANONYMOUS' at master log mysql-bin.000004, end_log_pos 1733. See error log and/or performance_schema.replication_applier_status_by_worker table for more details about this failure or others, if any.
Replicate_Ignore_Server_Ids:
Master_Server_Id: 323308
Master_UUID:
Master_Info_File: mysql.slave_master_info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State:
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp: 180210 20:02:04
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set: 3b6f4462-0190-11e8-ac03-525400a44d53:1-37445291
Auto_Position: 0
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)由此推断,复制过滤规则没有问题,是生效的
但是判断时需要先解析sql,当解析时发现sql语法错误(实际为get为MySQL5.7保留字),导致sql thread报错