MySQL5.7 的GTID复制
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
在MySQL5.6之后其官方推出了GTID复制方式，和传统的基于bin log复制方式有所不同，接下来我们一起来了解一下它！
一.什么是GTID（global transaction identitifiers）
GTID复制是完全基于事务的复制，即每个在主库上执行的事务都会被分配到一个唯一的全局ID并记录和应用在主库上。
这种复制方式简化了简历slave和master/slave之间的切换工作，因为其完全不需要找当前执行的bin log和log中的位置完成切换。
一个GTID是master上执行的任何commit事务所分配的全局唯一ID标识，其由两部分组成。即GTID = source_id:transaction_id。source_id代表主库的server_uuid，transaction_id代表事务按顺序提交的ID,比如第一个提交则是1，第十个提交的事物则是10.
GTID集合代表一组GTID.
二.MySQL基于GTID的复制
1>.MySQL基于GTID复制的原理
当一个事务在主库提交时，该事务就被赋予了一个GTID，并记录在主库的binary log
主库的binary log会被传输到从库的relay log中，从库读取此GTID并生成gtid_next系统参数
从库验证此GTID并没有在自己的binary log 中使用，则应用此事物在从库上。
2>.MySQL5.6和MySQL5.7GTID复制模式之间的区别
MySQL5.6的GTID复制模式，slave必须开启binary log和log_slave_updates参数，否则启动就报错，因为需要在binary log找到同步复制的信息（UUID：事务号），注意，开启log_slave_updates参数，是把relay-log里的日志内容再记录到本地的binary log里。
但在MySQL5.7里，官方卫做了调整，用一张gtid_executed系统表记录同步复制的信息（UUID：事务号），这样就可以不用开启log_slave_updates参数，减少了从库的压力。
从MySQL5.7.4版本开始，GTID会存在在mysql系统库的gtid_executed表中。
[root@node102 ~]# mysql -uroot -pyinzhengjie mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 4 Server version: 5.7.25-log MySQL Community Server (GPL) Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> mysql> mysql> mysql> USE mysql Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> mysql> desc gtid_executed; +----------------+------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +----------------+------------+------+-----+---------+-------+ | source_uuid | char(36) | NO | PRI | NULL | | | interval_start | bigint(20) | NO | PRI | NULL | | | interval_end | bigint(20) | NO | | NULL | | +----------------+------------+------+-----+---------+-------+ 3 rows in set (0.00 sec) mysql> mysql> SELECT * FROM gtid_executed; Empty set (0.00 sec) mysql> mysql>
#node101.yinzhengjie.org.cn配置gtid生效 [root@node101 ~]# [root@node101 ~]# cat /etc/my.cnf [mysqld] basedir=/yinzhengjie/softwares/mysql/ datadir=/yinzhengjie/softwares/mysql/data/ log-bin=yinzhengjie-mysql-bin server-id=1 gtid-mode=on enforce-gtid-consistency=on [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# /etc/init.d/mysql.server restart Shutting down MySQL............ SUCCESS! Starting MySQL. SUCCESS! [root@node101 ~]# #node102.yinzhengjie.org.cn配置gtid生效 [root@node102 ~]# /etc/init.d/mysql.server start Starting MySQL SUCCESS! [root@node102 ~]# 2019-03-06T14:57:02.069380Z mysqld_safe A mysqld process already exists [root@node102 ~]# [root@node102 ~]# [root@node102 ~]# /etc/init.d/mysql.server restart Shutting down MySQL.. SUCCESS! Starting MySQL. SUCCESS! [root@node102 ~]# [root@node102 ~]#
[root@node102 ~]# mysql -uroot -pyinzhengjie mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 7 Server version: 5.7.25-log MySQL Community Server (GPL) Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> mysql> mysql> STOP SLAVE; Query OK, 0 rows affected (0.00 sec) mysql> mysql> RESET SLAVE ALL; Query OK, 0 rows affected (0.00 sec) mysql> mysql> CHANGE MASTER TO -> MASTER_HOST='node101.yinzhengjie.org.cn', -> MASTER_PORT=3306, -> MASTER_USER='copy', -> MASTER_PASSWORD='yinzhengjie', -> MASTER_AUTO_POSITION=1; Query OK, 0 rows affected, 2 warnings (0.01 sec) mysql> mysql> START SLAVE; Query OK, 0 rows affected (0.00 sec) mysql> mysql> mysql> mysql> SHOW SLAVE STATUS\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: node101.yinzhengjie.org.cn Master_User: copy Master_Port: 3306 Connect_Retry: 60 Master_Log_File: yinzhengjie-mysql-bin.000005 Read_Master_Log_Pos: 154 Relay_Log_File: node102-relay-bin.000002 Relay_Log_Pos: 391 Relay_Master_Log_File: yinzhengjie-mysql-bin.000005 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 154 Relay_Log_Space: 600 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 1 Master_UUID: b2127a6e-3cf8-11e9-ae0d-000c29fe9bef Master_Info_File: /yinzhengjie/softwares/mysql/data/master.info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: Executed_Gtid_Set: Auto_Position: 1 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: 1 row in set (0.00 sec) mysql>
[root@node101 ~]# mysql -uroot -pyinzhengjie mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 3 Server version: 5.7.25-log MySQL Community Server (GPL) Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> mysql> use course Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> show tables; +------------------+ | Tables_in_course | +------------------+ | course | | dept | | score | | students | | teacher | +------------------+ 5 rows in set (0.00 sec) mysql> select * from students; +-----+-----------+--------+---------+ | sid | sname | gender | dept_id | +-----+-----------+--------+---------+ | 1 | Jason Yin | 0 | 1 | | 2 | Andy | 0 | 1 | | 3 | Bob | 0 | 1 | | 4 | Ruth | 1 | 2 | | 5 | Mike | 0 | 2 | +-----+-----------+--------+---------+ 5 rows in set (0.00 sec) mysql> mysql> mysql> insert into students values(6,'Jenny',1,1); Query OK, 1 row affected (0.00 sec) mysql> mysql> show master status; +------------------------------+----------+--------------+------------------+----------------------------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +------------------------------+----------+--------------+------------------+----------------------------------------+ | yinzhengjie-mysql-bin.000005 | 436 | | | b2127a6e-3cf8-11e9-ae0d-000c29fe9bef:1 | +------------------------------+----------+--------------+------------------+----------------------------------------+ 1 row in set (0.00 sec) mysql> mysql> mysql> show master status; +------------------------------+----------+--------------+------------------+------------------------------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +------------------------------+----------+--------------+------------------+------------------------------------------+ | yinzhengjie-mysql-bin.000005 | 718 | | | b2127a6e-3cf8-11e9-ae0d-000c29fe9bef:1-2 | +------------------------------+----------+--------------+------------------+------------------------------------------+ 1 row in set (0.00 sec) mysql> mysql> select * from students; +-----+-----------+--------+---------+ | sid | sname | gender | dept_id | +-----+-----------+--------+---------+ | 1 | Jason Yin | 0 | 1 | | 2 | Andy | 0 | 1 | | 3 | Bob | 0 | 1 | | 4 | Ruth | 1 | 2 | | 5 | Mike | 0 | 2 | | 6 | Jenny | 1 | 1 | | 7 | DANNY | 0 | 2 | +-----+-----------+--------+---------+ 7 rows in set (0.00 sec) mysql>
[root@node102 ~]# [root@node102 ~]# mysql -uroot -pyinzhengjie mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 10 Server version: 5.7.25-log MySQL Community Server (GPL) Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> mysql> mysql> USE course Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> mysql> SELECT * FROM students; +-----+-----------+--------+---------+ | sid | sname | gender | dept_id | +-----+-----------+--------+---------+ | 1 | Jason Yin | 0 | 1 | | 2 | Andy | 0 | 1 | | 3 | Bob | 0 | 1 | | 4 | Ruth | 1 | 2 | | 5 | Mike | 0 | 2 | | 6 | Jenny | 1 | 1 | | 7 | DANNY | 0 | 2 | +-----+-----------+--------+---------+ 7 rows in set (0.00 sec) mysql> mysql>
mysql> SELECT * FROM mysql.gtid_executed; #这张表记录了复制了哪些事物！ +--------------------------------------+----------------+--------------+ | source_uuid | interval_start | interval_end | +--------------------------------------+----------------+--------------+ | b2127a6e-3cf8-11e9-ae0d-000c29fe9bef | 1 | 1 | | b2127a6e-3cf8-11e9-ae0d-000c29fe9bef | 2 | 2 | +--------------------------------------+----------------+--------------+ 2 rows in set (0.00 sec) mysql> mysql>
三.MySQL使用GTID复制的限制条件
由于GTID复制是依赖于事物的，所以MySQL的一些属性不支持。
当一个事物中既包含对InnoDB表的操作，也包含对非事物型存储引擎表（MyISAM）的操作时，就会导致一个事物中可能会产生多个GTID的情况；或者是当master和slave的表使用的存储引擎不一样时，都会导致GTID复制功能不正常。
CREATE TABLE ... SELECT语句在基于语句复制的环境中是不安全的，在基于行复制的环境中，此语句会被拆分成两个事件，一个是创建表，一个是INSERT数据，在某些情况下这两个事件会被分配相同的GTID，而导致insert的操作被忽略，所以GTID复制不支持CREATE TABLE .... SELECT语句。
[root@node101 ~]# mysql -uroot -pyinzhengjie mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 3 Server version: 5.7.25-log MySQL Community Server (GPL) Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> mysql> mysql> CREATE TABLE temp SELECT * FROM course.students; ERROR 1786 (HY000): Statement violates GTID consistency: CREATE TABLE ... SELECT. mysql> mysql>
CREATE/DROP TEMPORARY TABLE语句在GTID复制环境中不能放在事物中执行，只能单独执行，并且autocommit要开启
sql_slave_skip_counter语句是不支持的，如果想要跳过事务，可以使用gitd_exected变量。