(以下情况仅针对StoneDB 1.0版本不支持的部分DML和DDL操作,StoneDB 2.0及以上版本将无需此类操作)
主从复制中,主库的任何更新都会同步到从库,如果从库不想重做主库的某个更新动作,可以使用以下两种方法进行规避。当然,最终带来的影响是主从环境数据不一致的问题。
以下的测试环境中,主库是 InnoDB,从库是 StoneDB,在主库做从库不支持的 DML 或者 DDL。
从库执行 GTID 的空事务
###主库
mysql> show create table ttt\G
*************************** 1. row ***************************
Table: ttt
Create Table: CREATE TABLE `ttt` (
`id` int(11) NOT NULL,
`name` varchar(5) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
1 row in set (0.00 sec)
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
| 3 | CCC |
+----+------+
3 rows in set (0.00 sec)
###从库
mysql> show create table ttt\G
*************************** 1. row ***************************
Table: ttt
Create Table: CREATE TABLE `ttt` (
`id` int(11) NOT NULL,
`name` varchar(5) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=STONEDB DEFAULT CHARSET=utf8mb4
1 row in set (0.02 sec)
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
| 3 | CCC |
+----+------+
3 rows in set (0.00 sec)
###主库
mysql> delete from ttt where id=3;
Query OK, 1 row affected (0.00 sec)
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
+----+------+
2 rows in set (0.00 sec)
###从库
mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.30.101
Master_User: u_repl
Master_Port: 33306
Connect_Retry: 60
Master_Log_File: binlog.000002
Read_Master_Log_Pos: 1053
Relay_Log_File: ub01-relay-bin.000002
Relay_Log_Pos: 993
Relay_Master_Log_File: binlog.000002
Slave_IO_Running: Yes
Slave_SQL_Running: No
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 1031
Last_Error: Error 'Table storage engine for 'ttt' doesn't have this option' on query. Default database: 'db'. Query: 'delete from ttt where id=3'
Skip_Counter: 0
Exec_Master_Log_Pos: 786
Relay_Log_Space: 1466
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 1031
Last_SQL_Error: Error 'Table storage engine for 'ttt' doesn't have this option' on query. Default database: 'db'. Query: 'delete from ttt where id=3'
Replicate_Ignore_Server_Ids:
Master_Server_Id: 101
Master_UUID: ae40cabd-efb2-11ec-ac20-44a84203989a
Master_Info_File: /data/stonedb/install/data/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State:
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp: 220729 02:26:29
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set: ae40cabd-efb2-11ec-ac20-44a84203989a:1-4
Executed_Gtid_Set: 4ddecc1a-ee49-11ec-96fe-f219e7257407:1,
ae40cabd-efb2-11ec-ac20-44a84203989a:1-3
Auto_Position: 1
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
| 3 | CCC |
+----+------+
3 rows in set (0.00 sec)
主库执行 delete后,由于 StoneDB 不支持 delete,从库会有报错,并且主从复制中断。
下一步需要在主库找到执行 delete操作的gtid值。
###主库
mysql> show binary logs;
+---------------+-----------+
| Log_name | File_size |
+---------------+-----------+
| binlog.000001 | 177 |
| binlog.000002 | 1053 |
+---------------+-----------+
2 rows in set (0.00 sec)
mysql> show binlog events in '/data/stonedb/install/binlog/binlog.000002';
+---------------+------+----------------+-----------+-------------+-------------------------------------------------------------------+
| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |
+---------------+------+----------------+-----------+-------------+-------------------------------------------------------------------+
| binlog.000002 | 4 | Format_desc | 101 | 123 | Server ver: 5.7.36-StoneDB-log, Binlog ver: 4 |
| binlog.000002 | 123 | Previous_gtids | 101 | 154 | |
| binlog.000002 | 154 | Gtid | 101 | 219 | SET @@SESSION.GTID_NEXT= 'ae40cabd-efb2-11ec-ac20-44a84203989a:1' |
| binlog.000002 | 219 | Query | 101 | 307 | create database db |
| binlog.000002 | 307 | Gtid | 101 | 372 | SET @@SESSION.GTID_NEXT= 'ae40cabd-efb2-11ec-ac20-44a84203989a:2' |
| binlog.000002 | 372 | Query | 101 | 494 | use `db`; create table ttt(id int primary key,name varchar(5)) |
| binlog.000002 | 494 | Gtid | 101 | 559 | SET @@SESSION.GTID_NEXT= 'ae40cabd-efb2-11ec-ac20-44a84203989a:3' |
| binlog.000002 | 559 | Query | 101 | 634 | BEGIN |
| binlog.000002 | 634 | Query | 101 | 755 | use `db`; insert into ttt values(1,'AAA'),(2,'BBB'),(3,'CCC') |
| binlog.000002 | 755 | Xid | 101 | 786 | COMMIT /* xid=20 */ |
| binlog.000002 | 786 | Gtid | 101 | 851 | SET @@SESSION.GTID_NEXT= 'ae40cabd-efb2-11ec-ac20-44a84203989a:4' |
| binlog.000002 | 851 | Query | 101 | 926 | BEGIN |
| binlog.000002 | 926 | Query | 101 | 1022 | use `db`; delete from ttt where id=3 |
| binlog.000002 | 1022 | Xid | 101 | 1053 | COMMIT /* xid=28 */ |
+---------------+------+----------------+-----------+-------------+-------------------------------------------------------------------+
14 rows in set (0.00 sec)
如果是在生产环境找主库 delete 操作的 gtid 值,需要知道哪个时间点,然后用 mysqlbinlog 解析binlog。
这里由于是做测试,可以简单快递的找到 delete 操作的 gtid 值,ae40cabd-efb2-11ec-ac20-44a84203989a:4。
gtid 值由参数 server_uuid 和事务 id 组成,标识一个这个操作的唯一性。
###从库
set gtid_next='ae40cabd-efb2-11ec-ac20-44a84203989a:4';
begin;
commit;
set gtid_next=automatic;
start slave;
mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.30.101
Master_User: u_repl
Master_Port: 33306
Connect_Retry: 60
Master_Log_File: binlog.000002
Read_Master_Log_Pos: 1053
Relay_Log_File: ub01-relay-bin.000002
Relay_Log_Pos: 1260
Relay_Master_Log_File: binlog.000002
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 1053
Relay_Log_Space: 1466
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 101
Master_UUID: ae40cabd-efb2-11ec-ac20-44a84203989a
Master_Info_File: /data/stonedb/install/data/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set: ae40cabd-efb2-11ec-ac20-44a84203989a:1-4
Executed_Gtid_Set: 4ddecc1a-ee49-11ec-96fe-f219e7257407:1,
ae40cabd-efb2-11ec-ac20-44a84203989a:1-4
Auto_Position: 1
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
| 3 | CCC |
+----+------+
3 rows in set (0.00 sec)
利用 gtid 跳过一个空事务后,主从复制的线程已经正常启动,但由于 StoneDB 不支持 delete,现在主从环境数据是不一致的。
###主库
mysql> insert into ttt values(4,'DDD');
Query OK, 1 row affected (0.00 sec)
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
| 4 | DDD |
+----+------+
3 rows in set (0.00 sec)
###从库
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
| 3 | CCC |
| 4 | DDD |
+----+------+
4 rows in set (0.00 sec)
主从复制的线程启动后,主库的更新,从库同步正常。
如果觉得在主库找 delete 的 gtid 值麻烦,在主库执行 delete 前,可以指定 delete 的 gtid 值。在从库还是根据这个 gtid 值执行空事务。
###
mysql> show variables like 'server_uuid';
+---------------+--------------------------------------+
| Variable_name | Value |
+---------------+--------------------------------------+
| server_uuid | ae40cabd-efb2-11ec-ac20-44a84203989a |
+---------------+--------------------------------------+
1 row in set (0.01 sec)
mysql> set gtid_next='ae40cabd-efb2-11ec-ac20-44a84203989a:100';
Query OK, 0 rows affected (0.00 sec)
mysql> begin;
Query OK, 0 rows affected (0.00 sec)
mysql> delete from ttt where id=1;
Query OK, 1 row affected (0.00 sec)
mysql> commit;
Query OK, 0 rows affected (0.00 sec)
mysql> set gtid_next=automatic;
Query OK, 0 rows affected (0.00 sec)
###从库
set gtid_next='ae40cabd-efb2-11ec-ac20-44a84203989a:100';
begin;
commit;
set gtid_next=automatic;
start slave;
关闭当前线程的binlog
###主库
mysql> show create table ttt\G
*************************** 1. row ***************************
Table: ttt
Create Table: CREATE TABLE `ttt` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(5) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb4
1 row in set (0.00 sec)
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
+----+------+
2 rows in set (0.00 sec)
###从库
mysql> show create table ttt\G
*************************** 1. row ***************************
Table: ttt
Create Table: CREATE TABLE `ttt` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(5) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=STONEDB DEFAULT CHARSET=utf8mb4
1 row in set (0.00 sec)
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
+----+------+
2 rows in set (0.00 sec)
###主库
mysql> set sql_log_bin=off;
Query OK, 0 rows affected (0.00 sec)
mysql> alter table ttt modify name varchar(10);
Query OK, 0 rows affected (0.00 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> show create table ttt\G
*************************** 1. row ***************************
Table: ttt
Create Table: CREATE TABLE `ttt` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(10) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb4
1 row in set (0.00 sec)
###从库
mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.30.101
Master_User: u_repl
Master_Port: 33306
Connect_Retry: 60
Master_Log_File: binlog.000002
Read_Master_Log_Pos: 2288
Relay_Log_File: ub01-relay-bin.000002
Relay_Log_Pos: 2495
Relay_Master_Log_File: binlog.000002
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 2288
Relay_Log_Space: 2701
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 101
Master_UUID: ae40cabd-efb2-11ec-ac20-44a84203989a
Master_Info_File: /data/stonedb/install/data/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set: ae40cabd-efb2-11ec-ac20-44a84203989a:1-8:100
Executed_Gtid_Set: 4ddecc1a-ee49-11ec-96fe-f219e7257407:1-3,
ae40cabd-efb2-11ec-ac20-44a84203989a:1-8:100
Auto_Position: 1
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)
mysql> show create table ttt\G
*************************** 1. row ***************************
Table: ttt
Create Table: CREATE TABLE `ttt` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(5) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=STONEDB DEFAULT CHARSET=utf8mb4
1 row in set (0.00 sec)
主库关闭当前线程的binlog,对表做DDL,将字段 name 的长度扩大。
主从复制正常,从库表的字段 name 的长度不变。
###主库
开启新的线程,注意一点是开启新的线程!!!
mysql> insert into ttt(name) values('CCC');
Query OK, 1 row affected (0.00 sec)
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
| 3 | CCC |
+----+------+
3 rows in set (0.00 sec)
###从库
mysql> select * from ttt;
+----+------+
| id | name |
+----+------+
| 1 | AAA |
| 2 | BBB |
| 3 | CCC |
+----+------+
3 rows in set (0.00 sec)
sql_log_bin=off,关闭的是当前线程的binlog,不影响其他线程的任何更新。
以上两种方法都可以规避从库不想重做主库的某个更新动作,目的是让从库遇到不支持的操作时可以让主从复制的线程正常工作,但带来的问题是主从环境数据不一致。