MySql GR启动时节点状态卡在RECOVERING
问题描述：
root@localhost:(none) 04:47:06>SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| CHANNEL_NAME | MEMBER_ID | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| group_replication_applier | 574d6e9a-fc15-11e7-9fcf-000c29879d51 | centos129 | 3306 | ONLINE |
| group_replication_applier | b376058d-5762-11e7-baa0-000c29e6b568 | centos128 | 3306 | RECOVERING |
| group_replication_applier | e87b5168-fc15-11e7-9fcf-000c29879d51 | centos130 | 3306 | ONLINE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+并且错误日志显示内容如下：
2018-05-15T08:08:18.041338Z 54 [ERROR] Error reading packet from server for channel 'group_replication_recovery': The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires. (server_errno=1236)
2018-05-15T08:08:18.041384Z 54 [ERROR] Slave I/O for channel 'group_replication_recovery': Got fatal error 1236 from master when reading data from binary log: 'The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.', Error_code: 1236
2018-05-15T08:10:18.511895Z 18 [Note] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_recovery' executed'. Previous state master_host='centos129', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='', master_port= 0, master_log_file='', master_log_pos= 4, master_bind=''.
2018-05-15T08:10:18.523029Z 18 [ERROR] Plugin group_replication reported: 'Maximum number of retries when trying to connect to a donor reached. Aborting group replication recovery.'
2018-05-15T08:10:18.523049Z 18 [Note] Plugin group_replication reported: 'Terminating existing group replication donor connection and purging the corresponding logs.'
2018-05-15T08:10:18.531944Z 18 [Note] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_recovery' executed'. Previous state master_host='', master_port= 0, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='', master_port= 0, master_log_file='', master_log_pos= 4, master_bind=''.
2018-05-15T08:10:18.547788Z 18 [ERROR] Plugin group_replication reported: 'Fatal error during the Recovery process of Group Replication. The server will leave the group.'
2018-05-15T08:10:18.549152Z 0 [Note] Plugin group_replication reported: 'getstart group_id 4317e324'
2018-05-15T08:10:22.045063Z 0 [Note] Plugin group_replication reported: 'state 4330 action xa_terminate'
2018-05-15T08:10:22.045482Z 0 [Note] Plugin group_replication reported: 'new state x_start'
2018-05-15T08:10:22.045511Z 0 [Note] Plugin group_replication reported: 'state 4257 action xa_exit'
2018-05-15T08:10:22.045634Z 0 [Note] Plugin group_replication reported: 'Exiting xcom thread'
2018-05-15T08:10:22.045655Z 0 [Note] Plugin group_replication reported: 'new state x_start' 查看centos128的状态：
root@localhost:qn 05:02:15>select * from performance_schema.replication_group_member_stats \G ;
*************************** 1. row ***************************
CHANNEL_NAME: group_replication_applier
VIEW_ID: 15263657103992362:28
MEMBER_ID: b376058d-5762-11e7-baa0-000c29e6b568
COUNT_TRANSACTIONS_IN_QUEUE: 0
COUNT_TRANSACTIONS_CHECKED: 0
COUNT_CONFLICTS_DETECTED: 0
COUNT_TRANSACTIONS_ROWS_VALIDATING: 0
TRANSACTIONS_COMMITTED_ALL_MEMBERS:
LAST_CONFLICT_FREE_TRANSACTION:此时没有显示centos128上的gtid信息。查看其它节点状态：
centos129:
root@localhost:qn 04:47:47>select * from performance_schema.replication_group_member_stats \G ;
*************************** 1. row ***************************
CHANNEL_NAME: group_replication_applier
VIEW_ID: 15263657103992362:28
MEMBER_ID: 574d6e9a-fc15-11e7-9fcf-000c29879d51
COUNT_TRANSACTIONS_IN_QUEUE: 0
COUNT_TRANSACTIONS_CHECKED: 1
COUNT_CONFLICTS_DETECTED: 0
COUNT_TRANSACTIONS_ROWS_VALIDATING: 0
TRANSACTIONS_COMMITTED_ALL_MEMBERS: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1-151,
b376058d-5762-11e7-baa0-000c29e6b568:1-47678
LAST_CONFLICT_FREE_TRANSACTION: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:142
1 row in set (0.00 sec)centos130:
root@localhost:(none) 05:05:06>select * from performance_schema.replication_group_member_stats \G ;
*************************** 1. row ***************************
CHANNEL_NAME: group_replication_applier
VIEW_ID: 15263657103992362:33
MEMBER_ID: e87b5168-fc15-11e7-9fcf-000c29879d51
COUNT_TRANSACTIONS_IN_QUEUE: 5
COUNT_TRANSACTIONS_CHECKED: 1
COUNT_CONFLICTS_DETECTED: 0
COUNT_TRANSACTIONS_ROWS_VALIDATING: 0
TRANSACTIONS_COMMITTED_ALL_MEMBERS: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1-151,
b376058d-5762-11e7-baa0-000c29e6b568:1-47678
LAST_CONFLICT_FREE_TRANSACTION: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:142查看节点的GTID状态：
centos128:
root@localhost:(none) 04:48:02>show global variables like '%gtid%' ;
+---------------------------------------------------+---------+
| Variable_name | Value |
+---------------------------------------------------+---------+
| binlog_gtid_simple_recovery | ON |
| enforce_gtid_consistency | ON |
| group_replication_allow_local_disjoint_gtids_join | OFF |
| group_replication_gtid_assignment_block_size | 1000000 |
| gtid_executed | |
| gtid_executed_compression_period | 1000 |
| gtid_mode | ON |
| gtid_owned | |
| gtid_purged | |
| session_track_gtids | OFF |
+---------------------------------------------------+---------+centos129和centos130一样：
root@localhost:qn 04:44:32>show global variables like '%gtid%' ;
+---------------------------------------------------+------------------------------------------------------------------------------------------+
| Variable_name | Value |
+---------------------------------------------------+------------------------------------------------------------------------------------------+
| binlog_gtid_simple_recovery | ON |
| enforce_gtid_consistency | ON |
| group_replication_allow_local_disjoint_gtids_join | OFF |
| group_replication_gtid_assignment_block_size | 1000000 |
| gtid_executed | aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1-149,
b376058d-5762-11e7-baa0-000c29e6b568:1-47678 |
| gtid_executed_compression_period | 1000 |
| gtid_mode | ON |
| gtid_owned | |
| gtid_purged | aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1-135,
b376058d-5762-11e7-baa0-000c29e6b568:1-47678 |
| session_track_gtids | OFF 根据centos129与centos130的gtid_purged ,更改centos128的gtid_purged使其按gtid_purged开始复制：
root@localhost:(none) 04:53:03>set global gtid_purged='aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1-135,b376058d-5762-11e7-baa0-000c29e6b568:1-47678' ;
Query OK, 0 rows affected (0.24 sec)
root@localhost:(none) 04:53:17>show global variables like '%gtid%' ;
+---------------------------------------------------+------------------------------------------------------------------------------------------+
| Variable_name | Value |
+---------------------------------------------------+------------------------------------------------------------------------------------------+
| binlog_gtid_simple_recovery | ON |
| enforce_gtid_consistency | ON |
| group_replication_allow_local_disjoint_gtids_join | OFF |
| group_replication_gtid_assignment_block_size | 1000000 |
| gtid_executed | aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1-135,
b376058d-5762-11e7-baa0-000c29e6b568:1-47678 |
| gtid_executed_compression_period | 1000 |
| gtid_mode | ON |
| gtid_owned | |
| gtid_purged | aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1-135,
b376058d-5762-11e7-baa0-000c29e6b568:1-47678 |
| session_track_gtids | OFF |
+---------------------------------------------------+------------------------------------------------------------------------------------------+开启centos128的GR：
root@localhost:(none) 04:53:22>CHANGE MASTER TO MASTER_USER='rpl_user', MASTER_PASSWORD='rpl_pass' FOR CHANNEL 'group_replication_recovery';
Query OK, 0 rows affected, 2 warnings (0.02 sec)
root@localhost:(none) 04:54:17>show global variables like '%gtid%' ;
+---------------------------------------------------+------------------------------------------------------------------------------------------+
| Variable_name | Value |
+---------------------------------------------------+------------------------------------------------------------------------------------------+
| binlog_gtid_simple_recovery | ON |
| enforce_gtid_consistency | ON |
| group_replication_allow_local_disjoint_gtids_join | OFF |
| group_replication_gtid_assignment_block_size | 1000000 |
| gtid_executed | aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1-135,
b376058d-5762-11e7-baa0-000c29e6b568:1-47678 |
| gtid_executed_compression_period | 1000 |
| gtid_mode | ON |
| gtid_owned | |
| gtid_purged | aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1-135,
b376058d-5762-11e7-baa0-000c29e6b568:1-47678 |
| session_track_gtids | OFF |
+---------------------------------------------------+------------------------------------------------------------------------------------------+
10 rows in set (0.00 sec)
root@localhost:(none) 04:54:21>START GROUP_REPLICATION;
Query OK, 0 rows affected (1.93 sec)
root@localhost:(none) 04:54:35>SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| CHANNEL_NAME | MEMBER_ID | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| group_replication_applier | 574d6e9a-fc15-11e7-9fcf-000c29879d51 | centos129 | 3306 | ONLINE |
| group_replication_applier | b376058d-5762-11e7-baa0-000c29e6b568 | centos128 | 3306 | ONLINE |
| group_replication_applier | e87b5168-fc15-11e7-9fcf-000c29879d51 | centos130 | 3306 | ONLINE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
3 rows in set (0.00 sec)