Group Replication 环境搭建

---

Group Replication 环境搭建

基本环境

类型	说明
操作系统	Linux version 3.10.0-693.el7.x86_64
MySQL 版本	mysql/mysql-5.7.23-linux 64 位

部署规划

本次部署为多机单实例
basedir

/usr/local/mysql

IP	端口号	数据目录	group_replication 通信端口
172.18.0.11	3306	/data/mgr/mysql3306{data,logs,tmp}	23306
172.18.0.12	3306	/data/mgr/mysql3306{data,logs,tmp}	23306
172.18.0.13	3306	/data/mgr/mysql3306{data,logs,tmp}	23306

基本初始化

解压 MySQL

# cd /opt/mysql/
# tar zxvf /path/mysql-5.7.25-linux-glibc2.12-x86_64.tar.gz 
# ln -s /opt/mysql/mysql-5.7.25-linux-glibc2.12-x86_64.tar.gz mysql

添加 mysql 用户

# groupadd mysql
# useradd -g mysql  -d /usr/local/mysql -s /sbin/nologin -MN mysql

更改 mysql 目录权限

# chown -R mysql:mysql mysql/

创建数据库数据目录

在三个服务器上都创建这样的目录

# mkdir -p mgr/mysql3306/{data,logs,tmp}

更改目录权限

# chown -R mysql:mysql mgr

基本安装

配置文件说明

IP	端口号	配置文件
172.18.0.11	3306	/data/mgr/mysql3306/my3306.cnf
172.18.0.12	3306	/data/mgr/mysql3306/my3306.cnf
172.18.0.13	3306	/data/mgr/mysql3306/my3306.cnf

配置文件中注意参数

#group replication
gtid_mode=ON
enforce_gtid_consistency=ON

master_info_repository=TABLE
relay_log_info_repository=TABLE

binlog_checksum=NONE
log_slave_updates=ON
binlog_format=ROW

transaction_write_set_extraction=XXHASH64
loose-group_replication_group_name="3db33b36-0e51-409f-a61d-c99756e90155"
loose-group_replication_start_on_boot=off
loose-group_replication_local_address= "172.18.0.11:23306"
loose-group_replication_group_seeds= "172.18.0.11:23306,172.18.0.12:23307,172.18.0.13:23308"
loose-group_replication_bootstrap_group= off

## configure single-master OR multi-master
#loose-group_replication_single_primary_mode=off
#loose-group_replication_enforce_update_everywhere_checks=on

重要参数说明

group_replication_group_name

The name of the group which this server instance belongs to. Must be a valid UUID. This UUID is used internally when setting GTIDs for Group Replication events in the binary log.

Important

A unique UUID must be used.

可在命令行中通过 uuidgen 生成唯一的 UUID

# uuidgen
621f32ad-54e2-4f43-93b9-5808f463b931

group_replication_start_on_boot

Whether the server should start Group Replication or not during server start.

将 group_replication_start_on_boot 设置为 off,尽量手动操作集群节点的加入

group_replication_local_address

The network address which the member provides for connections from other members, specified as a host:port formatted string. This address must be reachable by all members of the group because it is used by XCOM, the internal group communication system.

Warning

Do not use this address for communication with the member.

Other Group Replication members contact this member through this host:port for all internal group communication. This is not the MySQL server SQL protocol host and port.

group_replication_group_seeds

A list of group members that provide a member which joins the group with the data required for the joining member to gain synchrony with the group. The list consists of the seed member’s network addresses specified as a comma separated list, such as host1:port1,host2:port2.

Important

These addresses must not be the member’s SQL hostname and port.

Usually this list consists of all members of the group, but you can choose a subset of the group members to be seeds. The list must contain at least one valid member address. Each address is validated when starting Group Replication. If the list does not contain any valid host names, issuing START GROUP_REPLICATION fails.

group_replication_bootstrap_group

Configure this server to bootstrap the group. This option must only be set on one server and only when starting the group for the first time or restarting the entire group. After the group has been bootstrapped, set this option to OFF. It should be set to OFF both dynamically and in the configuration files. Starting two servers or restarting one server with this option set while the group is running may lead to an artificial split brain situation, where two independent groups with the same name are bootstrapped.

group_replication_single_primary_mode

Instructs the group to automatically pick a single server to be the one that handles read/write workload. This server is the PRIMARY and all others are SECONDARIES.

group_replication_enforce_update_everywhere_checks

Enable or disable strict consistency checks for multi-primary update everywhere.

group_replication_allow_local_disjoin

Allow the current server to join the group even if it has transactions not present in the group.

Warning

Use caution when enabling this option as incorrect usage could lead to inconsistencies in the group.

新节点未作 reset master 处理，具有 GTID 信息,设置该参数为 1，允许带有 GTID 信息的节点加入集群

启动第一个节点

初始化第一个节点

# /usr/local/mysql/bin/mysqld --defaults-file=/data/mgr/mysql3306/my3306.cnf --initialize-insecure

启动第一个节点

# /usr/local/mysql/bin/mysqld --defaults-file=/data/mgr/mysql3306/my3306.cnf &

创建 group replication 通信账号

进入 MySQL

加载 group replication 的 plugin 2 种方式

1. 进入到 mysql 后，手动加入

mysql> install plugin group_replication soname 'group_replication.so';
mysql> show plugins;

2. 在 my.cnf 中配置好

# group replication plugin
plugin_load_add=group_replication.so

mysql> show plugins;
+----------------------------+----------+--------------------+----------------------+---------+
| Name                       | Status   | Type               | Library              | License |
+----------------------------+----------+--------------------+----------------------+---------+
| group_replication          | ACTIVE   | GROUP REPLICATION  | group_replication.so | GPL     |
+----------------------------+----------+--------------------+----------------------+---------+

mysql> set sql_log_bin=0;
mysql> create user group_repl_user@'172.18.0.%';
mysql> grant replication slave on *.* to group_repl_user@'172.18.0.%' identified by 'group_repl_password';
mysql> set sql_log_bin=1;

# channel 名称 group_replication_recovery 不能改变

mysql> change master to master_user='group_repl_user',
master_password='group_repl_password' for channel 'group_replication_recovery';

启动第一个节点的 Group Replication

使用第一个节点来引导集群（只在第一个节点使用）

mysql> set global group_replication_bootstrap_group = on;
mysql> start group_replication;
ysql> set global group_replication_bootstrap_group = off;

集群引导结束后，需要将 group_replication_bootstrap_group 设为 off

mysql> set global group_replication_bootstrap_group = off;

确认节点加入集群情况

mysql> select * from performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| group_replication_applier | 655d0632-7910-11e9-b24a-0242ac12000b | zst1        |        3306 | ONLINE       |
+---------------------------+--------------------------------------+-------------+-------------+--------------+

创建测试数据

Important

在 MGR 环境中建立的表必须要有主键，否则不允许向该表插入数据

mysql> create database test;
mysql> create table t1(id int not null auto_increment primary key, name varchar(10));
mysql> insert into t1(name) select 'a';
mysql> insert into t1(name) select 'b';
mysql> insert into t1(name) select 'c';

检索数据

mysql> select * from t1;
+----+------+
| id | name |
+----+------+
|  1 | a    |
|  2 | b    |
|  3 | c    |
+----+------+

启动第二个节点

# /usr/local/mysql/bin/mysqld --defaults-file=/data/mgr/mysql3306/my3306.cnf &

创建 group replication 通信账号

进入 MySQL

加载 group replication 的 plugin 2 种方式

1. 进入到 mysql 后，手动加入

mysql> install plugin group_replication soname 'group_replication.so';
mysql> show plugins;

2. 在 my.cnf 中配置好

# group replication plugin
plugin_load_add=group_replication.so

mysql> show plugins;
+----------------------------+----------+--------------------+----------------------+---------+
| Name                       | Status   | Type               | Library              | License |
+----------------------------+----------+--------------------+----------------------+---------+
| group_replication          | ACTIVE   | GROUP REPLICATION  | group_replication.so | GPL     |
+----------------------------+----------+--------------------+----------------------+---------+

mysql> set sql_log_bin=0;
mysql> create user group_repl_user@'172.18.0.%';
mysql> grant replication slave on *.* to group_repl_user@'172.18.0.%' identified by 'group_repl_password';
mysql> set sql_log_bin=1;

# channel 名称 group_replication_recovery 不能改变

mysql> change master to master_user='group_repl_user',
master_password='group_repl_password' for channel 'group_replication_recovery';

启动第二个节点的 Group Replication

使用第一个节点来引导集群（只在第一个节点使用）

mysql> start group_replication;

确认节点加入集群情况

mysql> select * from performance_schema.replication_group_members;

启动第三个节点

参考启动第二个节点

为有数据的 MGR 集群添加新的节点

1. 修改新节点的相关参数

   server_id
   loose-group_replication_local_address
   loose-group_replication_group_seeds
   

2. 初始化实例
3. 备份集群数据
4. 加载 plugin

   truncate table slave_worker_info
   

5. 还原数据
6. flush privileges 待定 (reset master)
7. set @@ global.gtid_pruged='xxx'
8. start group_replication;

测试

在三个节点上分别进行

3306：insert into t1(name) select '3306';
3307：insert into t1(name) select '3307';
3308: insert into t1(name) select '3308';

观察下哪个语句会执行成功，思考为什么？
MGR 默认是单主模式，也就是组中三个节点中只有一个节点可以写，剩余的两个节点只能读。

Group Replication 的两种工作模型

基于 single-master 环境

在 Group Replication 配置中，默认的模式是 Single-master 模式——集群中只有一个节点可进行写操作，其他节点是开启着：read-only 模式。

故障转换

当主节点宕机后，集群会从其他节点选举出新的 master.

故障切换流程

在 single-master 模式下，主节点挂掉后，会从集群中原来的节点选举出一个节点成为新的 primary 节点，而其他节点依然是 read-only 状态

查看集群中主节点

在 single-master 模式中可利用下面 SQL 来发现主节点

SQL 1

show status like 'group_replication_primary_member';

SQL 2

select * from performance_schema.global_status where variable_name = 'group_replication_primary_member';

基于 multi-master 环境

使用 multi-master 模式时，可配置为只写其中一个节点，以此避免发生故障。

该结构不是默认结构，如果要使用 multi-master 结构，需要在配置文件的 [mysqld] 部分添加以下配置

loose-group_replication_single_primary_mode=off
loose-group_replication_enforce_update_everywhere_checks=on

在 multi-master 结构中需要注意，不能同时在不同节点对同一行数据进行 update 操作，如果进行，客户端会收到报错。在使用这个结构时，需要应用自己去控制并行度。

故障转换

故障切换流程

在该结构中所有的节点上都可以进行读和写操作，任何一个节点挂掉后，可以从负载设备中自动摘除即可。例如(haproxy, proxysql 等等)

查看集群中主节点

在 multi-master 结构中，每个节点都可进行读写，因此不存所谓的 primary 节点，使用以下 SQL

select * from performance_schema.global_status where variable_name = 'group_replication_primary_member';

返回查询结果 group_replication_primary_member 为空

集群重启

集群重启分成两种情况

滚动重启

1. 节点启动后，执行：

   start group_replication;
   

2. 查询集群节点状态，确认节点状态无误即可

   select * from performance_schema.replication_group_members;
   

所有节点一块重启(集体掉电)

第一个节点启动 (single-master & multi-master 相同)

set global group_replication_bootstrap_group=on;
start group_replication;
set global group_replication_bootstrap_group=off;

其他节点启动

start group_replication;

MGR 崩溃场景

1. 大事务
2. DDL 和 DML 不在同一节点操作同一张表
3. 节点数据不一致，DML 在一运行，其他节点立马退出集群

MGR 监控

当前节点可用性监控

select * from performance_schema.replication_group_members where member_id = @@server_uuid;
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| group_replication_applier | ea468068-78a5-11e9-9caf-000c29aae378 | LinuxModel  |        3306 | ONLINE       |
+---------------------------+--------------------------------------+-------------+-------------+--------------+

查看当前节点是否可写

select * from performance_schema.global_variables where variable_name in ('read_only', 'super_read_only');
+-----------------+----------------+
| VARIABLE_NAME   | VARIABLE_VALUE |
+-----------------+----------------+
| read_only       | ON             |
| super_read_only | ON             |
+-----------------+----------------+

查看当前节点与集群的延迟情况

• 从远程获取的 GTID

select RECEIVED_TRANSACTION_SET from performance_schema.replication_connection_status where channel_name = 'group_replication_applier';

• 本节点执行的 GTID

select @@global.gtid_executed;

• 延迟的 GTID 数量

延迟的 GTID 数量 = 远程获取的 GTID - 本节点执行的 GTID

查看本节点执行队列是否有堆积（大于 0 表示有延迟）

select * from performance_schema.replication_group_member_stats where member_id=@@server_uuid

流控 (flow control)

开启流控

set group_replication_flow_control_mode=QUOTA

• group_replication_flow_control_applier_threshold(默认：25000)
• group_replication_flow_control_certifier_threshold(默认：25000)

关闭流控

set group_replication_flow_control_mode='DISABLED'

参数设置

slave_parallel_type=logical_clock
slave_parallel_workers=4|8
group_replication_compression_threshold=1000000~2000000