MySQL MGR 集群搭建（单主模式&多主模式）

1. 背景

MySQL Group Replication（MGR）是MySQL官方在5.7.17版本引进的一个数据库高可用与高扩展的解决方案，以插件形式提供，实现了分布式下数据的最终一致性，总结MGR特点如下：

• 高一致性：基于分布式paxos协议实现组复制，保证数据一致性；
• 高容错性：自动检测机制，只要不是大多数节点都宕机就可以继续工作，内置防脑裂保护机制；
• 高扩展性：节点的增加与移除会自动更新组成员信息，新节点加入后，自动从其他节点同步增量数据，直到与其他节点数据一致；
• 高灵活性：提供单主模式和多主模式，单主模式在主库宕机后能够自动选主，所有写入都在主节点进行，多主模式支持多节点写入。

2. 环境：

3台服务器搭建3节点MGR集群，MySQL版本8.0.11，操作系统版本CentOS 7.4.

• 10.253.3.101
• 10.253.3.102
• 10.253.3.103

3. 安装步骤

3.1 安装MySQL8.0（所有节点）

MySQL官方版8.0.11，采用Linux Generic安装方式，直接下载压缩包，解压到目录/usr/local/mysql8.0。

解压文件

tar -zxvf mysql-8.0.11-linux-glibc2.12-x86_64.tar.gz

mkdir -p /usr/local/mysql8.0

mv mysql-8.0.11-linux-glibc2.12-x86_64/* /usr/local/mysql8.0

 

3.2 修改配置文件（所有节点）

编辑配置文件 /home/mysql/etc/my_mgr_8.0.cnf，3个节点除了server_id、loose-group_replication_local_address、report_host 三个参数不一样外，其他保持一致。

mkdir -p /home/mysql/etc/

vi /home/mysql/etc/my_mgr_8.0.cnf

 

# /home/mysql/etc/my_mgr_8.0.cnf

[mysqld]

port=3306

basedir=/usr/local/mysql8.0

datadir=/mysql/data_mgr_8.0/

socket=/mysql/data_mgr_8.0/mysql.sock

pid_file=/mysql/data_mgr_8.0/mysql.pid

 

server_id=101

gtid_mode=ON

enforce_gtid_consistency=ON

binlog_checksum=NONE

 

log_bin=binlog

log_slave_updates=ON

binlog_format=ROW

master_info_repository=TABLE

relay_log_info_repository=TABLE

 

transaction_write_set_extraction=XXHASH64

loose-group_replication_group_name="aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"

loose-group_replication_start_on_boot=OFF

loose-group_replication_local_address= "pxc1:33061"

loose-group_replication_group_seeds= "pxc1:33061,pxc2:33061,pxc3:33061"

loose-group_replication_bootstrap_group=OFF

 

report_host=pxc1

report_port=3306

3.3 初始化数据库（所有节点执行）

useradd mysql(我的系统本来就有mysql用户)

 

mkdir -p /mysql/data_mgr_8.0

chown -R mysql.mysql /mysql/data_mgr_8.0/

chown -R mysql.mysql /home/mysql/etc/

/usr/local/mysql8.0/bin/mysqld --initialize-insecure --basedir=/usr/local/mysql8.0 --datadir=/mysql/data_mgr_8.0 --user=mysql

2019-03-24T10:06:54.471521Z 0 [System] [MY-013169] [Server] /usr/local/mysql8.0/bin/mysqld (mysqld 8.0.11) initializing of server in progress as process 53872

2019-03-24T10:06:55.896530Z 5 [Warning] [MY-010453] [Server] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.

2019-03-24T10:06:56.896915Z 0 [System] [MY-013170] [Server] /usr/local/mysql8.0/bin/mysqld (mysqld 8.0.11) initializing of server has completed

3.4 启动数据库，安装MGR插件，设置复制账号（所有节点执行）

# 启动数据库

su - mysql -c "/usr/local/mysql8.0/bin/mysqld_safe --defaults-file=/home/mysql/etc/my_mgr_8.0.cnf &"

数据库报错日志：

tail -100f /mysql/data_mgr_8.0/pxc1.err

tail -100f /mysql/data_mgr_8.0/pxc2.err

tail -100f /mysql/data_mgr_8.0/pxc3.err

 

# 登录数据库

/usr/local/mysql8.0/bin/mysql -S /mysql/data_mgr_8.0/mysql.sock

 

# 安装MGR插件

mysql>INSTALL PLUGIN group_replication SONAME 'group_replication.so';

 

#设置复制账号

mysql> SET SQL_LOG_BIN=0;

mysql> CREATE USER repl@'%' IDENTIFIED BY 'repl';

mysql> GRANT REPLICATION SLAVE ON *.* TO repl@'%';

mysql> FLUSH PRIVILEGES;

mysql> SET SQL_LOG_BIN=1;

mysql> CHANGE MASTER TO MASTER_USER='repl', MASTER_PASSWORD='repl' FOR CHANNEL 'group_replication_recovery';

3.5 启动MGR单主模式

# 启动MGR，在主库(10.253.3.101)上执行

mysql> SET GLOBAL group_replication_bootstrap_group=ON;

mysql> START GROUP_REPLICATION;

mysql> SET GLOBAL group_replication_bootstrap_group=OFF;

 

# 查看MGR组信息

mysql> SELECT * FROM performance_schema.replication_group_members;（101节点看）

+---------------------------+--------------------------------------+--------------+-------------+--------------+-------------+----------------+

| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST  | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION |

+---------------------------+--------------------------------------+--------------+-------------+--------------+-------------+----------------+

| group_replication_applier | 8daa9f72-4e1c-11e9-8bb6-000c295b4eb7 | 10.253.3.101 |        3306 | ONLINE       | PRIMARY     | 8.0.11         |

+---------------------------+--------------------------------------+--------------+-------------+--------------+-------------+----------------+

 

# 其他节点加入MGR，在从库（10.253.3.102，10.253.3.103）上执行

mysql> START GROUP_REPLICATION;

 

 

# 查看MGR组信息(三个节点皆可看)

mysql> SELECT * FROM performance_schema.replication_group_members;

+---------------------------+--------------------------------------+--------------+-------------+--------------+-------------+----------------+

| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST  | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION |

+---------------------------+--------------------------------------+--------------+-------------+--------------+-------------+----------------+

| group_replication_applier | 8daa9f72-4e1c-11e9-8bb6-000c295b4eb7 | 10.253.3.101 |        3306 | ONLINE       | PRIMARY     | 8.0.11         |

| group_replication_applier | bffc8972-4e1c-11e9-a949-000c29124ee9 | 10.253.3.102 |        3306 | ONLINE       | SECONDARY   | 8.0.11         |

| group_replication_applier | c0cde392-4e1c-11e9-b99e-000c291fa60d | 10.253.3.103 |        3306 | ONLINE       | SECONDARY   | 8.0.11         |

+---------------------------+--------------------------------------+--------------+-------------+--------------+-------------+----------------+

可以看到，3个节点状态为online，并且主节点为10.253.3.101，只有主节点可以写入，其他节点只读，MGR单主模式搭建成功。

select * from performance_schema.replication_connection_status\G;

3.6 切换到多主模式

MGR切换模式需要重新启动组复制，因些需要在所有节点上先关闭组复制，设置 group_replication_single_primary_mode=OFF 等参数，再启动组复制。

# 停止组复制(所有节点执行)：

mysql> stop group_replication;

mysql> set global group_replication_single_primary_mode=OFF;

mysql> set global group_replication_enforce_update_everywhere_checks=ON;

 

# 随便选择某个节点执行

mysql> SET GLOBAL group_replication_bootstrap_group=ON;

mysql> START GROUP_REPLICATION;

mysql> SET GLOBAL group_replication_bootstrap_group=OFF;

 

# 其他节点执行

mysql> START GROUP_REPLICATION;

 

# 查看组信息，所有节点的 MEMBER_ROLE 都为 PRIMARY

mysql>  SELECT * FROM performance_schema.replication_group_members;

+---------------------------+--------------------------------------+--------------+-------------+--------------+-------------+----------------+

| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST  | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION |

+---------------------------+--------------------------------------+--------------+-------------+--------------+-------------+----------------+

| group_replication_applier | 8daa9f72-4e1c-11e9-8bb6-000c295b4eb7 | 10.253.3.101 |        3306 | ONLINE       | PRIMARY     | 8.0.11         |

| group_replication_applier | bffc8972-4e1c-11e9-a949-000c29124ee9 | 10.253.3.102 |        3306 | ONLINE       | PRIMARY     | 8.0.11         |

| group_replication_applier | c0cde392-4e1c-11e9-b99e-000c291fa60d | 10.253.3.103 |        3306 | ONLINE       | PRIMARY     | 8.0.11         |

+---------------------------+--------------------------------------+--------------+-------------+--------------+-------------+----------------+

3 rows in set (0.00 sec)

可以看到所有节点状态都是online，角色都是PRIMARY，MGR多主模式搭建成功。

3.7 切回单主模式

# 所有节点执行

mysql> stop group_replication;

mysql> set global group_replication_enforce_update_everywhere_checks=OFF;

mysql> set global group_replication_single_primary_mode=ON;

 

# 主节点（10.253.3.101）执行

SET GLOBAL group_replication_bootstrap_group=ON;

START GROUP_REPLICATION;

SET GLOBAL group_replication_bootstrap_group=OFF;

 

 

# 从节点（10.253.3.102、10.253.3.103）执行

START GROUP_REPLICATION;

 

# 查看MGR组信息

mysql> SELECT * FROM performance_schema.replication_group_members;

+---------------------------+--------------------------------------+--------------+-------------+--------------+-------------+----------------+

| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST  | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION |

+---------------------------+--------------------------------------+--------------+-------------+--------------+-------------+----------------+

| group_replication_applier | 8daa9f72-4e1c-11e9-8bb6-000c295b4eb7 | 10.253.3.101 |        3306 | ONLINE       | PRIMARY     | 8.0.11         |

| group_replication_applier | bffc8972-4e1c-11e9-a949-000c29124ee9 | 10.253.3.102 |        3306 | ONLINE   | SECONDARY   | 8.0.11         |

| group_replication_applier | c0cde392-4e1c-11e9-b99e-000c291fa60d | 10.253.3.103 |        3306 | ONLINE   | SECONDARY   | 8.0.11         |

+---------------------------+--------------------------------------+--------------+-------------+--------------+-------------+----------------+

3 rows in set (0.00 sec)

 

更改root密码

select host,user,authentication_string from mysql.user;

use mysql;

ALTER user 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'Qaz123$%^';

更改密码后，登陆方式必须如下（更改密码，针对所有节点生效）：

[root@pxc1 mysql]# /usr/local/mysql8.0/bin/mysql -S /mysql/data_mgr_8.0/mysql.sock

ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)

[root@pxc1 mysql]# /usr/local/mysql8.0/bin/mysql -S /mysql/data_mgr_8.0/mysql.sock -p

Enter password:

 

开启远程连接

CREATE USER 'root'@'%' IDENTIFIED WITH mysql_native_password BY 'Qaz123$%^';

GRANT ALL ON *.* TO 'root'@'%';

flush privileges;

 

如果不使用WITH mysql_native_password，navicat远程连接会报错：
navicat连接mysql报错1251解决方案

 

这期间还涉及MySQL的密码认证插件是mysql_native_password，而现在使用的是caching_sha2_password。

因为当前有很多数据库工具和链接包都不支持caching_sha2_password，为了方便，我暂时还是改回了mysql_native_password认证插件。

在MySQL中执行命令：

ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'password';
修改密码验证插件，同时修改密码。

如果想默认使用mysql_native_password插件认证，可以在配置文件中配置default_authentication_plugin项，这个在新建my.ini中已经设置过了。 参考链接

[mysqld]

default_authentication_plugin=mysql_native_password

 

 

 

问题：

1. 2019-03-24T12:04:38.714331Z 0 [ERROR] [MY-011526] [Repl] Plugin group_replication reported: 'This member has more executed transactions than those present in the group. Local transactions: c0cde392-4e1c-11e9-b99e-000c291fa60d:1 > Group transactions: 8daa9f72-4e1c-11e9-8bb6-000c295b4eb7:1,

aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1-35'

2019-03-24T12:04:38.714550Z 0 [ERROR] [MY-011522] [Repl] Plugin group_replication reported: 'The member contains transactions not present in the group. The member will now exit the group.'

在出现问题的从库中：RESET MASTER

此命令据官方文档描述，含义为

RESET MASTER enables you to delete any binary log files and their related binary log index file, returning the master to its state before binary logging was started.

然后再把组复制的命令配置一遍就OK了。

 

问题2：102/103节点启动后，一直是RECOVERING状态，

mysql> SELECT * FROM performance_schema.replication_group_members;

+---------------------------+--------------------------------------+--------------+-------------+--------------+-------------+----------------+

| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST  | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION |

+---------------------------+--------------------------------------+--------------+-------------+--------------+-------------+----------------+

| group_replication_applier | 8daa9f72-4e1c-11e9-8bb6-000c295b4eb7 | 10.253.3.101 |        3306 | ONLINE       | PRIMARY     | 8.0.11         |

| group_replication_applier | bffc8972-4e1c-11e9-a949-000c29124ee9 | 10.253.3.102 |        3306 | RECOVERING   | SECONDARY   | 8.0.11         |

| group_replication_applier | c0cde392-4e1c-11e9-b99e-000c291fa60d | 10.253.3.103 |        3306 | RECOVERING   | SECONDARY   | 8.0.11         |

+---------------------------+--------------------------------------+--------------+-------------+--------------+-------------+----------------+

解决办法是，将IP地址改为主机名（三节点都改），在/etc/host文件中配置IP与主机名映射关系 

loose-group_replication_local_address= "pxc2:33061"

loose-group_replication_group_seeds= "pxc1:33061,pxc2:33061,pxc3:33061"

loose-group_replication_bootstrap_group=OFF

 

report_host=pxc2

report_port=3306