使用maxscale及mha搭建mysql主从同步、读写分离、高可用集群

尝试使用mysql存储部分数仓结果，需要满足高可用、读写分离等需求，尝试过mycat、atlas、maxscale这3个代理服务，前两个不适合：
mycat:新建数据库需要修改配置参数，这一点对于动态建库的场景很痛苦；
atlas:只支持mysql版本，你懂的。
最终选用maxscale，优点多缺点少，具体自行搜索。

so，使用mariadb10.3，maxscale2.4.8，mha 。

总体架构：

环境及版本：

centos 7.6
系统关闭selinux、防火墙、相互配置好ssh免密登录（mha必须）
mariadb10.3
maxscale2.4.8
mha4mysql-node-0.57-0.el7.noarch.rpm/mha4mysql-manager-0.57-0.el7.noarch.rpm
节点分配：
master、maxscale、mha: 192.168.1.10
slave1:192.168.1.11
slave2:192.168.1.12

一 、安装mariadb10.3

1、CentOS 7.6最小化方式安装系统后，其yum默认base源中的mariadb-server版本为5.5.60，且没有mariadb-client软件包

2、配置下载MariaDB的yum源：

 vi /etc/yum.repos.d/MariaDB.repo

[mariadb]

name=MariaDB Repo

baseurl=http://mirrors.ustc.edu.cn/mariadb/yum/10.3/centos7-amd64/

gpgkey=http://mirrors.ustc.edu.cn/mariadb/yum/RPM-GPG-KEY-MariaDB

gpgcheck=1

enabled=1

 yum clean all

 yum makecache

3、在新配置的mariadb源中MariaDB-server的最新版本为10.3.14，且MariaDB-client的最新版本也为10.3.14

 yum list MariaDB-server

4、安装启动MariaDB-server：

 yum -y install MariaDB-server

 systemctl start mariadb.service

 ss -tunlp | grep 3306

 systemctl enable mariadb.service

 systemctl status mariadb.service

 mysql_secure_installation

输入密码qingfeng@2020

5、简单修改配置文件server.cnf：

 vi /etc/my.cnf.d/server.cnf

[mysqld]

port=3306

socket=/var/lib/mysql/mysql.sock

datadir=/var/lib/mysql

log-error=/var/log/mariadb.log

lower_case_table_names=1

character-set-server=utf8mb4

collation-server=utf8mb4_unicode_ci

symbolic-links=0

备注：MariaDB Server 10.3.14的版本中默认主配置文件不再是/etc/my.cnf，而是my.cnf以及/etc/my.cnf.d/目录下的各类.cnf配置文件

 systemctl restart mariadb.service

6、授权远程用户登录：

 mysql -uroot -pqingfeng@2020

MariaDB> grant all on *.* to 'root'@'%' identified by 'qingfeng@2020';

MariaDB> flush privileges;

二、配置主从同步

1、主节点配置参数

vi /etc/my.cnf.d/server.cnf
Server ID
server-id =1
启动二进制日志
log-bin = mysql-bin
#Mysql有一些系统库不需要复制，可以配置多个
binlog-ignore-db = mysql 
binlog-ignore-db = information_schema
binlog-ignore-db = performance_schema


#设置logbin日志格式
binlog_format = STATEMENT

#从节点配置参数，注意id错开
Server ID
server-id =2
#启用中继日志
relay-log = mysql-relay
binlog-ignore-db = mysql 
binlog-ignore-db = information_schema
binlog-ignore-db = performance_schema

2、所有节点创建用户名为slave

MariaDB>CREATE USER 'slave' IDENTIFIED BY 'qingfeng@2020';
MariaDB>GRANT ALL PRIVILEGES ON *.* TO 'slave'@'%' IDENTIFIED BY 'qingfeng@2020' WITH GRANT OPTION;
刷新
MariaDB>flush privileges;

查看master状态

MariaDB>show master status;

将master信息填入,从节点执行

MariaDB>change master to master_host='192.168.1.10',
MariaDB>master_user='slave',
MariaDB>master_password='qingfeng@2020',
MariaDB>master_log_file='mysql-bin-2.000003上面显示的信息填入',
MariaDB>master_log_pos=上面显示的值;

3、从节点执行

MariaDB>start slave;
MariaDB>show slave status \G;

4、检查同步状态

MariaDB>show slave hosts;

三、安装配置maxscale

1、上传maxscale安装文件

#安装依赖
yum install gnutls -y
#官网下载安装包，也可以参考资料yum安装
rpm -ivh maxscale-2.4.8-1.centos.7.x86_64.rpm

#创建目录
mkdir -p /maxscale/cache
mkdir -p /maxscale/data
mkdir -p /maxscale/log
mkdir -p /maxscale/pid
mkdir -p /maxscale/tmp


#主从同步的情况下，在主节点执行即可
create user 'maxscale'@'%' identified by 'qingfeng@2020';
grant SELECT on MySQL.user to 'maxscale'@'%';
GRANT SELECT ON mysql.db TO 'maxscale'@'%';
GRANT all ON *.* TO 'maxscale'@'%';
GRANT SHOW DATABASES ON *.* TO 'maxscale'@'%';
grant REPLICATION CLIENT on *.* to 'maxscale'@'%';
GRANT replication slave, replication client,SELECT ON *.* TO maxscale@'%';

chown -R maxscale:maxscale /maxscale

vi /etc/maxscale.cnf

#MaxScale documentation:
#https://mariadb.com/kb/en/mariadb-maxscale-24/

#Global parameters
#
#Complete list of configuration options:
#https://mariadb.com/kb/en/mariadb-maxscale-24-mariadb-maxscale-configuration-guide/

[maxscale]
threads=auto

ms_timestamp=1             #timestamp精度  
syslog=1                   #将日志写入到syslog中    
maxlog=1                   #将日志写入到maxscale的日志文件中  
log_warning=1              #记录告警信息  
log_notice=1               #记录notice  
log_info=1                 #记录info  
log_debug=0                #不打开debug模式  
log_augmentation=1         #日志递增

logdir=/maxscale/log/  
datadir=/maxscale/data/  
libdir=/usr/lib64/maxscale/  
cachedir=/maxscale/cache/  
piddir=/maxscale/pid/
execdir=/usr/bin/


[server1]
type=server
address=192.168.1.10
port=3306
protocol=MariaDBBackend
serv_weight=1

[server2]
type=server
address=192.168.1.11
port=3306
protocol=MariaDBBackend
serv_weight=3

[server3]
type=server
address=192.168.1.12
port=3306
protocol=MariaDBBackend
serv_weight=3

#Monitor for the servers
#
#This will keep MaxScale aware of the state of the servers.
#MariaDB Monitor documentation:
#https://mariadb.com/kb/en/mariadb-maxscale-24-mariadb-monitor/

[MariaDB-Monitor]
type=monitor
module=mariadbmon
servers=server1,server2,server3
user=maxscale
password=qingfeng@2020
monitor_interval=2000
detect_stale_master=false

#Service definitions
#
#Service Definition for a read-only service and
#a read/write splitting service.
#

#ReadConnRoute documentation:
#https://mariadb.com/kb/en/mariadb-maxscale-24-readconnroute/

[Read-Only-Service]
type=service
router=readconnroute
servers=server1,server2,server3
user=maxscale
password=qingfeng@2020
router_options=slave
enable_root_user=1
weightby=serv_weight

#ReadWriteSplit documentation:
#https://mariadb.com/kb/en/mariadb-maxscale-24-readwritesplit/

[Read-Write-Service]
type=service
router=readwritesplit
servers=server1,server2,server3
user=maxscale
password=qingfeng@2020
max_slave_connections=100%
use_sql_variables_in=master
enable_root_user=1
max_slave_replication_lag=3600

#Listener definitions for the services
#
#These listeners represent the ports the
#services will listen on.
#

[Read-Only-Listener]
type=listener
service=Read-Only-Service
protocol=MariaDBClient
port=4008

[Read-Write-Listener]
type=listener
service=Read-Write-Service
protocol=MariaDBClient
port=4006

#注意看这里名称中有-，没有会报错
[MaxAdmin-Service]
type=service
router=cli

#注意看这里名称中有-，没有会报错
[MaxAdmin-Listener]
type=listener
service=MaxAdmin-Service
protocol=maxscaled
port=4099

2、启动服务

maxscale -f /etc/maxscale.cnf -U maxscale

#解决error 1577报错问题–因环境之前安装了mariadb5.5导致和10.3系统表结构不一致

mysqlcheck -u root -pqingfeng@2020 --all-databases --check-upgrade --auto-repair
mysql_upgrade -u root -pqingfeng@2020

3、管理工具登录

maxadmin -P4099 -uadmin -pmariadb
list servers

4、测试验证

create databaese qingfeng_test;

CREATE TABLE IF NOT EXISTS qingfeng_test.test(
	id INT NOT NULL,
    name VARCHAR(20) NOT NULL,
    PRIMARY KEY (`id`)
)ENGINE=InnoDB DEFAULT CHARSET=utf8;

INSERT INTO `qingfeng_test`.`test` (`id`, `name`) VALUES ('1', '旺财');
INSERT INTO `qingfeng_test`.`test` (`id`, `name`) VALUES ('2', '六一');
INSERT INTO `qingfeng_test`.`test` (`id`, `name`) VALUES ('3', '豪礼');

四、安装配置mha

1、安装yum 扩展包

yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

yum install perl-ExtUtils-CBuilder perl-ExtUtils-MakeMaker perl-Config-Tiny perl-Log-Dispatch-* perl-Parallel-ForkManager

yum install perl-DBD-mysql perl-DBI

2、下载后所有节点安装

rpm -ivh mha4mysql-node-0.57-0.el7.noarch.rpm

3、主节点安装

rpm -ivh mha4mysql-manager-0.57-0.el7.noarch.rpm

#mha需要：开启binlog,relay-log,关闭relay_log_purge

show variables like 'rpl_semi_sync%';

4、所有节点修改/etc/my.cnf.d/server.cnf文件

#跳过dns解析，只能用ip访问
skip_name_resolve = ON
#每个表单独保存文件
innodb_file_per_table = ON
#关闭自动清空中继日志
relay_log_purge = 0
rpl_semi_sync_master_enabled = 1
rpl_semi_sync_master_timeout = 3000
rpl_semi_sync_slave_enabled = 1

5、配置mha参数

mkdir -p /etc/mha_master
vi /etc/mha_master/mha.cnf
[server default]
user=slave
password=qingfeng@2020
manager_workdir=/etc/mha_master/app1
manager_log=/etc/mha_master/manager.log
remote_workdir=/data/mha_master/app1
repl_user=slave
repl_password=qingfeng@2020
ping_interval=1

[server1]
hostname=192.168.1.10
ssh_port=22

[server2]
hostname=192.168.1.11
ssh_port=22
candidate_master=1

[server3]
hostname=192.168.1.12
ssh_port=22

6、检测ssh

masterha_check_ssh -conf=/etc/mha_master/mha.cnf

7、检测主从同步状态

masterha_check_repl --conf=/etc/mha_master/mha.cnf
#如果主节点不小心执行了start slave，会检测通不过，主节点执行以下命令解决：
stop slave;
reset slave all;

8、重新执行检测到OK,启动mha监控服务

nohup masterha_manager --conf=/etc/mha_master/mha.cnf &> /etc/mha_master/manager.log &

#检查master节点状态
masterha_check_status --conf=/etc/mha_master/mha.cnf

#故障自动转移后masterha_manager会退出，因此只能解决一次master挂掉
#需要恢复故障服务后，才能再次启动mha监控服务
#故障的master只能以slave重新加入集群
#默认情况下，8个小时以内不允许再次转移，除非打开ignore_last_failover参数，或者删除上次故障转移记录文件：
rm -rf /etc/mha_master/app1/mha.failover.complete

至此基本安装配置完成，可以建库、建表、插入数据、查询，关闭master节点服务测试mha故障转移等。
对外只使用maxscale所在节点IP：192.168.1.10
读写访问端口 4006 （提供给应用使用）
只读访问端口 4008 （暂时没用上）