CentOS 7 安装 Percona XtraDB Cluster(PXC) 5.6,记录一下安装过程。 主要安装过程 Percona
的官方文档 Percona XtraDB Cluster 5.6 Documentation 已经有描述,本次记录一下步骤。 本文使用
MySQL 5.6 版本进行集群为例
CentOS 7(Minimal Install)
$ cat /etc/centos-release
CentOS Linux release 7.7.1908 (Core)
配置如下
Node | Host | IP | Descprition |
---|---|---|---|
Node1 | pxc1 | 192.168.50.10 | 集群节点1 |
Node2 | pxc2 | 192.168.50.11 | 集群节点2 |
Node3 | pxc3 | 192.168.50.12 | 集群节点3 |
Haproxy | haproxy | 192.168.50.13 | 负载均衡节点 |
本教程中描述的过程需要满足以下条件:
• 所有三个节点都安装了CentOS 7。
• 所有节点上的防火墙都配置为允许连接到端口3306、4444、4567和4568。
• 所有节点上的SELinux被禁用。
sudo firewall-cmd --permanent --add-port={3306/tcp,4444/tcp,4567/tcp,4568/tcp}
success
sudo firewall-cmd --reload
success
重载之后,查看一下防火墙状态
$ sudo firewall-cmd --list-all
public (active)
target: default
icmp-block-inversion: no
interfaces: eth0
sources:
services: dhcpv6-client ssh
ports: 3306/tcp 4444/tcp 4567/tcp 4568/tcp
protocols:
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
在SELinux的安全模块可以限制访问,为的Percona XtraDB Cluster数据。最好解决方案是从改变模式enforcing
以permissive通过运行下面的命令:
$ setenforce 0
这仅在运行时更改模式。要在重启后以许可模式运行SELinux,请SELINUX=permissive在/etc/selinux/config 配置文件中进行设置。
1、配置 yum 源如下,本文以 pxc1 为例,其他 pxc2, pxc3 相同
$ sudo yum install https://repo.percona.com/yum/percona-release-latest.noarch.rpm
2、安装 Percona XtraDB Cluster 的包
$ sudo yum install Percona-XtraDB-Cluster-56
...
...
-------------
* The suggested mysql options and settings are in /etc/percona-xtradb-cluster.conf.d/mysqld.cnf
* If you want to use mysqld.cnf as default configuration file please make backup of /etc/my.cnf
* Once it is done please execute the following commands:
rm -rf /etc/my.cnf
update-alternatives --install /etc/my.cnf my.cnf "/etc/percona-xtradb-cluster.cnf" 200
-------------
Percona XtraDB Cluster is distributed with several useful UDFs from Percona Toolkit.
Run the following commands to create these functions:
mysql -e "CREATE FUNCTION fnv1a_64 RETURNS INTEGER SONAME 'libfnv1a_udf.so'"
mysql -e "CREATE FUNCTION fnv_64 RETURNS INTEGER SONAME 'libfnv_udf.so'"
mysql -e "CREATE FUNCTION murmur_hash RETURNS INTEGER SONAME 'libmurmur_udf.so'"
See http://www.percona.com/doc/percona-server/5.6/management/udf_percona_toolkit.html for more details
...
...
Replaced:
mariadb-libs.x86_64 1:5.5.60-1.el7_5
Complete!
安装过程中会出现如下提示,可以根据需要进行配置。
3、启动Percona XtraDB群集服务器
$ sudo service mysql start
4、登录mysql
$ mysql
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 5
Server version: 5.6.48-88.0-56 Percona XtraDB Cluster (GPL), Release rel88.0, Revision 6faf3b7, WSREP version 28.40, wsrep_28.40
Copyright (c) 2009-2020 Percona LLC and/or its affiliates
Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
5、创建SST账号
mysql> CREATE USER 'sstuser'@'%' IDENTIFIED BY 'password';
Query OK, 0 rows affected (0.00 sec)
6、赋予SST账号权限
mysql> GRANT RELOAD,LOCK TABLES,REPLICATION CLIENT,PROCESS ON *.* TO 'sstuser'@'%';
Query OK, 0 rows affected (0.00 sec)
mysql>flush privileges;
7、关闭mysql服务
$ sudo systemctl stop mysql
应该将各个节点配置为能够引导群集。有关引导群集的更多信息,请参见引导第一个节点。
# Template my.cnf for PXC
# Edit to your requirements.
[mysqld]
log-error=/var/log/mysqld.log
# 指定二进制日志的格式
binlog_format = ROW
innodb_buffer_pool_size = 100M
innodb_flush_log_at_trx_commit = 0
innodb_flush_method = O_DIRECT
innodb_log_files_in_group = 2
innodb_log_file_size = 20M
innodb_file_per_table = 1
datadir = /var/lib/mysql
user=mysql
#PXC集群的名称和所有服务地址
wsrep_cluster_name = pxc-cluster
wsrep_cluster_address = gcomm://192.168.50.10,192.168.11.35,192.168.50.12
wsrep_provider = /usr/lib64/galera3/libgalera_smm.so
wsrep_slave_threads = 8
#当前节点的名称和服务地址
wsrep_node_name = pxc-node1
wsrep_node_address = 192.168.50.10
#指定同步方法和账户信息,这个用户在下文会进行创建
wsrep_sst_method=xtrabackup-v2
wsrep_sst_auth=admin:admin123
#开启严厉模式,它会阻止用户执行 Percona XtraDB Cluster 所不支持的功能。
#pxc_strict_mode=ENFORCING
# 指定默认的存储引擎
default_storage_engine=InnoDB
# 该模式下所有 INSERT SQL 都不会有表级 AUTO-INC 锁,多个语句可以同时执行
innodb_autoinc_lock_mode = 2
[mysqld_safe]
pid-file = /run/mysqld/mysql.pid
syslog
!includedir /etc/my.cnf.d
$ sudo systemctl start mysql@bootstrap.service
上一条命令将启动群集,并将初始wsrep_cluster_address变量设置为gcomm://。如果稍后重新启动节点或MySQL,则无需更改配置文件。
mysql> show status like 'wsrep%';
+----------------------------+--------------------------------------+
| Variable_name | Value |
+----------------------------+--------------------------------------+
| wsrep_local_state_uuid | c2883338-834d-11e2-0800-03c9c68e41ec |
...
| wsrep_local_state | 4 |
| wsrep_local_state_comment | Synced |
...
| wsrep_cluster_size | 1 |
| wsrep_cluster_status | Primary |
| wsrep_connected | ON |
...
| wsrep_ready | ON |
+----------------------------+--------------------------------------+
75 rows in set (0.00 sec)
此输出表明集群已成功引导。
# Template my.cnf for PXC
# Edit to your requirements.
[mysqld]
log-error=/var/log/mysqld.log
# 指定二进制日志的格式
binlog_format = ROW
innodb_buffer_pool_size = 100M
innodb_flush_log_at_trx_commit = 0
innodb_flush_method = O_DIRECT
innodb_log_files_in_group = 2
innodb_log_file_size = 20M
innodb_file_per_table = 1
datadir = /var/lib/mysql
user=mysql
#PXC集群的名称和所有服务地址
wsrep_cluster_name = pxc-cluster
wsrep_cluster_address = gcomm://192.168.50.10,192.168.11.35,192.168.50.12
wsrep_provider = /usr/lib64/galera3/libgalera_smm.so
wsrep_slave_threads = 8
#当前节点的名称和服务地址
wsrep_node_name = pxc-node2
wsrep_node_address = 192.168.50.11
#指定同步方法和账户信息,这个用户在下文会进行创建
wsrep_sst_method=xtrabackup-v2
wsrep_sst_auth=admin:admin123
#开启严厉模式,它会阻止用户执行 Percona XtraDB Cluster 所不支持的功能。
#pxc_strict_mode=ENFORCING
# 指定默认的存储引擎
default_storage_engine=InnoDB
# 该模式下所有 INSERT SQL 都不会有表级 AUTO-INC 锁,多个语句可以同时执行
innodb_autoinc_lock_mode = 2
[mysqld_safe]
pid-file = /run/mysqld/mysql.pid
syslog
!includedir /etc/my.cnf.d
$ systemctl start mysql
mysql> show status like 'wsrep%';
+----------------------------+--------------------------------------+
| Variable_name | Value |
+----------------------------+--------------------------------------+
| wsrep_local_state_uuid | c2883338-834d-11e2-0800-03c9c68e41ec |
...
| wsrep_local_state | 4 |
| wsrep_local_state_comment | Synced |
...
| wsrep_cluster_size | 2 |
| wsrep_cluster_status | Primary |
| wsrep_connected | ON |
...
| wsrep_ready | ON |
+----------------------------+--------------------------------------+
40 rows in set (0.01 sec)
此输出表明新节点已成功添加到集群中。
# Template my.cnf for PXC
# Edit to your requirements.
[mysqld]
log-error=/var/log/mysqld.log
# 指定二进制日志的格式
binlog_format = ROW
innodb_buffer_pool_size = 100M
innodb_flush_log_at_trx_commit = 0
innodb_flush_method = O_DIRECT
innodb_log_files_in_group = 2
innodb_log_file_size = 20M
innodb_file_per_table = 1
datadir = /var/lib/mysql
user=mysql
#PXC集群的名称和所有服务地址
wsrep_cluster_name = pxc-cluster
wsrep_cluster_address = gcomm://192.168.50.10,192.168.11.35,192.168.50.12
wsrep_provider = /usr/lib64/galera3/libgalera_smm.so
wsrep_slave_threads = 8
#当前节点的名称和服务地址
wsrep_node_name = pxc-node3
wsrep_node_address = 192.168.50.12
#指定同步方法和账户信息,这个用户在下文会进行创建
wsrep_sst_method=xtrabackup-v2
wsrep_sst_auth=admin:admin123
#开启严厉模式,它会阻止用户执行 Percona XtraDB Cluster 所不支持的功能。
#pxc_strict_mode=ENFORCING
# 指定默认的存储引擎
default_storage_engine=InnoDB
# 该模式下所有 INSERT SQL 都不会有表级 AUTO-INC 锁,多个语句可以同时执行
innodb_autoinc_lock_mode = 2
[mysqld_safe]
pid-file = /run/mysqld/mysql.pid
syslog
!includedir /etc/my.cnf.d
$ sudo systemctl start mysql
mysql> show status like 'wsrep%';
+----------------------------+--------------------------------------+
| Variable_name | Value |
+----------------------------+--------------------------------------+
| wsrep_local_state_uuid | c2883338-834d-11e2-0800-03c9c68e41ec |
...
| wsrep_local_state | 4 |
| wsrep_local_state_comment | Synced |
...
| wsrep_cluster_size | 3 |
| wsrep_cluster_status | Primary |
| wsrep_connected | ON |
...
| wsrep_ready | ON |
+----------------------------+--------------------------------------+
40 rows in set (0.01 sec)
此输出表明新节点已成功添加到集群中。
参数名 | 说明 |
---|---|
wsrep_provider | 指定 Galera 库的路径 |
wsrep_cluster_name | 指定集群的逻辑名称,集群内的所有节点,这个名称必须一致 |
wsrep_cluster_address | 指定集群内节点的 IP 地址,建议将集群节点都配上 |
wsrep_node_name | 指定单个节点的逻辑名称,如果没有指定,将使用 hostname 作为逻辑名称 |
wsrep_node_address | 指定此特定节点的 IP 地址 |
wsrep_sst_method | 默认的使用 Percona Xtrabackup 进行 State Snapshot Transfer (SST),强烈建议使用 |
wsrep_sst_auth | 指定 sst 的身份验证凭据,使用 |
pxc_strict_mode | 关闭实验性的或者不支持的特性 |
binlog_format | Galera 只支持 row-level replication,所以设置为 binlog_format=ROW |
pxc_strict_mode | 关闭实验性的或者不支持的特性 |
default_storage_engine | Galera 只支持 InnoDB 引擎,所以设置为 default_storage_engine=InnoDB |
innodb_autoinc_lock_mode | Galera 只支持 lock mode 为 2 的 InnoDB 引擎,所以设置为 innodb_autoinc_lock_mode=2 |
HAProxy是高性能TCP(第四层)/HTTP(第七层)反向代理负载均衡服务器。(The Reliable, High Performance TCP/HTTP Load Balancer)
$ yum list | grep haproxy
haproxy.x86_64 1.5.18-9.el7 base
pcp-pmda-haproxy.x86_64 4.3.2-7.el7_8 updates
$ yum -y install haproxy
$ rpm -qi haproxy
$ vim /etc/haproxy/haproxy.cfg
#---------------------------------------------------------------------
# Example configuration for a possible web application. See the
# full configuration options online.
#
# http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
#
#---------------------------------------------------------------------
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
# to have these messages end up in /var/log/haproxy.log you will
# need to:
#
# 1) configure syslog to accept network log events. This is done
# by adding the '-r' option to the SYSLOGD_OPTIONS in
# /etc/sysconfig/syslog
#
# 2) configure local2 events to go to the /var/log/haproxy.log
# file. A line like the following can be added to
# /etc/sysconfig/syslog
#
# local2.* /var/log/haproxy.log
#
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
# turn on stats unix socket
stats socket /var/lib/haproxy/stats
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode http
log global
option httplog
option dontlognull
option http-server-close
#option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000
#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
frontend main *:5000
acl url_static path_beg -i /static /images /javascript /stylesheets
acl url_static path_end -i .jpg .gif .png .css .js
use_backend static if url_static
default_backend app
#---------------------------------------------------------------------
# static backend for serving up images, stylesheets and such
#---------------------------------------------------------------------
backend static
balance roundrobin
server static 127.0.0.1:4331 check
#---------------------------------------------------------------------
# round robin balancing between the various backends
#---------------------------------------------------------------------
backend app
balance roundrobin
server app1 127.0.0.1:5001 check
server app2 127.0.0.1:5002 check
server app3 127.0.0.1:5003 check
server app4 127.0.0.1:5004 check
#配置web监控页面
listen admin_stats
bind *:8080 #监听端口
mode http
option httplog
log global
stats enable #统计接口启用开关
maxconn 10
stats refresh 30s #页面刷新时长
stats uri /haproxy?stats #haproxy ui访问后缀
stats realm haproxy #认证时的realm,作为提示用的
stats auth admin:admin #认证用户名和密码
stats hide-version #隐藏HAProxy版本号
stats admin if TRUE #管理界面只有认证通过后才能在ui上进行管理
#负载均衡配置
listen mysql_cluster
bind *:3306 #监听端口
mode tcp
#option mysql-check
option tcplog
balance roundrobin #调度算法
# inter 2000 健康检查时间间隔2秒
# rise 3 检测多少次才认为是正常的
# fall 3 失败多少次才认为是不可用的
server node1 192.168.50.173:3306 check inter 2000 rise 3 fall 3
server node2 192.168.50.35:3306 check inter 2000 rise 3 fall 3
server node3 192.168.50.108:3306 check inter 2000 rise 3 fall 3
$ haproxy -f /etc/haproxy/haproxy.cfg -c
$ service haproxy start
$ service haproxy status
http://192.168.50.13:8080/haproxy?stats
访问数据库IP:192.168.50.13 端口:3306 即可实现数据库负载均衡
2020-07-28 15:00:01 17227 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '192.168.100.135' --auth 'root:root123' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --parent '17227' '' : 32 (Broken pipe)
2020-07-28 15:00:01 17227 [ERROR] WSREP: Failed to read uuid:seqno from joiner script.
2020-07-28 15:00:01 17227 [ERROR] WSREP: SST failed: 32 (Broken pipe)
2020-07-28 15:00:01 17227 [ERROR] Aborting
解决方法:
把my.cnf中的wsrep_sst_method值从xtrabackup-v2改成了rsync
启动成功后把wsrep_sst_method的值再改回来
systemd[1]: Started HAProxy Load Balancer.
haproxy-systemd-wrapper[2358]: haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f /etc/haproxy/haproxy.cf...id -Ds
haproxy-systemd-wrapper[2358]: [ALERT] 012/095413 (2359) : Starting proxy stats: cannot bind socket [0.0.0.0:3306]
haproxy-systemd-wrapper[2358]: haproxy-systemd-wrapper: exit, haproxy RC=256
执行如下命令解决
setsebool -P haproxy_connect_any=1