ProxySQL cluster+MGR
部署规划
| 系统 | hostname | IP地址 | 端口 | 角色 | 安装程序 |
|---|---|---|---|---|---|
| Centos 7.8 | wl_01 | 172.31.0.101 | 3306 | PRIMARY | MySQL8.0.28 |
| Centos 7.8 | wl_02 | 172.31.0.102 | 3306 | SECONDARY | MySQL8.0.28 |
| Centos 7.8 | wl_03 | 172.31.0.103 | 3306 | SECONDARY | MySQL8.0.28 |
| Centos 7.8 | wl_04 | 172.31.0.104 | ProxySQL:6032、6033 | 读写分离、业务部署 | ProxySQL、业务代码 |
| Centos 7.8 | wl_05 | 172.31.0.105 | ProxySQL:6032、6033 | 读写分离、业务部署 | ProxySQL、业务代码 |
提前部署好mysql MGR环境
mysql> select MEMBER_ID,MEMBER_HOST,MEMBER_PORT,MEMBER_STATE,MEMBER_ROLE from performance_schema.replication_group_members;
+--------------------------------------+--------------+-------------+--------------+-------------+
| MEMBER_ID | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE |
+--------------------------------------+--------------+-------------+--------------+-------------+
| 1b3163f8-5a57-11ed-845e-52549e082d61 | 172.31.0.102 | 3306 | ONLINE | SECONDARY |
| 2569bcbb-5a57-11ed-8e01-52549e4107ef | 172.31.0.103 | 3306 | ONLINE | SECONDARY |
| f9181e7f-5a56-11ed-8ed6-52549e1b2dbf | 172.31.0.101 | 3306 | ONLINE | PRIMARY |
+--------------------------------------+--------------+-------------+--------------+-------------+
(1)'创建proxysql的监控用户'
mysql> create user 'proxysql_monitor'@'172.31.0.%' identified with mysql_native_password by 'proxysql_monitor123';
mysql> grant replication client on *.* to 'proxysql_monitor'@'172.31.0.%';
(2)'创建业务用户,用于测试'
mysql> create user 'appuser'@'172.31.0.%' identified with mysql_native_password by '123';
mysql> grant all on appdb.* to 'appuser'@'172.31.0.%';
创建系统视图
在MGR集群的primary节点上创建如下视图：(可以在官方网站获取视图https://proxysql.com/documentation/main-runtime/#mysql_group_replication_hostgroups)
USE sys;
DELIMITER $$
CREATE FUNCTION IFZERO(a INT, b INT)
RETURNS INT
DETERMINISTIC
RETURN IF(a = 0, b, a)$$
CREATE FUNCTION LOCATE2(needle TEXT(10000), haystack TEXT(10000), offset INT)
RETURNS INT
DETERMINISTIC
RETURN IFZERO(LOCATE(needle, haystack, offset), LENGTH(haystack) + 1)$$
CREATE FUNCTION GTID_NORMALIZE(g TEXT(10000))
RETURNS TEXT(10000)
DETERMINISTIC
RETURN GTID_SUBTRACT(g, '')$$
CREATE FUNCTION GTID_COUNT(gtid_set TEXT(10000))
RETURNS INT
DETERMINISTIC
BEGIN
DECLARE result BIGINT DEFAULT 0;
DECLARE colon_pos INT;
DECLARE next_dash_pos INT;
DECLARE next_colon_pos INT;
DECLARE next_comma_pos INT;
SET gtid_set = GTID_NORMALIZE(gtid_set);
SET colon_pos = LOCATE2(':', gtid_set, 1);
WHILE colon_pos != LENGTH(gtid_set) + 1 DO
SET next_dash_pos = LOCATE2('-', gtid_set, colon_pos + 1);
SET next_colon_pos = LOCATE2(':', gtid_set, colon_pos + 1);
SET next_comma_pos = LOCATE2(',', gtid_set, colon_pos + 1);
IF next_dash_pos < next_colon_pos AND next_dash_pos < next_comma_pos THEN
SET result = result +
SUBSTR(gtid_set, next_dash_pos + 1,
LEAST(next_colon_pos, next_comma_pos) - (next_dash_pos + 1)) -
SUBSTR(gtid_set, colon_pos + 1, next_dash_pos - (colon_pos + 1)) + 1;
ELSE
SET result = result + 1;
END IF;
SET colon_pos = next_colon_pos;
END WHILE;
RETURN result;
END$$
CREATE FUNCTION gr_applier_queue_length()
RETURNS INT
DETERMINISTIC
BEGIN
RETURN (SELECT sys.gtid_count( GTID_SUBTRACT( (SELECT
Received_transaction_set FROM performance_schema.replication_connection_status
WHERE Channel_name = 'group_replication_applier' ), (SELECT
@@global.GTID_EXECUTED) )));
END$$
CREATE FUNCTION gr_member_in_primary_partition()
RETURNS VARCHAR(3)
DETERMINISTIC
BEGIN
RETURN (SELECT IF( MEMBER_STATE='ONLINE' AND ((SELECT COUNT(*) FROM
performance_schema.replication_group_members WHERE MEMBER_STATE != 'ONLINE') >=
((SELECT COUNT(*) FROM performance_schema.replication_group_members)/2) = 0),
'YES', 'NO' ) FROM performance_schema.replication_group_members JOIN
performance_schema.replication_group_member_stats USING(member_id));
END$$
CREATE VIEW gr_member_routing_candidate_status AS SELECT
sys.gr_member_in_primary_partition() as viable_candidate,
IF( (SELECT (SELECT GROUP_CONCAT(variable_value) FROM
performance_schema.global_variables WHERE variable_name IN ('read_only',
'super_read_only')) != 'OFF,OFF'), 'YES', 'NO') as read_only,
sys.gr_applier_queue_length() as transactions_behind, Count_Transactions_in_queue as 'transactions_to_cert' from performance_schema.replication_group_member_stats;$$
DELIMITER ;
#给proxysql的监控用户添加关于系统视图的权限
mysql> GRANT SELECT ON sys.* TO 'proxysql_monitor'@'172.31.0.%';
#验证视图
mysql> select gr_member_in_primary_partition();
mysql> select * from sys.gr_member_routing_candidate_status;
视图报错的解决办法
如果上面视图查询时，出现
ERROR 1242 (21000): Subquery returns more than 1 row
的报错，可能是因为mysql版本太新，继续执行以下操作可以消除报错：
USE sys;
DELIMITER $$
DROP FUNCTION gr_member_in_primary_partition$$
DROP VIEW gr_member_routing_candidate_status$$
CREATE FUNCTION gr_member_in_primary_partition()
RETURNS VARCHAR(3)
DETERMINISTIC
BEGIN
RETURN (SELECT IF( MEMBER_STATE='ONLINE' AND ((SELECT COUNT(*) FROM
performance_schema.replication_group_members WHERE MEMBER_STATE != 'ONLINE') >=
((SELECT COUNT(*) FROM performance_schema.replication_group_members)/2) = 0),
'YES', 'NO' ) FROM performance_schema.replication_group_members JOIN
performance_schema.replication_group_member_stats rgms USING(member_id) WHERE rgms.MEMBER_ID=@@SERVER_UUID);
END$$
CREATE VIEW gr_member_routing_candidate_status AS SELECT
sys.gr_member_in_primary_partition() as viable_candidate,
IF( (SELECT (SELECT GROUP_CONCAT(variable_value) FROM
performance_schema.global_variables WHERE variable_name IN ('read_only',
'super_read_only')) != 'OFF,OFF'), 'YES', 'NO') as read_only,
sys.gr_applier_queue_length() as transactions_behind, Count_Transactions_in_queue as 'transactions_to_cert'
from performance_schema.replication_group_member_stats rgms
where rgms.MEMBER_ID=(select gv.VARIABLE_VALUE
from `performance_schema`.global_variables gv where gv.VARIABLE_NAME='server_uuid');$$
DELIMITER ;
1.下载安装proxysql（wl_04、wl_05）
(1)获取rpm包
[root@wl_05 opt]# wget https://github.com/sysown/proxysql/releases/download/v2.4.4/proxysql-2.4.4-1-centos7.x86_64.rpm
(2)安装依赖包
[root@wl_05 opt]# yum install -y perl-DBD-MySQL perl-DBI perl-Time-HiRes perl-IO-Socket-SSL gnutls
(3)安装proxysql
[root@wl_05 opt]# rpm -ivh proxysql-2.4.4-1-centos7.x86_64.rpm
(4)创建数据目录
[root@wl_05 opt]# mkdir /proxysql
[root@wl_05 opt]# chown proxysql.proxysql -R /proxysql
2.编辑proxysql的配置文件（wl_04、wl_05）
[root@wl_05 opt]# cat /etc/proxysql.cnf
datadir="/proxysql"
errorlog="/proxysql/proxysql.log"
admin_variables=
{
admin_credentials="admin:admin;cluster_user:cluster_user123" #配置用于proxysql cluster实例节点间通讯的账号
mysql_ifaces="0.0.0.0:6032" #全网开放登录
cluster_username="cluster_user" #集群用户名称,与admin_credentials中配置的相同
cluster_password="cluster_user123" #集群用户密码,与admin_credentials中配置的相同
cluster_check_interval_ms=200
cluster_check_status_frequency=100
cluster_mysql_query_rules_save_to_disk=true
cluster_mysql_servers_save_to_disk=true
cluster_mysql_users_save_to_disk=true
cluster_proxysql_servers_save_to_disk=true
cluster_mysql_query_rules_diffs_before_sync=3
cluster_mysql_servers_diffs_before_sync=3
cluster_mysql_users_diffs_before_sync=3
cluster_proxysql_servers_diffs_before_sync=3
}
mysql_variables=
{
threads=4
max_connections=2048
default_query_delay=0
default_query_timeout=36000000
have_compress=true
poll_timeout=2000
interfaces="0.0.0.0:6033" #代理请求的端口
default_schema="information_schema"
stacksize=1048576
server_version="8.0.28" #指定后端数据库版本
connect_timeout_server=3000
monitor_username="proxysql_monitor" #监控用户
monitor_password="proxysql_monitor123" #监控用户的密码
monitor_history=600000
monitor_connect_interval=60000
monitor_ping_interval=10000
monitor_read_only_interval=1500
monitor_read_only_timeout=500
ping_interval_server_msec=120000
ping_timeout_server=500
commands_stats=true
sessions_sort=true
connect_retries_on_failure=10
}
proxysql_servers = #在这个部分提前定义好集群的成员
(
{
hostname="172.31.0.104"
port=6032
weight=1
comment="ProxySQL_node1"
},
{
hostname="172.31.0.105"
port=6032
weight=1
comment="ProxySQL_node2"
}
)
3.启动proxysql服务（wl_04、wl_05）
[root@wl_04 opt]# /usr/bin/proxysql --idle-threads -c /etc/proxysql.cnf
[root@wl_04 opt]# mysql -uadmin -padmin -h127.0.0.1 -P6032
mysql> select * from runtime_proxysql_servers;
+--------------+------+--------+----------------+
| hostname | port | weight | comment |
+--------------+------+--------+----------------+
| 172.31.0.105 | 6032 | 1 | ProxySQL_node2 |
| 172.31.0.104 | 6032 | 1 | ProxySQL_node1 |
+--------------+------+--------+----------------+
mysql> select hostname,port,comment,Uptime_s,last_check_ms from stats_proxysql_servers_metrics;
+--------------+------+----------------+----------+---------------+
| hostname | port | comment | Uptime_s | last_check_ms |
+--------------+------+----------------+----------+---------------+
| 172.31.0.105 | 6032 | ProxySQL_node2 | 20 | 6827 |
| 172.31.0.104 | 6032 | ProxySQL_node1 | 20 | 11525 |
+--------------+------+----------------+----------+---------------+
4.添加数据库节点与读写组
4.1 添加mysql_servers
[root@wl_05 opt]# /usr/local/mysql_5733/bin/mysql -uadmin -padmin -h127.0.0.1 -P6032
mysql> insert into mysql_servers(hostgroup_id,hostname,port,max_connections,comment) values(10,'172.31.0.101',3306,2000,'MGR_01');
mysql> insert into mysql_servers(hostgroup_id,hostname,port,max_connections,comment) values(10,'172.31.0.102',3306,2000,'MGR_02');
mysql> insert into mysql_servers(hostgroup_id,hostname,port,max_connections,comment) values(10,'172.31.0.103',3306,2000,'MGR_03');
mysql> select hostgroup_id,hostname,port,max_connections,comment,status from mysql_servers;
4.2 添加MGR hostgroups信息
mysql> show create table mysql_group_replication_hostgroups\G
- writer_hostgroup：默认的写组。后端read_only=0的节点会自动分配到这个组中。
- backup_writer_hostgroup：为备份节点组。超出max_writers数量的但允许写的节点都会放进备份组backup_writer_hostgroup中。
- reader_hostgroup：负责读的组。读请求都会路由到该主机组中的节点,后端read_only=1的节点会自动分配到这个组中。
- offline_hostgroup：当ProxySQL监控并决定了某节点为OFFLINE后，会将其放进组offline_hostgroup中。
- active：当启用后，ProxySQL会监控该主机组，并在不同组之间合理地移动节点。
- max_writers：该字段的值决定writer_hostgroup中最大允许的节点数。
- writer_is_also_reader：决定一个节点升级为写节点(放进writer_hostgroup)后是否仍然保留在reader_hostgroup组中提供读服务。
- max_transactions_behind： 当某节点延后于写节点时，为了防止读取到过期数据，ProxySQL可能会自动避开该节点。
- comment：该字段用于说明、注释。
mysql> insert into mysql_group_replication_hostgroups(writer_hostgroup,backup_writer_hostgroup,reader_hostgroup,offline_hostgroup,active,max_writers,writer_is_also_reader,max_transactions_behind,comment) values(10,11,12,13,1,1,0,10,'MGR_cluster_01');
mysql> select * from mysql_group_replication_hostgroups;
4.3 将配置加载到内存
mysql> load mysql servers to runtime;
mysql> save mysql servers to disk;
#查看当前runtime_mysql_servers,可以看到hostgroup_id已经自动根据后端db的read_only参数,调整为对应值
mysql> select hostgroup_id,hostname,port,max_connections,status from runtime_mysql_servers;
5.添加路由规则
'---需要注意: select ... for update规则的rule_id必须要小于普通的select规则的rule_id，因为ProxySQL是根据rule_id的顺序进行规则匹配的。'
[root@wl_05 opt]# /usr/local/mysql_5733/bin/mysql -uadmin -padmin123 -h172.31.0.103 -P6032
mysql> insert into mysql_query_rules(rule_id,active,match_digest,destination_hostgroup,apply) VALUES (1,1,'^SELECT.*FOR UPDATE$',10,1), (2,1,'^SELECT',12,1);
mysql> load mysql query rules to runtime;
mysql> save mysql query rules to disk;
mysql> select rule_id,active,match_digest,destination_hostgroup,apply from mysql_query_rules;
6. 添加用户
mysql> insert into mysql_users(username,password,default_hostgroup) values('appuser','123',10);
mysql> load mysql users to runtime;
mysql> save mysql users to disk;
7.测试读写分离
[root@wl_04 ~]# mysql -uappuser -p123 -P6033 -h172.31.0.102 -e "select @@hostname;"
+------------+
| @@hostname |
+------------+
| wl_02 |
+------------+
[root@wl_04 ~]# mysql -uappuser -p123 -P6033 -h172.31.0.102 -e "select @@hostname;"
+------------+
| @@hostname |
+------------+
| wl_03 |
+------------+