proxysql1.4.12 安装及 集群测试 ~~~~~坑还是很多【亲测可用】

 

proxysql介绍

ProxySQL is a high performance, high availability, protocol aware proxy for MySQL and forks (like Percona Server and MariaDB). All the while getting the unlimited freedom that comes with a GPL license.

Its development is driven by the lack of open source proxies that provide high performance.

ProxySQL是一种针对mysql及其分支（如Percona,MariaDB）的一种高性能，高可用的代理协议。

ProxySQl之所以被开发出来是因为市面上缺少开源高可用的开源代理。

 

ProxySQL测试

proxysql安装：

cat <

 proxysql配置

  需要注意的是proxysql的配置是支持动态配置的。

  程序运行时会先读取sqlitedb（如果存在的话），否则就读取配置文件。部分参数是即使sqlitedb存在也读取配置文件的：

  如下。

 

配置流程图如下：

• datadir : it defines the path of ProxySQL datadir, where database file, logs and other files are stored
• restart_on_missing_heartbeats (new in 1.4.4) : if MySQL threads miss restart_on_missing_heartbeats heartbeats, proxysql will raise a SIGABRT signal and restart. Default is 10 . See watchdog .
• execute_on_exit_failure (new in 1.4.4) : if set, ProxySQL parent process will execute the defined script every time ProxySQL crashes. It is recommended to use this setting to generate an alert or log the event. Note that proxysql is able to restart in few milliseconds in case of crash, therefore it is possible that a normal failure is not detected by other monitoring tools.

+-------------------------+
|         RUNTIME         |
+-------------------------+
       /|\          |
        |           |
    [1] |       [2] |
        |          \|/
+-------------------------+
|         MEMORY          |
+-------------------------+ _
       /|\          |      |\
        |           |        \
    [3] |       [4] |         \ [5]
        |          \|/         \
+-------------------------+  +-------------------------+
|          DISK           |  |       CONFIG FILE       |
+-------------------------+  +-------------------------+

 如上：

runtime----内存中保存的配置数据结构。直接用来进程处理请求。包含：实际连接的后端server列表。实际的全局变量，能连接proxysql的用户。用户无法直接设置runtime的数据。需要通过底层调用来修改。

memory（也可被叫做main）--存在于内存的中的db.这个db暴露出兼容mysql的接口供用户连接进行配置项修改。以下配置可以在该处进行修改：

• mysql_servers -- the list of backend servers which ProxySQL connects to
• mysql_users -- the list of users and their credentials which connect to ProxySQL. Please note that ProxySQL will use the same credentials to connect to the backend servers as well!
• mysql_query_rules -- the list of query rules which are evaluated when routing traffic to the various backend servers. These rules can also rewrite queries or even cache the result of an executed query.
• global_variables -- the list of global variables which the proxy is configured to use, and which can be tweaked during runtime.

 disk------磁盘中的sqlite数据库， 位置：$(DATADIR)/proxysql.db. 

 config file---磁盘中的proxysql.cnf。

实际要将memory中的修改持久化时需要执行： 

SAVE MYSQL USERS TO DISK;

需要将memory中的修改应用到runtime时需要执行：

LOAD MYSQL USERS TO RUNTIME;

常见的命令如下：

LOAD MYSQL USERS TO RUNTIME;
SAVE MYSQL USERS TO DISK;
LOAD MYSQL SERVERS TO RUNTIME;
SAVE MYSQL SERVERS TO DISK;
LOAD MYSQL QUERY RULES TO RUNTIME;
SAVE MYSQL QUERY RULES TO DISK;
LOAD MYSQL VARIABLES TO RUNTIME;
SAVE MYSQL VARIABLES TO DISK;

LOAD ADMIN VARIABLES TO RUNTIME;
SAVE ADMIN VARIABLES TO DISK;

重置配置：

service proxysql stop

rm -f /var/lib/proxysql/*

service proxysql start

 

mgr搭建参照：https://blog.csdn.net/eaglejiawo1120/article/details/84544426

实验环境：

主机ip	mgr端口	是否可读写
192.168.1.103	3210（mysql启动端口），32100（mgr管理端口）	写
192.168.1.104	3210（mysql启动端口），32100（mgr管理端口）	读
192.168.1.104	3220（mysql启动端口），32200（mgr管理端口）	读

 

 

 

proxysql 安装位置

192.168.1.103，192.168.1.104

 

 

 

约定：

监控端：使用monitor用户访问：127.0.0.1@6032----mysql -umonitor -pbeacon -h 127.0.0.1 -P6032

管理端：使用monitor用户访问：127.0.0.1@6032----mysql -uadmin -padmin -h 127.0.0.1 -P6032

程序端：使用run用户访问127.0.0.1@6033---mysql -urun -pbeacon -h 127.0.0.1 -P6033

 

测试方式：

先在192.168.1.103启动 proxyshell 实例。并启动集群

然后192.168.1.104加入集群。并从192.168.1.103获取数据。

 

1. 查看两个实例中proxysql初始的集群状态：

   192.168.1.103, 192.168.1.104上分别执行：

    service proxysql stop

    cd  /var/lib/proxysql  && rm -f proxy*   

    service proxysql start

    mysql -uadmin -padmin -h 127.0.0.1 -P6032

    SELECT * FROM runtime_checksums_values ORDER BY name;

    结果如下：

   

可以看到。集群中同步的数据表：'mysql_query_rules', 'mysql_servers', 'mysql_users', 'proxysql_servers'。的version初始值均为1，且checksum初始都为0； 系统全局变量为verison为0.

 

2. 初始化集群

 192.168.1.103 上登录管理端执行如下语句：

set mysql-monitor_username='monitor';
set mysql-monitor_password='beacon';
save mysql variables to disk;
load mysql variables to runtime;


update global_variables set variable_value="admin:admin;cluster_20X:123456" where variable_name ='admin-admin_credentials'; 
update global_variables set variable_value="cluster_20X" where variable_name ='admin-cluster_username';
update global_variables set variable_value="123456" where variable_name ='admin-cluster_password';
save admin variables to disk;
load admin variables to runtime;

SELECT * FROM runtime_checksums_values ORDER BY name;

insert into proxysql_servers(hostname,port) values('192.168.1.103',6032);
save proxysql servers to disk;
load proxysql servers to runtime;

执行监控mysql集群的sql:

insert into mysql_servers(hostgroup_id,hostname,port,comment) values(10,'192.168.1.103',3210,'init write');
insert into mysql_servers(hostgroup_id,hostname,port,comment) values(30,'192.168.1.104',3210,'init read');
insert into mysql_servers(hostgroup_id,hostname,port,comment) values(30,'192.168.1.104',3220,'init read');
save mysql servers to disk;
load mysql servers to runtime;

insert into mysql_query_rules(rule_id,active,match_digest,destination_hostgroup,apply)values(1,1,'^SELECT.*FOR UPDATE$',10,1);
insert into mysql_query_rules(rule_id,active,match_digest,destination_hostgroup,apply)values(2,1,'^SELECT',30,1);
save mysql query rules to disk;
load mysql query rules to runtime;

insert into mysql_users(username,password,default_hostgroup) values('run','beacon',10);
save mysql users to disk;
load mysql users to runtime;

特别提醒
以下操作:  insert into mysql_group_replication_hostgroups(writer_hostgroup,backup_writer_hostgroup,reader_hostgroup,offline_hostgroup,active) values(10,20,30,40,1);
save mysql servers to disk;
load mysql servers to runtime;

在集群模式下不要使用。否则proxysql集群节点间会因为mysql servers checksum不一致而一直同步。

需要注意的是：表mysql_group_replication_hostgroups的数据以及cluster用户名，密码，凭证等参数如果有改变。那么集群中所有的主机都需要执行一遍。

 

3.  添加一个proxysql 实例到集群

 192.168.1.104 上登录管理端执行如下语句：

set mysql-monitor_username='monitor';
set mysql-monitor_password='beacon';
save mysql variables to disk;
load mysql variables to runtime;


update global_variables set variable_value="admin:admin;cluster_20X:123456" where variable_name ='admin-admin_credentials'; 
update global_variables set variable_value="cluster_20X" where variable_name ='admin-cluster_username';
update global_variables set variable_value="123456" where variable_name ='admin-cluster_password';
save admin variables to disk;
load admin variables to runtime;

SELECT * FROM runtime_checksums_values ORDER BY name;

insert into proxysql_servers(hostname,port) values('192.168.1.103',6032),('192.168.1.104',6032);
save proxysql servers to disk;
load proxysql servers to runtime;

查看同步状态：

执行sql: SELECT * FROM stats_proxysql_servers_checksums where name in ('mysql_query_rules', 'mysql_servers', 'mysql_users', 'proxysql_servers') order by name;

结果如下：
+-------------+------+-------------------+---------+------------+--------------------+------------+------------+------------+
| hostname    | port | name              | version | epoch      | checksum           | changed_at | updated_at | diff_check |
+-------------+------+-------------------+---------+------------+--------------------+------------+------------+------------+
| 192.168.1.103 | 6032 | mysql_query_rules | 2       | 1543474549 | 0xA04CC34FCAE0AA28 | 1543474549 | 1543474931 | 0          |
| 192.168.1.104 | 6032 | mysql_query_rules | 2       | 1543474601 | 0xA04CC34FCAE0AA28 | 1543474553 | 1543474931 | 0          |
| 192.168.1.103 | 6032 | mysql_servers     | 4       | 1543474553 | 0xBA8068367BCA3F2B | 1543474549 | 1543474931 | 0          |
| 192.168.1.104 | 6032 | mysql_servers     | 4       | 1543474601 | 0xBA8068367BCA3F2B | 1543474553 | 1543474931 | 0          |
| 192.168.1.103 | 6032 | mysql_users       | 2       | 1543474549 | 0x648ED57078047633 | 1543474549 | 1543474931 | 0          |
| 192.168.1.104 | 6032 | mysql_users       | 2       | 1543474601 | 0x648ED57078047633 | 1543474553 | 1543474931 | 0          |
| 192.168.1.103 | 6032 | proxysql_servers  | 2       | 1543474423 | 0xCA6D184ED24567CE | 1543474423 | 1543474931 | 0          |
| 192.168.1.104 | 6032 | proxysql_servers  | 2       | 1543474476 | 0xCA6D184ED24567CE | 1543474429 | 1543474931 | 0          |
+-------------+------+-------------------+---------+------------+--------------------+------------+------------+------------+

可以看到两个proxysql实例中'mysql_query_rules', 'mysql_servers', 'mysql_users', 'proxysql_servers'四个表的checksum都是一致的。

注意：

• epoch is the timestamp generated by the remote proxy when executing LOAD ... TO RUNTIME
• changed_at is the timestamp recorded when the local proxy detected a change in the remote proxy
• updated_at is the last timestamp for which the local proxy retrieved the checksum from the remote proxy

 

4. 将新添加入cluster的proxysql实例通知到其他节点

 192.168.1.103 上登录管理端执行如下语句：

insert into proxysql_servers(hostname,port) values('192.168.1.104',6032);
save proxysql servers to disk;
load proxysql servers to runtime;

查看同步状态：

执行sql: SELECT * FROM stats_proxysql_servers_checksums where name in ('mysql_query_rules', 'mysql_servers', 'mysql_users', 'proxysql_servers') order by name;

重点查看mysql_servers对应的实例见的checksum是否一致。

 

常见问题：

1. 无法将新添加的proxysql节点加入集群，无法访问数据库节点

新加入的proxysql 节点只会同步主节点的如下表数据：

• mysql_query_rules
• mysql_servers
• mysql_users
• proxysql_servers

对于除了这些的全局变量配置需要在新添加的节点中进行额外设置。

如监控的账号密码，集群的账号 密码 凭证等。

否则既有可能会出现如下错误：

ProxySQL_Cluster.cpp:215:ProxySQL_Cluster_Monitor_thread(): [WARNING] Cluster: unable to connect to peer 192.168.3.104:6032 . Error: ProxySQL Error: Access denied for user 'cluster_20X'@'' (using password: YES)

 

2.  一直在同步

ProxySQL_Cluster.cpp:523:set_checksums(): [WARNING] Cluster: detected a peer 192.168.1.104:6032 with mysql_query_rules version 1, epoch 1543461126, diff_check 450. Own version: 1, epoch: 1543457815. diff_check is increasing, but version 1 doesn't allow sync. This message will be repeated every 30 checks until LOAD MYSQL QUERY RULES TO RUNTIME is executed on candidate master.

错误原因一般是因为操作了表：mysql_group_replication_hostgroups。目前proxysql cluster还不支持mgr操作表：mysql_group_replication_hostgroups的同步。

 

注意事项：

1. 如果需要集群支持mysql group replication(MGR)的监控。那么需要在集群的所有机器上在表mysql_group_replication_hostgroups插入记录：
insert into mysql_group_replication_hostgroups(writer_hostgroup,backup_writer_hostgroup,reader_hostgroup,offline_hostgroup,active) values(10,20,30,40,1);
save mysql servers to disk;
load mysql servers to runtime;

2.  如果监控用户名，密码； 以及 cluster 用户名，密码和凭证修改也需要在集群内的所有机器上执行。

3.  新的机器加入proxysql cluster的时候，需要保证 监控用户名，密码； 以及 cluster 用户名，密码和凭证与集群内的机器一致，

如果不一致，则需要先进行一致化操作。另外在同步完数据（即执行：insert into proxysql_servers，save proxysql servers to

disk;load proxysql servers to runtime;）之后。才能对表mysql_group_replication_hostgroups进行操作。且对表mysql_group_replication_hostgroups操作时需要保证满足注意事项1.

 

 

参考：

1. proxysql cluster: https://github.com/sysown/proxysql/wiki/ProxySQL-Cluster

2.https://proxysql.com/blog/proxysql-cluster-part2

3. 官方安装proxysql cluster: https://proxysql.com/blog/proxysql-cluster