Linux高级篇--MySQL数据库读写分离

MySQL读写分离

MySQL读写分离

  • 读写分离应用:
      mysql-proxy:Oracle,https://downloads.mysql.com/archives/proxy/
      Atlas:Qihoo,https://github.com/Qihoo360/Atlas/blob/master/README_ZH.md
      dbproxy:美团,https://github.com/Meituan-Dianping/DBProxy
      Cetus:网易乐得,https://github.com/Lede-Inc/cetus
      Amoeba:https://sourceforge.net/projects/amoeba/
      Cobar:阿里巴巴,Amoeba的升级版
      Mycat:基于Cobar, http://www.mycat.io/
      ProxySQL:https://proxysql.com/

  • 一主一从

Linux高级篇--MySQL数据库读写分离_第1张图片

  • 一主多从

Linux高级篇--MySQL数据库读写分离_第2张图片

使用Proxysql实现mysql读写分离
具体实验步骤如下:

  • 思路:先配置完成主从复制,在读写分离节点配置Proxysql

  • 实验环境:
      读写分离节点:192.168.32.203
      master主机:192.168.32.200
      slave01主机:192.168.32.201
      slave02:192.168.32.202

  • 主从复制配置:

主节点配置:
    master主机:192.168.32.200配置:
      更改mysql配置文件
        vim /etc/my.cnf
        [mysqld]
        datadir=/var/lib/mysql
        socket=/var/lib/mysql/mysql.sock
        innodb_file_per_table
        skip_name_resolve
        server_id=1
        log_bin
      启动mysql服务
        systemctl start mariadb
      授权从节点能够访问主节点
        MariaDB [(none)]> grant replication slave on *.* to 'repluser'@'192.168.32.%' identified by 'centos';
      查看主节点日志位置
        MariaDB [(none)]> show master status;
        +--------------------+----------+--------------+------------------+
        | File               | Position | Binlog_Do_DB | Binlog_Ignore_DB |
        +--------------------+----------+--------------+------------------+
        | mariadb-bin.000001 |      402 |              |                  |
        +--------------------+----------+--------------+------------------+
    slave01主机:192.168.32.201配置:
      更改配置文件
        vim /etc/my.cnf
        [mysqld]
        datadir=/var/lib/mysql
        socket=/var/lib/mysql/mysql.sock
        innodb_file_per_table
        skip_name_resolve
        server_id=2
        relay_log=relay_log
        relay_log_index=relay-log.index
      指定从节点的主节点位置和日志信息
        CHANGE MASTER TO
        MASTER_HOST='192.168.32.200',
        MASTER_USER='repluser',
        MASTER_PASSWORD='centos',
        MASTER_PORT=3306,
        MASTER_LOG_FILE='mariadb-bin.000001',
        MASTER_LOG_POS=402;
      启动主从复制并查看主从复制创泰
        start slave;
        show slave status\G

    slave02主机:192.168.32.202配置:
      更改配置文件
        vim /etc/my.cnf
        [mysqld]
        datadir=/var/lib/mysql
        socket=/var/lib/mysql/mysql.sock
        innodb_file_per_table
        skip_name_resolve
        server_id=3              #注意server_id必须唯一
        relay_log=relay_log
        relay_log_index=relay-log.index
      指定从节点的主节点位置和日志信息
        CHANGE MASTER TO
        MASTER_HOST='192.168.32.200',
        MASTER_USER='repluser',
        MASTER_PASSWORD='centos',
        MASTER_PORT=3306,
        MASTER_LOG_FILE='mariadb-bin.000001',
        MASTER_LOG_POS=402;
      启动主从复制并查看主从复制创泰
        start slave;
        show slave status\G
  • 读写分离节点配置
读写分离节点:192.168.32.203配置:
  思路:配置读写分离,主节点只负责写操作,读操作全部调度到从节点上
  注意:在配置前一定要在从节点配置文件中添加read-only选项,否则
  proxysql节点无法判断该节点充当的角色
    从节点192.16.32.201
      vim /etc/my.cnf
      [mysqld]
      datadir=/var/lib/mysql
      socket=/var/lib/mysql/mysql.sock
      innodb_file_per_table
      skip_name_resolve
      server_id=2              #注意server_id必须唯一
      relay_log=relay_log
      relay_log_index=relay-log.index
      read-only             #注意读写分离时从节点必须添加该选项
    从节点192.16.32.202
      vim /etc/my.cnf
      [mysqld]
      datadir=/var/lib/mysql
      socket=/var/lib/mysql/mysql.sock
      innodb_file_per_table
      skip_name_resolve
      server_id=3             #注意server_id必须唯一
      relay_log=relay_log
      relay_log_index=relay-log.index
      read-only             #注意读写分离时从节点必须添加该选项

proxysql的安装:
  使用官方提供的yum源直接安装即可
    vim /etc/yum.repos.d/proxysql.repo
    [proxysql_repo]
    name= ProxySQL YUM repository
    baseurl=http://repo.proxysql.com/ProxySQL/proxysql-1.4.x/centos/\$releasever
    gpgcheck=1
    gpgkey=http://repo.proxysql.com/ProxySQL/repo_pub_key
  使用yum安装proxysql
    yum -y install proxysql
  启动服务
    service proxysql start
  默认监听端口
    6032 proxysql的配置端口
    6033:ProxySQL对外提供服务的端口
  使用mysql客户端连接到ProxySQL的管理接口6032,默认管理员用户和密码都是admin:
    mysql -uadmin -padmin -P6032 -h127.0.0.1
    MySQL [(none)]> status           #查看连接状态信息
    --------------
    mysql  Ver 15.1 Distrib 5.5.60-MariaDB, for Linux (x86_64) using readline 5.1

    Connection id:          1
    Current database:       admin
    Current user:           admin
    SSL:                    Not in use
    Current pager:          stdout
    Using outfile:          ''
    Using delimiter:        ;
    Server:                 MySQL
    Server version:         5.5.30 (ProxySQL Admin Module)
    Protocol version:       10
    Connection:             127.0.0.1 via TCP/IP
    Server characterset:    utf8
    Db     characterset:    utf8
    Client characterset:    utf8
    Conn.  characterset:    utf8
    TCP port:               6032
    Uptime:                 12 min 15 sec

  查看数据库信息
    MySQL [(none)]> show databases;
    +-----+---------------+-------------------------------------+
    | seq | name          | file                                |
    +-----+---------------+-------------------------------------+
    | 0   | main          |                                     |
    | 2   | disk          | /var/lib/proxysql/proxysql.db       |
    | 3   | stats         |                                     |
    | 4   | monitor       |                                     |
    | 5   | stats_history | /var/lib/proxysql/proxysql_stats.db |
    +-----+---------------+-------------------------------------+
    5 rows in set (0.00 sec)

    MySQL [(none)]> use main;
    Reading table information for completion of table and column names
    You can turn off this feature to get a quicker startup with -A

    Database changed
    MySQL [main]> show tables;
    +--------------------------------------------+
    | tables                                     |
    +--------------------------------------------+
    | global_variables                           |
    | mysql_collations                           |
    | mysql_group_replication_hostgroups         |
    | mysql_query_rules                          |
    | mysql_query_rules_fast_routing             |
    | mysql_replication_hostgroups               |
    | mysql_servers                              |
    | mysql_users                                |
    | proxysql_servers                           |
    | runtime_checksums_values                   |
    | runtime_global_variables                   |
    | runtime_mysql_group_replication_hostgroups |
    | runtime_mysql_query_rules                  |
    | runtime_mysql_query_rules_fast_routing     |
    | runtime_mysql_replication_hostgroups       |
    | runtime_mysql_servers                      |
    | runtime_mysql_users                        |
    | runtime_proxysql_servers                   |
    | runtime_scheduler                          |
    | scheduler                                  |
    +--------------------------------------------+
    20 rows in set (0.00 sec)
  说明:在main和monitor数据库中的表, runtime_开头的是运行时的配置,不能修改,只能修改非runtime_表,修改后必须执行LOAD (数据库名) TO RUNTIME才能加载到RUNTIME生效,执行save (数据库名) to disk将配置持久化保存到磁盘

  为了让proxysql能够调度三台主从节点服务器,需要把三台主从节点的ip地址加入proxysql的mysql_servers表中
    MySQL > select * from mysql_servers;  #查看表的内容,为空
    MySQL > insert into mysql_servers(hostgroup_id,hostname,port) values(10,'192.168.32.200',3306);   
    MySQL > insert into mysql_servers(hostgroup_id,hostname,port) values(10,'192.168.32.201',3306);
    MySQL > insert into mysql_servers(hostgroup_id,hostname,port) values(10,'192.168.32.202',3306);   #hostgroup_id是指把服务器进行分组,分为读组和写组便于区分主节点和从节点进行读写调度;如果没有进行分组,则会根据从节点配置文件中的read-only选项来判断主从节点进行读写调度
    MySQL [main]> select * from mysql_servers\G       #查看表的内容
    *************************** 1. row ***************************
           hostgroup_id: 10
               hostname: 192.168.32.200
                   port: 3306
                 status: ONLINE   #online说明加入成功
                 weight: 1
            compression: 0
        max_connections: 1000
    max_replication_lag: 0
                use_ssl: 0
         max_latency_ms: 0
                comment: 
    *************************** 2. row ***************************
           hostgroup_id: 10
               hostname: 192.168.32.201
                   port: 3306
                 status: ONLINE
                 weight: 1
            compression: 0
        max_connections: 1000
    max_replication_lag: 0
                use_ssl: 0
         max_latency_ms: 0
                comment: 
    *************************** 3. row ***************************
           hostgroup_id: 10
               hostname: 192.168.32.202
                   port: 3306
                 status: ONLINE
                 weight: 1
            compression: 0
        max_connections: 1000
    max_replication_lag: 0
                use_ssl: 0
         max_latency_ms: 0
                comment: 
    3 rows in set (0.00 sec)
    MySQL > load mysql servers to runtime;   #注意表名中不带下划线
    MySQL > save mysql servers to disk;   #保存到硬盘

添加监控后端节点的用户。ProxySQL通过每个节点的read_only值来自动调整它们是属于读组还是写组
  在master节点192.168.32.200上执行:
    mysql> grant replication client on *.* to monitor@'192.168.32.%' identified by 'centos';   #由于做了主从复制,因此该账号会自动被复制到从节点上
  在ProxySQL节点192.168.32.203上配置监控
    MySQL [(none)]> set mysql-monitor_username='monitor';  #如果为监控账号为自定义账号,则需要更改此选项,因为默认监控账号即为monitor,如果监控账号为monitor,则无需修改
    MySQL [(none)]> set mysql-monitor_password='centos';   #更改监控账号的密码,该选项来自于main数据库的global_variables表,该表中的该账号的默认密码为monitor,因此要更改该密码为自定义密码
  加载到RUNTIME,并保存到disk
    MySQL [(none)]> load mysql variables to runtime;
    MySQL [(none)]> save mysql variables to disk;

监控模块的指标保存在monitor库的log表中
  查看监控连接是否正常的 (对connect指标的监控):(如果connect_error的结果为NULL则表示正常)
    MySQL [(none)]> select * from mysql_server_connect_log;
  查看监控心跳信息 (对ping指标的监控):
    MySQL [(none)]>  select * from mysql_server_ping_log;
  查看read_only和replication_lag的监控日志
    MySQL [(none)]>  select * from mysql_server_read_only_log;
    MySQL [(none)]>  select * from mysql_server_replication_lag_log;

设置分组信息
  需要修改的是main库中的mysql_replication_hostgroups表,该表有3个字段:writer_hostgroup,reader_hostgroup,comment, 指定写组的id为10,读组的id为20
    MySQL [(none)]> insert into mysql_replication_hostgroups values(10,20,"test");
  将mysql_replication_hostgroups表的修改加载到RUNTIME生效
    MySQL [(none)]> load mysql servers to runtime;
    MySQL [(none)]> save mysql servers to disk;
  Monitor模块监控后端的read_only值,按照read_only的值将节点自动移动到读/写组
    MySQL [(none)]> select hostgroup_id,hostname,port,status,weight from mysql_servers;
    +--------------+----------------+------+--------+--------+
    | hostgroup_id | hostname       | port | status | weight |
    +--------------+----------------+------+--------+--------+
    | 10           | 192.168.32.200 | 3306 | ONLINE | 1      |
    | 20           | 192.168.32.201 | 3306 | ONLINE | 1      |
    | 20           | 192.168.32.202 | 3306 | ONLINE | 1      |
    +--------------+----------------+------+--------+--------+
  上表中,可以看到已经自动进行分组,从节点201和202为从节点负责读操作,200为主节点负责写操作

配置发送SQL语句的用户
  在master主节点上创建访问数据库的用户
    grant all on *.* to sqluser@'192.168.32.%' identified by 'centos';    #由于配置了主从复制,在从节点也存在该账户
  在ProxySQL读写分离节点192.168.32.203上进行配置,将用户sqluser添加到mysql_users表中,监控该用户,对该用户的访问进行读写调度。default_hostgroup默认组设置为写组10,当读写分离的路由规则不符合时,会访问默认组的数据库,即主节点
    insert into mysql_users(username,password,default_hostgroup) values('sqluser','centos',10);
    load mysql users to runtime;
    save mysql users to disk;
  使用sqluser用户测试是否能路由到默认的10写组实现读、写数据
  注意:此时要另外打开一个窗口进行数据库连接测试,防止与使用mysql -uadmin -padmin -P6032 -h127.0.0.1命令连接的数据库进行的相关操作混淆
    mysql -usqluser -pcentos -P6033 -h127.0.0.1         #测试连接数据库
    MySQL [(none)]> select @@server_id;    #查看mysql服务器的server_id
    +-------------+
    | @@server_id |
    +-------------+
    |           1 |
    +-------------+
  注意:此时对数据库做的操作都是在master节点上进行,因此还需要定义调度策略
  创建测试数据库和表查看效果
    MySQL [(none)]> use testdb;
    MySQL [testdb]> create table t(id int);

配置路由调度规则,实现读写分离
  在读写分离节点192.168.32.203上配置,注意:此时要在mysql -uadmin -padmin -P6032 -h127.0.0.1命令连接的窗口进行配置
  与规则有关的表:mysql_query_rules和mysql_query_rules_fast_routing,后者是前者的扩展表,1.4.7之后支持
  插入路由规则:将select语句分离到20的读组,select语句中有一个特殊语句SELECT...FOR UPDATE它会申请写锁,应路由到10的写组
    insert into mysql_query_rules
  (rule_id,active,match_digest,destination_hostgroup,apply)VALUES
  (1,1,'^SELECT.*FOR UPDATE$',10,1),(2,1,'^SELECT',20,1);    #注意:SELECT.*FOR UPDATE虽然以select命令开头,但属于写操作,属于分组10。其他以select开头的命令都属于读(查询)操作,因此属于分组20。
    load mysql query rules to runtime;
    save mysql query rules to disk;
  注意:因ProxySQL根据rule_id顺序进行规则匹配,select ... for update规则的rule_id必须要小于普通的select规则的rule_id
  查看路由规则表
    MySQL [(none)]> select * from mysql_query_rules\G
  • 验证
  测试读操作是否路由给20的读组,server_id为2或3,说明在从节点,也就是读组。
    [root@centos7 ~]# mysql -usqluser -pcentos -P6033 -h127.0.0.1 -e 'select @@server_id'
    +-------------+
    | @@server_id |
    +-------------+
    |           2 |
    +-------------+
    [root@centos7 ~]# mysql -usqluser -pcentos -P6033 -h127.0.0.1 -e 'select @@server_id'
    +-------------+
    | @@server_id |
    +-------------+
    |           3 |
    +-------------+
  测试写操作,以事务方式进行测试,server_id为1,说明在主节点,也就是写组
    [root@centos7 ~]# mysql -usqluser -pcentos -P6033 -h127.0.0.1 -e 'begin;select @@server_id;commit;'      
    +-------------+
    | @@server_id |
    +-------------+
    |           1 |
    +-------------+
    mysql -usqluser -pcentos -P6033 -h127.0.0.1 -e 'insert testdb.t values (1)'
    mysql -usqluser -pcentos -P6033 -h127.0.0.1 -e 'select id from testdb.t'
  路由的信息:查询stats库中的stats_mysql_query_digest表
    SELECT hostgroup hg,sum_time, count_star, digest_text FROM stats_mysql_query_digest ORDER BY sum_time DESC;

你可能感兴趣的:(Linux高级篇--MySQL数据库读写分离)