AWS上搭建MySQL主从-配置半自动异步复制策略

1、准备机器

worker-smarthome     主  172.31.76.147
worker-oss                 从  172.31.80.14

2、rpm安装msql yum源
这里安装MySQL5.7版本，MySQL5.7版本

$ wget 'https://dev.mysql.com/get/mysql57-community-release-el7-11.noarch.rpm'
$ sudo rpm -Uvh mysql57-community-release-el7-11.noarch.rpm
$ yum repolist all | grep mysql

3、安装数据库

$ yum install -y mysql-community-server

4、启动mysql，并查看mysql状态

$ systemctl enable mysqld
$ systemctl start  mysqld
$ systemctl status mysqld

5、查看mysql临时密码

$ grep "password" /var/log/mysqld.log

2018-07-19T09:26:49.073788Z 1 [Note] A temporary password is generated for root@localhost: &&s%n)XXXX

6、修改密码并允许远程登录

$ mysql -uroot -p

  输入初始密码，此时不能做任何事情，因为MySQL默认必须修改密码之后才能操作数据库：

mysql> set global validate_password_policy=0;
mysql> ALTER USER 'root'@'localhost' IDENTIFIED BY 'new password';

允许远程连接：

mysql> CREATE USER 'slave'@'localhost' IDENTIFIED BY '123456';
mysql>GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'new password' WITH GRANT OPTION;
mysql>flush privileges;

 
6、永久关闭selinux

$ vim /etc/selinux/config

将SELINUX=enforcing改为SELINUX=disabled，保存后退出

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

7、重启
此时获取当前selinux防火墙的安全策略仍为Enforcing，配置文件并未生效。

$ getenforce

Enforcing

重启机器

$ reboot
验证

$ /usr/sbin/sestatus
SELinux status:                 disabled
$ getenforce
Disabled

8、创建软链接：

$ mv /var/lib/mysql /db/
$ cd /var/lib/
$ ln -s /db/mysql mysql

9、重启mysql

$ systemctl restart mysqld
$ systemctl status mysqld

12、master db修改配置/etc/my.cnf配置，添加如下内容：

[mysqld]

datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock

# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

log-error=/db/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

server-id=1             #配置server-id，让主服务器有唯一ID号
log-bin=mysql-bin   #打开Mysql日志，日志格式为二进制
skip-name-resolve=1   #关闭名称解析，（非必须）
lower_case_table_names = 1 #不区分大小写
binlog_format = row
expire_logs_days = 7
#开启gtid，必须主从全开
gtid_mode = ON
enforce_gtid_consistency = 1
##从服务器的更新是否写入二进制日志
log_slave_updates = 1
master-verify-checksum = 1
log-slave-updates = ON
character_set_server=utf8mb4
max_binlog_size=100M

[client]
default_character-set=utf8

13、创建授权slave用户的权限

$ mysql -uroot -p

mysql> set global validate_password_policy=0;
mysql> CREATE USER 'slave'@'172.31.%.%' IDENTIFIED BY 'bosma2018';
mysql>GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO slave@'172.31.%.%' IDENTIFIED BY 'bosma2018';
mysql>flush privileges;
mysql>show master status;
+------------------+----------+--------------+------------------+-------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000001 |      721 |              |                  |                   |
+------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)

$ systemctl restart mysqld

14、修改slave数据库，在/etc/my.cnf配置添加如下内容：

红色字体部分，特别针对主从同步延时问题做了优化处理。

[mysqld]

datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock

# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

log-error=/var/lib/mysql/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
lower_case_table_names = 1 #不区分大小写
server-id=2       #配置server-id，让从服务器有唯一ID号
relay_log = mysql-relay-bin    #打开Mysql日志，日志格式为二进制
read_only = 1    #设置只读权限
log_bin = mysql-bin         #开启从服务器二进制日志，（非必须）

log_slave_updates = 1
gtid_mode = ON
enforce_gtid_consistency = ON
log-slave-updates = ON
skip-slave-start = true
expire_logs_days = 7
read_only = ON
slave-sql-verify-checksum = 1
log-bin = /var/lib/mysql/mysql-bin
log_bin_index = /var/lib/mysql/mysql-bin.index
relay_log = /var/lib/mysql/relay-log
relay_log_index = /var/lib/mysql/relay-log-index

slow_query_log=1 # 是否开启慢查询日志收集, 1为启用, 0为禁用
slow_query_log_file=/var/lib/mysql/log/slow.log
log_slow_slave_statements=1 # 记录 slave 产生的慢查询
log_throttle_queries_not_using_indexes=10 # 每分钟记录到日志的未使用索引的语句数目, 超过这个值后只记录语句数量和花费的总时间
max_binlog_size=100M # binlog日志文件大小
long_query_time=3 # 记录慢查询超时时间, 默认为秒
min_examined_row_limit=100  #查询返回少于该参数指定行的SQL不被记录到慢查询日志

# 当slave从库宕机后, 假如relay-log损坏了, 导致一部分中继日志没有处理, 则自动放弃所有未执行的relay-log, 并且重新从master上获取日志, 这样就保证了relay-log的完整
性
relay_log_recovery=1
# 这个参数控制了当mysql启动或重启时, mysql在搜寻GTIDs时是如何迭代使用binlog文件的.  这个选项设置为真, 会提升mysql执行恢复的性能. 因为这样mysql-server启动和binlog日志清理更快
binlog_gtid_simple_recovery=1
slave_skip_errors=ddl_exist_errors # 跳过指定error no类型的错误, 设成all 跳过所有错误

replicate-do-db=smarthome  #待同步的数据库
slave-parallel-type=LOGICAL_CLOCK
slave-parallel-workers=16 #开启16个线程
master_info_repository=TABLE
relay_log_info_repository=TABLE
relay_log_recovery=ON

[client]
default_character-set=utf815、重启slave数据库服务器复制线程，让slave连接master，并开始重做master二进制日志中的事件

$ systemctl restart mysqld
mysql> change master to master_host=' 172.31.76.147',master_user='slave',master_password='XXXX',master_log_file='mysql-bin.000001',master_log_pos=721;
Query OK, 0 rows affected, 2 warnings (0.02 sec)

mysql> start slave;
Query OK, 0 rows affected (0.00 sec)
mysql> show processlist;
+----+-------------+-----------+------+---------+------+--------------------------------------------------------+------------------+
| Id | User        | Host      | db   | Command | Time | State                                                  | Info             |
+----+-------------+-----------+------+---------+------+--------------------------------------------------------+------------------+
|  3 | system user |           | NULL | Connect |  275 | Connecting to master                                   | NULL             |
|  4 | system user |           | NULL | Connect |  275 | Slave has read all relay log; waiting for more updates | NULL             |
|  5 | root        | localhost | NULL | Query   |    0 | starting                                               | show processlist |
+----+-------------+-----------+------+---------+------+--------------------------------------------------------+------------------+
3 rows in set (0.00 sec)

16、测试mysql主从是否配置成功。

$ mysql -uroot -p

 

mysql> change master to master_host='172.31.76.147',master_user='slave',master_password='bosma2018',
 master_log_file='mysql-bin.000003',master_log_pos=307;
mysql> start slave;
mysql> SHOW SLAVE STATUS\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 172.31.76.147
                  Master_User: slave
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000003
          Read_Master_Log_Pos: 307
               Relay_Log_File: mysql-relay-bin.000002
                Relay_Log_Pos: 320
        Relay_Master_Log_File: mysql-bin.000003
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB:
          Replicate_Ignore_DB:
           Replicate_Do_Table:
       Replicate_Ignore_Table:
      Replicate_Wild_Do_Table:
  Replicate_Wild_Ignore_Table:
                   Last_Errno: 0
                   Last_Error:
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 307
              Relay_Log_Space: 527
              Until_Condition: None
               Until_Log_File:
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File:
           Master_SSL_CA_Path:
              Master_SSL_Cert:
            Master_SSL_Cipher:
               Master_SSL_Key:
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error:
               Last_SQL_Errno: 0
               Last_SQL_Error:
  Replicate_Ignore_Server_Ids:
             Master_Server_Id: 1
                  Master_UUID: 6c0f317c-8bcd-11e8-be49-028c605d1146
             Master_Info_File: /db/mysql/master.info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind:
      Last_IO_Error_Timestamp:
     Last_SQL_Error_Timestamp:
               Master_SSL_Crl:
           Master_SSL_Crlpath:
           Retrieved_Gtid_Set:
            Executed_Gtid_Set:
                Auto_Position: 0
         Replicate_Rewrite_DB:
                 Channel_Name:
           Master_TLS_Version:
1 row in set (0.00 sec)

注意Slave_IO_Running和Slave_SQL_Running状态都为，如下所示，才表示主从配置成功：
Slave_IO_Running: Yes，Slave_SQL_Running: Yes

17、配置主从策略

mysql的主从复制主要有3种模式：

a..主从同步复制：数据完整性好，但是性能消耗高

b.主从异步复制：性能消耗低，但是容易出现主从数据唯一性问题

c.主从半自动复制：介于上面两种之间。既能很好的保持完整性，又能提高性能

这里生产，我配置主从半自动复制策略。

18、给MySQL安装插件(worker-smarthome、worker-oss)

主库：

$ mysql -uroot -p

mysql> install plugin rpl_semi_sync_master soname 'semisync_master.so';

从库：

$ mysql -uroot -p

mysql>INSTALL PLUGIN rpl_semi_sync_slave SONAME 'semisync_slave.so';

19、修改MySQL配置文件（worker-smarthome、worker-oss)

主库：

$ vi /etc/my.cnf

主库添加以下内容：

[mysqld]

rpl_semi_sync_master_enabled=1

rpl_semi_sync_master_timeout=1000   #此单位是毫秒

从库：

$ vi /etc/my.cnf

从库添加添加以下内容：
[mysqld]

rpl_semi_sync_slave_enabled=1

21、重启MySQL数据库

$ systemctl restart mysqld

$ mysql -uroot -p

mysql> show global variables like 'rpl%';
+-------------------------------------------+------------+
| Variable_name                             | Value      |
+-------------------------------------------+------------+
| rpl_semi_sync_master_enabled              | ON         |
| rpl_semi_sync_master_timeout              | 10000      |
| rpl_semi_sync_master_trace_level          | 32         |
| rpl_semi_sync_master_wait_for_slave_count | 1          |
| rpl_semi_sync_master_wait_no_slave        | ON         |
| rpl_semi_sync_master_wait_point           | AFTER_SYNC |
| rpl_stop_slave_timeout                    | 31536000   |
+-------------------------------------------+------------+
7 rows in set (0.00 sec)

$ systemctl restart mysqld
$ mysql -uroot -p

mysql> show global variables like 'rpl%';
+---------------------------------+----------+
| Variable_name                   | Value    |
+---------------------------------+----------+
| rpl_semi_sync_slave_enabled     | ON       |
| rpl_semi_sync_slave_trace_level | 32       |
| rpl_stop_slave_timeout          | 31536000 |
+---------------------------------+----------+
3 rows in set (0.00 sec)

 

看到 rpl_semi_sync_slave_enabled 为ON，则代表半自动异步复制策略成功。