主从复制原理:在主数据库执行后,都会写入本地的日志系统A中。假设,实时的将变化了的日志系统中的数据库事件操作,在主数据库的3306端口,通过网络发给从数据。从数据库收到后,写入本地日志系统B,然后一条条的将数据库事件在数据库中完成。那么,主数据库的变化,从数据库也会变化,这样就是所谓的MYSQL的复制。
读写分离可以提高数据库的读写性能。
实验环境
iptables和selinux关闭
redhat6.5
主数据库 server2 172.25.12.2
从数据库 server3 172.25.12.3
实验所需的的rpm包
mysql-community-client-5.7.17-1.el6.x86_64.rpm
mysql-community-common-5.7.17-1.el6.x86_64.rpm
mysql-community-libs-5.7.17-1.el6.x86_64.rpm
mysql-community-libs-compat-5.7.17-1.el6.x86_64.rpm
mysql-community-server-5.7.17-1.el6.x86_64.rpm
1.数据库的配置
安装数据库
/etc/init.d/mysqld start
修改密码:
grep password /var/log/mysqld.log
查看那数据库的临时密码
2017-08-03T07:20:46.472777Z 1 [Note] A temporary password is generated for root@localhost: bty;2uN;C1qF
mysql -p 登陆数据库的修改密码
Enter password:
mysql> show databases;
mysql> ALTER USER root@localhost identified by ‘Redhat+007’;
添加本地数据库用户
mysql> show databases;
mysql> select * from mysql.user\G;
检查数据库
[root@server3 ~]# mysql -p
mysql> show databases;
其他数据库主机操作类似。
2.主从复制:
主数据库
配置主数据文件
[root@server2 ~]# vim /etc/my.cnf
server-id=1
log-bin=mysql-bin
binlog-do-db=test
binlog-ignore-db=mysql
配置从数据文件
[root@server3 ~]# vim /etc/my.cnf
server-id=3
[root@server2 ~]# /etc/init.d/mysqld restart
[root@server3 ~]# /etc/init.d/mysqld restart
主数据库:
[root@server2 ~]# /etc/init.d/mysqld restart
[root@server2 ~]# mysql -p
mysql> show databases;
mysql> grant replication slave on . to wax@’172.25.12.%’ identified by ‘Redhat+007’;
主服务上添加一个授权的账户,使得到时候从服务器可以使用该账号来完成数据的获取.
可以在从数据库端进行测试:
mysql -uwax -pRedhat+007 -h 172.25.12.11
mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000001 | 833 | test | mysql | |
+------------------+----------+--------------+------------------+-------------------+
mysql-bin.000001 是二进制日志文件
833 是结束的位置
在从数据库上
[root@server3 ~]# mysql -p
Enter password:
mysql> show slave status;
mysql> change master to master_host=’172.25.12.11’,master_user=’wax’, master_password=’Redhat+007’, master_log_file=’mysql-bin.000001’, master_log_pos=833;
mysql> start slave;
mysql> show slave status;
mysql> show slave status\G;
Relay_Log_Pos: 638
Relay_Master_Log_File: mysql-bin.000001
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
测试:
在主数据库上
mysql> create database test;
mysql> use test;
Database changed
mysql> create table usertb (
-> username varchar(15) not null ,
-> password varchar(25) not null );
mysql> insert into usertb values (“user1”,”123”);
mysql> insert into usertb values (“user2”,”123”);
mysql> select * from usertb ;
mysql> update usertb set password=’333’ where username=’user2’;
mysql> select * from usertb ;
从数据库
[root@server3 ~]# mysql -p
mysql> show databases;
mysql> use test;
mysql> select * from usertb;
mysql> quit
主数据库
[root@server2 mysql]# vim /etc/my.cnf
gtid_mode=ON
enforce-gtid-consistency=true
[root@server2 mysql]# /etc/init.d/mysqld restart
从服务器
[root@server3 ~]# vim /etc/my.cnf
gtid_mode=ON
enforce-gtid-consistency=true
[root@server3 ~]# /etc/init.d/mysqld restart
[root@server3 ~]# mysql -pRedhat+007
mysql> stop slave;
mysql> change master to master_host=’172.25.12.2’,master_user=’wax’, master_password=’Redhat+007’,MASTER_AUTO_POSITION=1;
mysql> start slave;
mysql> show slave status\G;
mysql> set global read_only=on ;
mysql> use test;
mysql> show tables;
mysql> use mysql;
mysql> show tables;
mysql> select * from gtid_executed;
3.并行复制
MySQL 5.7并行复制的思想简单易懂,一言以蔽之:一个组提交的事务都是可以并行
回放,因为这些事务都已进入到事务的prepare阶段,则说明事务之间没有任何冲突
(否则就不可能提交)。
在从服务器
[root@server3 ~]# vim /etc/my.cnf
slave-parallel-type=LOGICAL_CLOCK
slave-parallel-workers=16
master_info_repository=TABLE
relay_log_info_repository=TABLE
relay_log_recovery=ON
[root@server3 ~]# /etc/init.d/mysqld restart
[root@server3 ~]# mysql -pRedhat+007
mysql> show processlist;
会出现16个默认的进程
[root@server3 ~]# mysql -pRedhat+007
mysql> show databases;
mysql> use mysql;
mysql> show tables;
+---------------------------+
| Tables_in_mysql |
+---------------------------+
| server_cost |
| servers |
| slave_master_info |
| slave_relay_log_info |
| slave_worker_info |
| slow_log |
| tables_priv |
数据的的一些信息以表格的形式存储,而不是以库的形式存储
4.半同步复制 指IO线程 为尽可能的数据一致性
[root@server2 mysql]# cd /usr/lib64/mysql/plugin/
[root@server2 plugin]# ls 查看半同步模块
[root@server2 mysql]# mysql -pRedhat+007
mysql> INSTALL PLUGIN rpl_semi_sync_master SONAME ‘semisync_master.so’;
导入半同步模块
mysql> show plugins;查看模块
mysql> show variables like ‘%semi_sync%’;
| Variable_name | Value |
+-------------------------------------------+------------+
| rpl_semi_sync_master_enabled | OFF |
| rpl_semi_sync_master_timeout | 10000 |
会等待十妙。如果不能收到从服务器的回应,则转化为异步
| rpl_semi_sync_master_trace_level | 32 |
| rpl_semi_sync_master_wait_for_slave_count | 1 |
| rpl_semi_sync_master_wait_no_slave | ON |
| rpl_semi_sync_master_wait_point | AFTER_SYNC |
+-------------------------------------------+------------+
6 rows in set (0.00 sec)
mysql> set global rpl_semi_sync_master_enabled=1;
mysql> show variables like ‘%semi_sync%’;
mysql> show status like ‘%semi_sync%’;
从数据库上:
mysql> INSTALL PLUGIN rpl_semi_sync_slave SONAME ‘semisync_slave.so’; 导入模块
mysql> show plugins;
| validate_password | ACTIVE | VALIDATE PASSWORD | validate_password.so | GPL |
| rpl_semi_sync_slave | ACTIVE | REPLICATION | semisync_slave.so | GPL
mysql> show variables like ‘%semi_sync%’;
| rpl_semi_sync_slave_enabled | OFF |
| rpl_semi_sync_slave_trace_level | 32 |
mysql> set global rpl_semi_sync_slave_enabled=1;
mysql> show variables like ‘%semi_sync%’;
| rpl_semi_sync_slave_enabled | ON |
| rpl_semi_sync_slave_trace_level | 32 |
mysql> show status like ‘%semi_sync%’;
| Rpl_semi_sync_slave_status | OFF |
停掉开启使其生效
mysql> stop slave io_thread;
mysql> start slave io_thread;
mysql> show status like ‘%semi_sync%’;
| Rpl_semi_sync_slave_status | ON |
测试:
在主数据库上:
mysql> use test;
mysql> insert into usertb values (‘user3’,’1234’);
mysql> show status like ‘%semi_sync%’;
mysql> insert into usertb values (‘user4’,’5432’);
mysql> show status like ‘%semi_sync%’;
mysql> insert into usertb values (‘user5’,’545532’);
可以停掉从数据库上的IO线程再测试
mysql> stop slave io_thread;
mysql> start slave io_thread;
5.读写分离
MySQL的主从复制和MySQL的读写分离两者有着紧密联系,首先部署主从复制,只有主从复制完了,才能在此基础上进行数据的读写分离。简单来说,读写分离就是只在主服务器上写,只在从服务器上读,基本的原理是让主数据库处理事务性查询,而从数据库处理select查询,数据库复制被用来把事务性查询导致的改变更新同步到集群中的从数据库。
数据库的读写分离:
新建一个虚拟机作为调度器
mysql-proxy-0.8.5-linux-el6-x86-64bit.tar.gz
tar zxf mysql-proxy-0.8.5-linux-el6-x86-64bit.tar.gz -C /usr/local/
cd /usr/local/
ln -s mysql-proxy-0.8.5-linux-el6-x86-64bit/ mysql-proxy
cd mysql-proxy
mkdir etc 创建脚本存放目录
mkdir log 创建日志目录
修改读写分离配置文件
vi /usr/local/mysql-proxy/lua/rw-splitting.lua
min_idle_connections = 1,#默认超过4个连接数时,才开始读写分离,改为1
max_idle_connections = 2, #默认8,改为2
vim mysql-proxy.conf
[mysql-proxy]
user=root 运行mysql-proxy用户
daemon=true 以守护进程方式运行
keepalive=true 崩溃时,尝试重启
log-level=debug 定义log日志级别,由高到低分别有(error|warning|info|message|debug)
log-file=/usr/local/mysql-proxy/log/mysql-proxy.log 日志位置
admin-username=wax 主从mysql共有的用户
admin-password=Redhat+007 用户的密码
admin-lua-script=/usr/local/mysql-proxy/lib/mysql-proxy/lua/admin.lua 管理脚本
proxy-lua-script=/usr/local/mysql-proxy/share/doc/mysql-proxy/rw-splitting.lua 读写分离配置脚本
proxy-read-only-backend-addresses=172.25.12.3:3306 指定后端从slave读取数据
proxy-backend-addresses=172.25.12.2:3306 指定后端主master写入数据
plugins=proxy,admin
~
chmod 660 /usr/local/mysql-proxy/etc/mysql-proxy.conf
bin/mysql-proxy –defaults-file=/usr/local/mysql-proxy/etc/mysql-proxy.conf
测试:
在调度数据库上
[root@server4 mysql-proxy]# ps ax
1177 ? S 0:00 /usr/local/mysql-proxy/libexec/mysql-proxy --defaults-file=/usr/local/mysql-pro
1178 ? S 0:00 /usr/local/mysql-proxy/libexec/mysql-proxy --defaults-file=/usr/local/mysql-pro
1179 pts/0 R+ 0:00 ps ax
[root@server4 mysql-proxy]# netstat -antlpe
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State User Inode PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 0 8125 903/sshd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 0 8331 981/master
tcp 0 0 0.0.0.0:4040 0.0.0.0:* LISTEN 0 9734 1178/mysql-proxy
tcp 0 0 0.0.0.0:4041 0.0.0.0:* LISTEN 0 9737 1178/mysql-proxy
远程登录数据库看是否可以查看
[root@server4 mysql-proxy]# yum install -y mysql
[root@server4 mysql-proxy]# mysql -h 172.25.12.4 -P 4041 -u admin -p
mysql-proxy启动后会启动两个端口4040和4041,4040用于SQL转发,4041用于管理mysql-proxy。如有多个mysql-slave可以依次在后面添加
mysql> show databases;
ERROR 1105 (07000): use 'SELECT * FROM help' to see the supported commands
mysql> SELECT * FROM help;
+------------------------+------------------------------------+
| command | description |
+------------------------+------------------------------------+
| SELECT * FROM help | shows this help |
| SELECT * FROM backends | lists the backends and their state |
+------------------------+------------------------------------+
2 rows in set (0.00 sec)
mysql> SELECT * FROM backends;
+-------------+------------------+---------+------+------+-------------------+
| backend_ndx | address | state | type | uuid | connected_clients |
+-------------+------------------+---------+------+------+-------------------+
| 1 | 172.25.12.4:3306 | unknown | rw | NULL | 0 |
| 2 | 172.25.12.2:3306 | unknown | ro | NULL |
在主数据库中为用户授权
mysql> select * from mysql.user\G;
mysql> grant insert,select,update on . to admin@’172.25.12.%’ identified by ‘Westos+007’;
mysql> quit
在从数据库中为用户授权
[root@server3 mysql]# mysql -pRedhat+007
mysql> grant select on . to admin@’172.25.12.%’ identified by ‘Westos+007’;
Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> quit
Bye
[root@server4 mysql-proxy]# yum install -y lsof
lsof(list open files)是一个列出当前系统打开文件的工具。在linux环境下,任何事物都以文件的形式存在,通过文件不仅仅可以访问常规数据,还可以访问网络连接和硬件。所以如传输控制协议 (TCP) 和用户数据报协议 (UDP) 套接字等,系统在后台都为该应用程序分配了一个文件描述符,无论这个文件的本质如何,该文件描述符为应用程序与基础操作系统之间的交互提供了通用接口。
在真机上进行多次连接连接后查看调度器和后台的来年接情况
yum install -y mysql
mysql -h 172.25.12.4 -P 4040 -u admin -pWestos+007
在调度器和数据库服务器查看打开的连接
[root@server4 mysql-proxy]# lsof -i :3306
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
mysql-pro 1238 root 13u IPv4 10349 0t0 TCP server4:38410->server2:mysql (ESTABLISHED)
mysql-pro 1238 root 15u IPv4 10368 0t0 TCP server4:38411->server2:mysql (ESTABLISHED)
mysql-pro 1238 root 16u IPv4 10373 0t0 TCP server4:55728->server3:mysql (ESTABLISHED)
mysql-pro 1238 root 18u IPv4 10375 0t0 TCP server4:55729->server3:mysql (ESTABLISHED)
[root@server2 mysql]# yum install -y lsof
[root@server2 mysql]# lsof -i :3306
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
mysqld 1208 mysql 34u IPv6 8528 0t0 TCP *:mysql (LISTEN)
mysqld 1208 mysql 52u IPv6 9402 0t0 TCP server2:mysql->server4:38393 (ESTABLISHED)
mysqld 1208 mysql 53u IPv6 9285 0t0 TCP server2:mysql->server3:40559 (ESTABLISHED)
mysqld 1208 mysql 55u IPv6 9404 0t0 TCP server2:mysql->server4:38395 (ESTABLISHED)
[root@server3 mysql]# yum install -y lsof
[root@server3 mysql]# lsof -i :3306
[root@server3 mysql]# lsof -i :3306
6.数据库日志的查看
可在主数据库上查看日志文件
[root@server2 ~]# cd /var/lib/mysql
[root@server2 mysql]# mysqlbinlog mysql-bin.000001
查看数据库的二进制文件
查看数据库的一些参数
mysql> show variables like ‘%sync%’; 频率
mysql> show variables like ‘%binlog%’; 日志文件
mysql> show variables like ‘%log%’;
数据库的编码查询
mysqlbinlog binlog.000003
mysqlbinlog binlog.000003 –base64-output=DECODE-ROWS
mysqlbinlog binlog.000003 –base64-output=DECODE-ROWS -v
mysqlbinlog binlog.000003 –base64-output=DECODE-ROWS -vv
mysqlbinlog binlog.000003 –base64-output=DECODE-ROWS -vvv