Mysql-5.5主从实现同步、半同步、过滤、ssl安全复制、以及mysql-proxy实现mysql-5.6读写分离

Mysql-5.5主从实现同步、半同步、过滤、基于ssl安全复制、以及mysql-proxy实现mysql-5.6读写分离

案例拓扑图

创建mysql数据目录及进程用户并安装mysql  

创建mysql服务进程的用户,提供mysql数据存放目录,并修改数据的目录。    

[root@master ~]# mkdir -pv /mydata/data
mkdir: created directory `/mydata'
mkdir: created directory `/mydata/data'
[root@master ~]# useradd -r mysql
[root@master ~]# chown -R mysql.mysql /mydata/data

将从网上下载的mysql二进制包复制到从服务端。

 

[root@master ~]# scp mysql-5.5.28-linux2.6-i686.tar.gz 172.16.20.7:/root/
The authenticity of host '172.16.20.7 (172.16.20.7)' can't be established.
RSA key fingerprint is 0a:0b:2f:67:c7:29:af:79:fe:2f:64:51:ca:01:1d:b0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.16.20.7' (RSA) to the list of known hosts.
[email protected]'s password:
mysql-5.5.28-linux2.6-i686.tar.gz                                                    100%  172MB   6.9MB/s   00:25

这里以主服务端安装二进制源码包mysql。

[root@master ~]# tar xf mysql-5.5.28-linux2.6-i686.tar.gz -C /usr/local/
[root@master ~]# cd /usr/local/
[root@master local]#ln -sv  mysql-5.5.28-linux2.6-86.i386 mysql
[root@master local]# cd mysql
[root@master mysql]# scripts/mysql_install_db --user=mysql --datadir=/mydata/data
WARNING: The host 'master.yangyaru.com' could not be looked up with resolveip.
This probably means that your libc libraries are not 100 % compatible
with this binary MySQL version. The MySQL daemon, mysqld, should work
normally with the exception that host name resolving will not work.
This means that you should use IP addresses instead of hostnames
when specifying MySQL privileges !
Installing MySQL system tables...
OK
Filling help tables...
OK
To start mysqld at boot time you have to copy
support-files/mysql.server to the right place for your system
PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:
./bin/mysqladmin -u root password 'new-password'
./bin/mysqladmin -u root -h master.yangyaru.com password 'new-password'
Alternatively you can run:
./bin/mysql_secure_installation
which will also give you the option of removing the test
databases and anonymous user created by default.  This is
strongly recommended for production servers.
See the manual for more instructions.
You can start the MySQL daemon with:
cd . ; ./bin/mysqld_safe &
You can test the MySQL daemon with mysql-test-run.pl
cd ./mysql-test ; perl mysql-test-run.pl
Please report any problems with the ./bin/mysqlbug script!

给mysql服务提供一个启动服务脚本并开机自动启动。

[root@master mysql]# cp support-files/mysql.server /etc/init.d/mysqld
[root@master mysql]# chkconfig --add mysqld

将mysql的命令添加到PATH变量中并重读下脚本。

[root@master mysql]# vim /etc/profile.d/mysql.sh
export PATH=$PATH:/usr/local/mysql/bin
[root@master mysql]# .  /etc/profile.d/mysql.sh

给mysql提供一个配置文件并配置mysql配置文件

[root@master mysql]# cp support-files/my-large.cnf /etc/my.cnf
[root@master ~]# cat /etc/my.cnf
添加如下几行,其余与的不动
datadir = /mydata/data
innodb_file_per_table = 1
log-bin=master-bin
log_bin_index = master_bin.index
binlog_format=mixed
server-id = 6

启动mysql服务器进程。

[root@master ~]# service mysqld start

Starting MySQL................                             [  OK  ]   
查看进程是否已启动。

wps_clip_image-31723

直接在命令行提示符下连接上mysql测下是否可以连接上mysql,如果不可以再打开一个终端测试。   
连接上mysql授权172.16.20.7允许复制数据。

[root@master ~]# mysql
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.5.28-log MySQL Community Server (GPL)
Copyright (c) 2000, 2012, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> grant replication slave on *.* to 'yangyaru'@'172.16.20.7' identified by 'yangyaru';
Query OK, 0 rows affected (0.09 sec)
mysql>flush privileges;
mysql>exit

从服务端的相关配置大多和主服务端的配置相同这里,我只列出不一样的配置。   
从服务端的配置文件    

[root@master ~]# cat /etc/my.cnf
添加如下几行,其余与的不动
datadir = /mydata/data
innodb_file_per_table = 1
relay_log=relay_log
relay_bin_index = relay_log.index
binlog_format=mixed
server-id = 7

查看下主服务端现在二进制日志的文件和位置。   
wps_clip_image-23851
开启主服务端授权的帐号连接到主服务器端并开启复制。

[root@slave1 ~]# mysql
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.5.28 MySQL Community Server (GPL)
Copyright (c) 2000, 2012, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> CHANGE MASTER TO  MASTER_HOST = '172.16.20.6', MASTER_PORT = 3306, MASTER_USER='yangyaru',MASTER_PASSWORD='yangyaru', MASTER_LOG_FILE='master-bin.000001', MASTER_LOG_POS=340;
Query OK, 0 rows affected (0.17 sec)
mysql> start slave;
Query OK, 0 rows affected (0.03 sec)
mysql> \q
Bye

查看下当前我们的从服务器端记录二进制数据文件和position位置。

mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 172.16.20.6
Master_User: yangyaru
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: master-bin.000001
Read_Master_Log_Pos: 340
Relay_Log_File: relay_log.000002
Relay_Log_Pos: 254
Relay_Master_Log_File: master-bin.000001
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 340
Relay_Log_Space: 404
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 6
1 row in set (0.03 sec)

可以看到当前是Read_Master_Log_Pos: 340。

到此简单的主从mysql服务配置就已经完成^_^。


下面对此配置测试下,在主服务器端创建一个yangyarudb的数据库。看下从服务器上是否复制。

wps_clip_image-14705

wps_clip_image-31019



现在对我们的配置做下延伸。


一、主从服务器读写分离实现负载均衡。

一般我们从服务器端是只负责客户的读请求的,主服务端负责写请求的。那么配置下吧!

首先查看下从服务器端的只读方式是否打开。

mysql> show global variables like 'read%';
+----------------------+---------+
| Variable_name        | Value   |
+----------------------+---------+
| read_buffer_size     | 1048576 |
| read_only            | OFF     |
| read_rnd_buffer_size | 4194304 |
+----------------------+---------+
3 rows in set (0.00 sec)

打开我们的只读方式有两种

一种是在mysql服务器端的全局模式下配置,但是在全局模式下配置mysql重启之后就会失效;

一种是在/etc/my.cnf配置文件中配置,这个配置是永久生效;这里我们选择第二种。

给/etc/my.cnf配置文件添加一行如下:

[root@slave1 ~]# vim /etc/my.cnf 

read_only = ON

然后重启启动下我们的mysql服务器。

[root@slave1 ~]# service mysqld restart

Shutting down MySQL......                                  [  OK  ]

Starting MySQL..................                           [  OK  ]

进入mysql服务测试下是否启动只读模式。   
wps_clip_image-329

主服务器端执行了写操作,日志文件立即同步到从服务器,为了保证事务的完整性。可以在

从服务器端配置文件的[mysqld]段中添加skip_slave_start = 1来实现从服务器的mysql服务在启动时候不要自动启动从服务线程。

二、实现主从服务器半同步。

半同步的定义:主服务器端执行了写操作,必须往从服务器端复制一份,才能给客户端返回提交状态。

这里我们需要在主服务器端安装semisync_master.so从服务器端安装semisync_slave.so

具体步骤如下:

主服务器端:

mysql> install plugin rpl_semi_sync_master SONAME 'semisync_master.so';

Query OK, 0 rows affected (0.36 sec)

wps_clip_image-1086
安装好之后看下是否启动如果没有启动我们启用下。    
wps_clip_image-5599

主服务器端开启semi_sync功能,并设置等待时候为3秒。

mysql> set global rpl_semi_sync_master_enabled =1;

Query OK, 0 rows affected (0.02 sec)

mysql> set global rpl_semi_sync_master_timeout = 3000 ;

Query OK, 0 rows affected (0.00 sec)

wps_clip_image-3121

从服务器端:

mysql> install plugin rpl_semi_sync_slave  SONAME 'semisync_slave.so';

Query OK, 0 rows affected (0.28 sec)

wps_clip_image-30608

从服务器端开启semi_sync功能。

mysql> set global rpl_semi_sync_slave_enabled =1;

Query OK, 0 rows affected (0.03 sec)

wps_clip_image-6814

验收从主服务端:   
wps_clip_image-20830

半同步复制是如果从服务端没有开启的话,主服务端第一次会延迟3秒中之后提交,之后主服务端会降低延迟不再等待从服务端。从服务端开启之后在追赶上主服务端让后在实现半同步。

实现演示下:

1、先关闭从服务端。

mysql> stop slave;

Query OK, 0 rows affected (0.02 sec)

2、主服务端创建一个semidb数据库

wps_clip_image-7957

wps_clip_image-11006

3、从服务端启动服务。

mysql> start slave;

Query OK, 0 rows affected (0.03 sec)   
mysql> show slave status\G    
wps_clip_image-1864


三、percona-toolkit是一个专门对msyql主从服务的管理的一个工具。这里我们可以安装下使用下。

主从服务都安装percona-toolkit

主服务端下载安装下

[root@master ~]# yum -y --nogpgcheck localinstall percona-toolkit-2.2.2-1.noarch.rpm 

从服务端从主服务端复制过来安装:

[root@master ~]# scp percona-toolkit-2.2.2-1.noarch.rpm 172.16.20.7:/root/

[email protected]'s password: 

percona-toolkit-2.2.2-1.noarch.rpm                                                     100% 1632KB 816.0KB/s   00:02 

安装之后会给我们生成很多以pt开头的命令行工具这些命令,如pt-slave-delay:这个命令行工具,他是专门让我们的从服务端比主服务端慢一点的的设置。

这些命令我们不一一做介绍了哈:

四、基于ssl加密的方式实现主从复制。

Mysql的主从复制是明文传送的,但在生产环境中我们的主从服务器不可以在同一个机房或在同一个局域网中,这时候我们的数据传输肯定需要用到ssl啦。Ssl的实现其实很简单的,如下操作步骤。

主服务器端自创创建CA,并为主服务器端和从服务器端颁发CA。   

[root@master ~]#vim /etc/pki/tls/openssl.conf  #如果你的系统平台是5.8的就需要该这些
将dir = ../../CA 更改为:dir = /etc/pki/CA
[root@master ~]# (umask 077;openssl genrsa -out /etc/pki/CA/private/cakey.pem 1024)
[root@master ~]# openssl req -new -x509 -key /etc/pki/CA/private/cakey.pem -out /etc/pki/CA/cacert.pem -days 3650  #CA 的相关信息这里我们根据自己的实际情况填写。
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:HENAN
Locality Name (eg, city) [Default City]:ZHENGZHOU
Organization Name (eg, company) [Default Company Ltd]:magedu
Organizational Unit Name (eg, section) []:tech
Common Name (eg, your name or your server's hostname) []:ca.magedu.com
Email Address []:caadmin.magedu.com
[root@master ~]# mkdir /etc/pki/CA/{certs,newcerts,crl}  #如果你的系统平台是5.8需要创建这些文件。
[root@master ~]# touch /etc/pki/CA/index.txt
[root@master ~]# echo 01 > /etc/pki/CA/serial

主从服务器端都创建密钥和申请文件,步骤一样这里以主服务端为例。   

[root@master ~]#mkdir /usr/local/mysql/ssl
[root@master ~]#(umask 077;openssl genrsa -out  /usr/local/mysql/ssl/mysql.key 1024)
[root@master ~]#openssl req -new -key /usr/local/mysql/ssl/mysql.key -out  /usr/local/mysql/ssl/mysql.csr -days 365 #申请书的内容根据自己的实际情况填写。
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:HENAN
Locality Name (eg, city) [Default City]:ZHENGZHOU
Organization Name (eg, company) [Default Company Ltd]:magedu
Organizational Unit Name (eg, section) []:tech
Common Name (eg, your name or your server's hostname) []:master.yangyaru.com
Email Address []:master.yangyaru.com
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

主服务器端的CA证书颁发:

[root@master ~]#openssl ca -in /usr/local/mysql/ssl/mysql.csr -out /mydata/data/ssl/mysql.crt -days 365
Using configuration from /etc/pki/tls/openssl.cnf
Check that the request matches the signature
Signature ok
Certificate Details:
Serial Number: 1 (0x1)
Validity
Not Before: May 19 23:47:49 2013 GMT
Not After : May 19 23:47:49 2014 GMT
Subject:
countryName               = CN
stateOrProvinceName       = HENAN
organizationName          = magedu
organizationalUnitName    = tech
commonName                = master.yangyaru.com
emailAddress              = master.yangyaru.com
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
Netscape Comment:
OpenSSL Generated Certificate
X509v3 Subject Key Identifier:
9B:42:46:CF:1D:83:56:7D:04:37:CB:40:89:A2:07:EC:C7:9D:C2:0D
X509v3 Authority Key Identifier:
keyid:07:7C:CF:69:74:1D:4D:D8:09:7A:3C:D9:F3:07:B6:46:40:E0:47:0C
Certificate is to be certified until May 19 23:47:49 2014 GMT (365 days)
Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated

从服务器端创建申请书之后将申请书发送到CA让CA签署。   

[root@slave ~]#scp  /usr/local/mysql/ssl/mysql.csr  172.16.20.6:/tmp
The authenticity of host '172.16.20.6 (172.16.20.6)' can't be established.
RSA key fingerprint is 0a:0b:2f:67:c7:29:af:79:fe:2f:64:51:ca:01:1d:b0.
Are you sure you want to continue connecting (yes/no)? y
Please type 'yes' or 'no': yes
Warning: Permanently added '172.16.20.6' (RSA) to the list of known hosts.
[email protected]'s password:
mysql.csr            100%  708     0.7KB/s   00:00
[root@master ~]#openssl ca -in  /tmp/mysql.csr -out /tmp/mysql.crt -days 365
Using configuration from /etc/pki/tls/openssl.cnf
Check that the request matches the signature
Signature ok
Certificate Details:
Serial Number: 2 (0x2)
Validity
Not Before: May 19 23:51:08 2013 GMT
Not After : May 19 23:51:08 2014 GMT
Subject:
countryName               = CN
stateOrProvinceName       = HENAN
organizationName          = magedu
organizationalUnitName    = tech
commonName                = slave.yangyaru.com
emailAddress              = slave.yangyaru.com
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
Netscape Comment:
OpenSSL Generated Certificate
X509v3 Subject Key Identifier:
A8:DB:F3:88:F1:25:A0:02:B6:F5:49:EE:67:2B:C1:BE:66:B9:E8:A7
X509v3 Authority Key Identifier:
keyid:07:7C:CF:69:74:1D:4D:D8:09:7A:3C:D9:F3:07:B6:46:40:E0:47:0C
Certificate is to be certified until May 19 23:51:08 2014 GMT (365 days)
Sign the certificate? [y/n]:yes
1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
[root@master ~]# scp  /tmp/mysql.crt  172.16.20.7:/tmp
[email protected]'s password:
mysql.crt                                       100% 3239     3.2KB/s   00:00
[root@slave ~]# mv  /tmp/mysql.crt  /mydata/data/ssl

将CA文件按拷贝到从服务端证书所在目录    
[root@master ~]# scp  /etc/pki/CA/cacert.pem 172.16.20.7:/mydata/data/ssl/ 

[email protected]'s password: 

cacert.pem               100% 1070     1.0KB/s   00:00   
注意:生成的证书和密钥的属组和属主都要改成mysql用户。    
至此,证书颁发成功!!

下面配置主从服务器让他们支持ssl功能。   
首先看下ssl功能的开启状态,如果没有开启就开启。    
Master:    
主服务器端配置开启ssl功能    
[root@master ~]#vim /etc/my.cnf 

添加如下几行,其余不动   
ssl  #开启服务

ssl_ca = /mydata/data/ssl/cacert.pem   #指定下CA文件所在的位置在那   
ssl_cert = /mydada/data/ssl/mysql.crt   #证书文件的位置    
ssl_key = /usr/local/mysql/ssl/mysql.key #密钥所在的位置

重启服务,查看其变量的状态。

[root@master ~]#service mysqld restart
mysql>show variables like ‘%ssl%’;
+---------------+--------------------------------+
| Variable_name | Value                          |
+---------------+--------------------------------+
| have_openssl  | YES                       |
| have_ssl      | YES                       |
| ssl_ca        | /mydata/data/ssl/cacert.pem    |
| ssl_capath    |                                |
| ssl_cert      | /mydata/data/ssl/mysql.crt     |
| ssl_cipher    |                                |
| ssl_key       | /usr/local/mysql/ssl/mysql.key |
+---------------+--------------------------------+
7 rows in set (0.00 sec)

创建一个帐号让从服务器端可以连接到主服务器端复制数据,但是要添加上基于密钥认证才可以。

mysql>grant replication slave on *.*  to  'yangyaru'@'172.16.20.7'  identified by 'yangyaru'  require ssl;   
mysql>flush privileges;

查看主服务器端现在二进制的文件和所在的位置记下它,等会从服务端连接直接从这里开始复制。   
wps_clip_image-5320
slave:    
配置mysql配置文件:    
[root@slave ~]# vim /etc/my.cnf    
添加如下几行,其余不动    
ssl  #开启服务

ssl_ca = /mydata/data/ssl/cacert.pem   #指定下CA文件所在的位置在那   
ssl_cert = /mydada/data/ssl/mysql.crt   #证书文件的位置    
ssl_key = /usr/local/mysql/ssl/mysql.key #密钥所在的位置

因为我们之前连接过主服务器端所以这里我们做下修改设置。

mysql> change master to
-> master_host='172.16.20.6',
-> master_user='yangyaru',
-> master_password='yangyaru',
-> master_log_file='master-bin.***',
-> master_log_pos=***,
-> master_ssl=1,
-> master_ssl_ca='/etc/pki/CA/cacert.pem',
-> master_ssl_cert='/mydata/data/ssl/mysql.crt',
-> master_ssl_key='/mydata/data/ssl/mysql.key';
mysql>start slave;

如果出现下面红色的显示就说明,我们的配置已经生效。

mysql>show slave status\G

mysql数据复制的过滤。   
(一般我们不建议主服务器端过滤数据,它会使我们二进制日志不完整,所以我们做实验演示也是不再主服务器端操作只在从服务端进行过滤。)    
m’syql从服务器只复制主服务器的magedudb这个库。    
mysql>show global variables like ‘binlog-%’;    
[root@slave ~]#vim  /etc/my.cnf

replicate-do-db =  discuzdb 

[root@slave ~]#service mysqld restart

在从服务器检查下是否   
mysql>show slave status\G;

在主服务端创建数据库。   
mysql>create database magedudb;    
mysql>create database discuzdb;

在从服务器端看下结果。

mysql>show databases;

#############################################################################

MySQL 5.6引入的GTID(Global Transaction IDs)使得其复制功能的配置、监控及管理变得更加易于实现,且更加健壮。 


要在MySQL 5.6中使用复制功能,其服务配置段[mysqld]中于少应该定义如下选项:

1、log-slave-updates、gtid-mode、enforce-gtid-consistency、report-port和report-host:用于启动GTID及满足附属的其它需求;

2、master-info-repository、relay-log-info-repository:启用此两项,可用于实现在崩溃时保证二进制及从服务器安全的功能;

3、sync-master-info:启用之可确保无信息丢失;

4、slave-paralles-workers:设定从服务器的SQL线程数;0表示关闭多线程复制功能;

5、binlog-checksum、master-verify-checksum、slave-sql-verify-checksum:启用复制有关的所有校验功能;

6、binlog-rows-query-log-events:启用之可用于在二进制日志记录事件相关的信息,可降低故障排除的复杂度;

7、log-bin:启用二进制日志,这是保证复制功能的基本前提;

server-id:同一个复制拓扑中的所有服务器的id号必须惟一;   
8、binlog-format:二进制日志的格式,有row、statement和mixed几种类型;

注意:当设置隔离级别为READ-COMMITED必须设置二进制日志格式为ROW,现在MySQL官方认为STATEMENT这个已经不再适合继续使用;但mixed类型在默认的事务隔离级别下,可能会导致主从数据不一致;

mysql-5.6的配置演示:

1、配置主从节点的服务配置文件

1.1、配置master节点:

[mysqld]
binlog-format=ROW
log-bin=master-bin.log
log-slave-updates=true
gtid-mode=on
enforce-gtid-consistency=true
master-info-repository=TABLE
relay-log-info-repository=TABLE
sync-master-info=1
slave-parallel-workers=2
binlog-checksum=CRC32
master-verify-checksum=1
slave-sql-verify-checksum=1
binlog-rows-query-log_events=1
server-id=1
report-port=3306
port=3306
datadir=/mydata/data
socket=/tmp/mysql.sock
report-host=master.magedu.com

1.2、配置slave节点:

[mysqld]
binlog-format=ROW
log-slave-updates=true
gtid-mode=on
enforce-gtid-consistency=true
master-info-repository=TABLE
relay-log-info-repository=TABLE
sync-master-info=1
slave-parallel-workers=2
binlog-checksum=CRC32
master-verify-checksum=1
slave-sql-verify-checksum=1
binlog-rows-query-log_events=1
server-id=11
report-port=3306
port=3306
log-bin=mysql-bin.log
datadir=/mydata/data
socket=/tmp/mysql.sock
report-host=slave.magedu.com

2、创建复制用户

mysql> GRANT REPLICATION SLAVE ON *.* TO [email protected] IDENTIFIED BY 'yangyaru';

说明:172.16.20.7是从节点服务器;如果想一次性授权更多的节点,可以自行根据需要修改;

3、为备节点提供初始数据集

锁定主表,备份主节点上的数据,将其还原至从节点;如果没有启用GTID,在备份时需要在master上使用show master status命令查看二进制日志文件名称及事件位置,以便后面启动slave节点时使用。

4、启动从节点的复制线程

如果启用了GTID功能,则使用如下命令:

mysql> CHANGE MASTER TO MASTER_HOST='master.yangyaru.com', MASTER_USER='yangyaru', MASTER_PASSWORD='yangyaru', MASTER_AUTO_POSITION=1;

没启用GTID,需要使用如下命令:

slave> CHANGE MASTER TO MASTER_HOST='172.16.100.6',

-> MASTER_USER='yangyaru',

-> MASTER_PASSWORD='yangyaru',

-> MASTER_LOG_FILE='master-bin.000003',

-> MASTER_LOG_POS=1174;

实现半同步复制

1、分别在主从节点上安装相关的插件

master> INSTALL PLUGIN rpl_semi_sync_master SONAME 'semisync_master.so';

slave> INSTALL PLUGIN rpl_semi_sync_slave SONAME 'semisync_slave.so';

2、启用半同步复制

在master上的配置文件中,添加

rpl_semi_sync_master_enabled=ON

在至少一个slave节点的配置文件中添加

rpl_semi_sync_slave_enabled=ON

而后重新启动mysql服务即可生效。

或者,也可以mysql服务上动态启动其相关功能:

master> SET GLOBAL rpl_semi_sync_master_enabled = ON;

slave> SET GLOBAL rpl_semi_sync_slave_enabled = ON;

slave> STOP SLAVE IO_THREAD; START SLAVE IO_THREAD;

3、确认半同步功能已经启用

master> CREATE DATABASE magedudb;

master> SHOW STATUS LIKE 'Rpl_semi_sync_master_yes_tx';

slave> SHOW DATABASES; 

拓展:

使用mysql proxy实现mysql主从服务读写分离功能。   
这里的系统平台为rhel6.4 32位系统,因此就以mysql-proxy-0.8.3-linux-glibc2.3-x86-32bit.tar.gz为例。    
安装配置mysql-proxy:    
[root@proxy ~]# tar xf mysql-proxy-0.8.3-linux-glibc2.3-x86-32bit.tar.gz -C /usr/local

[root@proxy ~]# cd /usr/local

[root@proxy ~]# ln -sv mysql-proxy-0.8.3-linux-glibc2.3-x86-32bit  mysql-proxy   
为mysql-proxy提供SysV服务脚本    

[root@proxy ~]vim /etc/rc.d/init.d/mysql-proxy
#!/bin/bash
#
# mysql-proxy This script starts and stops the mysql-proxy daemon
#
# chkconfig: - 78 30
# processname: mysql-proxy
# description: mysql-proxy is a proxy daemon for mysql
# Source function library.
. /etc/rc.d/init.d/functions
prog="/usr/local/mysql-proxy/bin/mysql-proxy"
# Source networking configuration.
if [ -f /etc/sysconfig/network ]; then
. /etc/sysconfig/network
fi
if [ -f /etc/sysconfig/mysql-proxy ]; then
. /etc/sysconfig/mysql-proxy
fi
# Check that networking is up.
[ ${NETWORKING} = "no" ] && exit 0
# Set default mysql-proxy configuration.
ADMIN_USER=${ADMIN_USER:-admin}
ADMIN_PASSWD=${ADMIN_PASSWD:-""}
PROXY_OPTIONS=${PROXY_OPTIONS:="--daemon"}
PROXY_USER=${PROXY_USER:-"mysql-proxy"}
ADMIN_ADDRESS="${ADMIN_ADDRESS:-0.0.0.0:4040}"
PROXY_ADDRESS="${PROXY_ADDRESS:-0.0.0.0:4041}"
PROXY_PID=/var/run/mysql-proxy.pid
# Source mysql-proxy configuration.
if [ -f /etc/sysconfig/mysql-proxy ]; then
. /etc/sysconfig/mysql-proxy
fi
RETVAL=0
start() {
echo -n $"Starting $prog: "
daemon $prog $PROXY_OPTIONS --pid-file=$PROXY_PID --user=$PROXY_USER --admin-username="$ADMIN_USER" --admin-lua-script="$ADMIN_LUA_SCRIPT" --admin-password="$ADMIN_PASSWORD" --admin-address="$ADMIN_ADDRESS" --proxy-address="$PROXY_ADDRESS"
RETVAL=$?
echo
if [ $RETVAL -eq 0 ]; then
touch /var/lock/subsys/mysql-proxy
fi
}
stop() {
echo -n $"Stopping $prog: "
killproc -p $PROXY_PID -d 3 $prog
RETVAL=$?
echo
if [ $RETVAL -eq 0 ]; then
rm -f /var/lock/subsys/mysql-proxy
rm -f $PROXY_PID
fi
}
# See how we were called.
case "$1" in
start)
start
;;
stop)
stop
;;
restart)
stop
start
;;
condrestart|try-restart)
if status -p $PROXY_PIDFILE $prog >&/dev/null; then
stop
start
fi
;;
status)
status -p $PROXY_PID $prog
;;
*)
echo "Usage: $0 {start|stop|restart|reload|status|condrestart|try-restart}"
RETVAL=1
;;
esac
exit $RETVAL
[root@proxy ~]# chmod +x /etc/rc.d/init.d/mysql-proxy
[root@proxy ~]# chkconfig --add mysql-proxy

为服务脚本提供配置文件:


[root@proxy ~]# vim  /etc/sysconfig/mysql-proxy
# Options for mysql-proxy
ADMIN_USER="admin"
ADMIN_PASSWORD=""
ADMIN_ADDRESS=""
PROXY_ADDRESS=""
PROXY_USER="mysql-proxy"
PROXY_OPTIONS="--daemon --log-level=info --log-use-syslog"
其中最后一行,需要按实际场景进行修改,例如:
PROXY_OPTIONS="--daemon --log-level=info --log-use-syslog --plugins=proxy --plugins=admin --proxy-backend-addresses=172.16.20.6:3306 --proxy-read-only-backend-addresses=172.16.20.7:3306  --proxy-lua-script=/usr/local/mysql-proxy/share/doc/mysql-proxy/rw-splitting.lua"
说明:
--proxy-address=host:port ―――― 代理服务监听的地址和端口;
--admin-address=host:port ―――― 管理模块监听的地址和端口;
--proxy-lua-script=file_name ―――― 完成mysql代理功能的Lua脚本;
提供续写分离脚本,这个脚本mysql-proxy-0.8.3提供,所以这里我们只需要复制到对应的读取脚本的位置即可。
[root@proxy ~]# vim  /usr/local/mysql-proxy/share/doc/mysql-proxy/rw-splitting.lua
--[[ $%BEGINLICENSE%$
Copyright (c) 2007, 2012, Oracle and/or its affiliates. All rights reserved.
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License as
published by the Free Software Foundation; version 2 of the
License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
02110-1301  USA
$%ENDLICENSE%$ --]]
---
-- a flexible statement based load balancer with connection pooling
--
-- * build a connection pool of min_idle_connections for each backend and maintain
--   its size
-- *
--
--
local commands    = require("proxy.commands")
local tokenizer   = require("proxy.tokenizer")
local lb          = require("proxy.balance")
local auto_config = require("proxy.auto-config")
--- config
--
-- connection pool
if not proxy.global.config.rwsplit then
proxy.global.config.rwsplit = {
min_idle_connections = 4,
max_idle_connections = 8,
is_debug = false
}
end
---
-- read/write splitting sends all non-transactional SELECTs to the slaves
--
-- is_in_transaction tracks the state of the transactions
local is_in_transaction       = false
-- if this was a SELECT SQL_CALC_FOUND_ROWS ... stay on the same connections
local is_in_select_calc_found_rows = false
---
-- get a connection to a backend
--
-- as long as we don't have enough connections in the pool, create new connections
--
function connect_server()
local is_debug = proxy.global.config.rwsplit.is_debug
-- make sure that we connect to each backend at least ones to
-- keep the connections to the servers alive
--
-- on read_query we can switch the backends again to another backend
if is_debug then
print()
print("[connect_server] " .. proxy.connection.client.src.name)
end
local rw_ndx = 0
-- init all backends
for i = 1, #proxy.global.backends do
local s        = proxy.global.backends[i]
local pool     = s.pool -- we don't have a username yet, try to find a connections which is idling
local cur_idle = pool.users[""].cur_idle_connections
pool.min_idle_connections = proxy.global.config.rwsplit.min_idle_connections
pool.max_idle_connections = proxy.global.config.rwsplit.max_idle_connections
if is_debug then
print("  [".. i .."].connected_clients = " .. s.connected_clients)
print("  [".. i .."].pool.cur_idle     = " .. cur_idle)
print("  [".. i .."].pool.max_idle     = " .. pool.max_idle_connections)
print("  [".. i .."].pool.min_idle     = " .. pool.min_idle_connections)
print("  [".. i .."].type = " .. s.type)
print("  [".. i .."].state = " .. s.state)
end
-- prefer connections to the master
if s.type == proxy.BACKEND_TYPE_RW and
s.state ~= proxy.BACKEND_STATE_DOWN and
cur_idle < pool.min_idle_connections then
proxy.connection.backend_ndx = i
break
elseif s.type == proxy.BACKEND_TYPE_RO and
s.state ~= proxy.BACKEND_STATE_DOWN and
cur_idle < pool.min_idle_connections then
proxy.connection.backend_ndx = i
break
elseif s.type == proxy.BACKEND_TYPE_RW and
s.state ~= proxy.BACKEND_STATE_DOWN and
rw_ndx == 0 then
rw_ndx = i
end
end
if proxy.connection.backend_ndx == 0 then
if is_debug then
print("  [" .. rw_ndx .. "] taking master as default")
end
proxy.connection.backend_ndx = rw_ndx
end
-- pick a random backend
--
-- we someone have to skip DOWN backends
-- ok, did we got a backend ?
if proxy.connection.server then
if is_debug then
print("  using pooled connection from: " .. proxy.connection.backend_ndx)
end
-- stay with it
return proxy.PROXY_IGNORE_RESULT
end
if is_debug then
print("  [" .. proxy.connection.backend_ndx .. "] idle-conns below min-idle")
end
-- open a new connection
end
---
-- put the successfully authed connection into the connection pool
--
-- @param auth the context information for the auth
--
-- auth.packet is the packet
function read_auth_result( auth )
if is_debug then
print("[read_auth_result] " .. proxy.connection.client.src.name)
end
if auth.packet:byte() == proxy.MYSQLD_PACKET_OK then
-- auth was fine, disconnect from the server
proxy.connection.backend_ndx = 0
elseif auth.packet:byte() == proxy.MYSQLD_PACKET_EOF then
-- we received either a
--
-- * MYSQLD_PACKET_ERR and the auth failed or
-- * MYSQLD_PACKET_EOF which means a OLD PASSWORD (4.0) was sent
print("(read_auth_result) ... not ok yet");
elseif auth.packet:byte() == proxy.MYSQLD_PACKET_ERR then
-- auth failed
end
end
---
-- read/write splitting
function read_query( packet )
local is_debug = proxy.global.config.rwsplit.is_debug
local cmd      = commands.parse(packet)
local c        = proxy.connection.client
local r = auto_config.handle(cmd)
if r then return r end
local tokens
local norm_query
-- looks like we have to forward this statement to a backend
if is_debug then
print("[read_query] " .. proxy.connection.client.src.name)
print("  current backend   = " .. proxy.connection.backend_ndx)
print("  client default db = " .. c.default_db)
print("  client username   = " .. c.username)
if cmd.type == proxy.COM_QUERY then
print("  query             = "        .. cmd.query)
end
end
if cmd.type == proxy.COM_QUIT then
-- don't send COM_QUIT to the backend. We manage the connection
-- in all aspects.
proxy.response = {
type = proxy.MYSQLD_PACKET_OK,
}
if is_debug then
print("  (QUIT) current backend   = " .. proxy.connection.backend_ndx)
end
return proxy.PROXY_SEND_RESULT
end
-- COM_BINLOG_DUMP packet can't be balanced
--
-- so we must send it always to the master
if cmd.type == proxy.COM_BINLOG_DUMP then
-- if we don't have a backend selected, let's pick the master
--
if proxy.connection.backend_ndx == 0 then
proxy.connection.backend_ndx = lb.idle_failsafe_rw()
end
return
end
proxy.queries:append(1, packet, { resultset_is_needed = true })
-- read/write splitting
--
-- send all non-transactional SELECTs to a slave
if not is_in_transaction and
cmd.type == proxy.COM_QUERY then
tokens     = tokens or assert(tokenizer.tokenize(cmd.query))
local stmt = tokenizer.first_stmt_token(tokens)
if stmt.token_name == "TK_SQL_SELECT" then
is_in_select_calc_found_rows = false
local is_insert_id = false
for i = 1, #tokens do
local token = tokens[i]
-- SQL_CALC_FOUND_ROWS + FOUND_ROWS() have to be executed
-- on the same connection
-- print("token: " .. token.token_name)
-- print("  val: " .. token.text)
if not is_in_select_calc_found_rows and token.token_name == "TK_SQL_SQL_CALC_FOUND_ROWS" then
is_in_select_calc_found_rows = true
elseif not is_insert_id and token.token_name == "TK_LITERAL" then
local utext = token.text:upper()
if utext == "LAST_INSERT_ID" or
utext == "@@INSERT_ID" then
is_insert_id = true
end
end
-- we found the two special token, we can't find more
if is_insert_id and is_in_select_calc_found_rows then
break
end
end
-- if we ask for the last-insert-id we have to ask it on the original
-- connection
if not is_insert_id then
local backend_ndx = lb.idle_ro()
if backend_ndx > 0 then
proxy.connection.backend_ndx = backend_ndx
end
else
print("   found a SELECT LAST_INSERT_ID(), staying on the same backend")
end
end
end
-- no backend selected yet, pick a master
if proxy.connection.backend_ndx == 0 then
-- we don't have a backend right now
--
-- let's pick a master as a good default
--
proxy.connection.backend_ndx = lb.idle_failsafe_rw()
end
-- by now we should have a backend
--
-- in case the master is down, we have to close the client connections
-- otherwise we can go on
if proxy.connection.backend_ndx == 0 then
return proxy.PROXY_SEND_QUERY
end
local s = proxy.connection.server
-- if client and server db don't match, adjust the server-side
--
-- skip it if we send a INIT_DB anyway
if cmd.type ~= proxy.COM_INIT_DB and
c.default_db and c.default_db ~= s.default_db then
print("    server default db: " .. s.default_db)
print("    client default db: " .. c.default_db)
print("    syncronizing")
proxy.queries:prepend(2, string.char(proxy.COM_INIT_DB) .. c.default_db, { resultset_is_needed = true })
end
-- send to master
if is_debug then
if proxy.connection.backend_ndx > 0 then
local b = proxy.global.backends[proxy.connection.backend_ndx]
print("  sending to backend : " .. b.dst.name);
print("    is_slave         : " .. tostring(b.type == proxy.BACKEND_TYPE_RO));
print("    server default db: " .. s.default_db)
print("    server username  : " .. s.username)
end
print("    in_trans        : " .. tostring(is_in_transaction))
print("    in_calc_found   : " .. tostring(is_in_select_calc_found_rows))
print("    COM_QUERY       : " .. tostring(cmd.type == proxy.COM_QUERY))
end
return proxy.PROXY_SEND_QUERY
end
---
-- as long as we are in a transaction keep the connection
-- otherwise release it so another client can use it
function read_query_result( inj )
local is_debug = proxy.global.config.rwsplit.is_debug
local res      = assert(inj.resultset)
local flags    = res.flags
if inj.id ~= 1 then
-- ignore the result of the USE <default_db>
-- the DB might not exist on the backend, what do do ?
--
if inj.id == 2 then
-- the injected INIT_DB failed as the slave doesn't have this DB
-- or doesn't have permissions to read from it
if res.query_status == proxy.MYSQLD_PACKET_ERR then
proxy.queries:reset()
proxy.response = {
type = proxy.MYSQLD_PACKET_ERR,
errmsg = "can't change DB ".. proxy.connection.client.default_db ..
" to on slave " .. proxy.global.backends[proxy.connection.backend_ndx].dst.name
}
return proxy.PROXY_SEND_RESULT
end
end
return proxy.PROXY_IGNORE_RESULT
end
is_in_transaction = flags.in_trans
local have_last_insert_id = (res.insert_id and (res.insert_id > 0))
if not is_in_transaction and
not is_in_select_calc_found_rows and
not have_last_insert_id then
-- release the backend
proxy.connection.backend_ndx = 0
elseif is_debug then
print("(read_query_result) staying on the same backend")
print("    in_trans        : " .. tostring(is_in_transaction))
print("    in_calc_found   : " .. tostring(is_in_select_calc_found_rows))
print("    have_insert_id  : " .. tostring(have_last_insert_id))
end
end
---
-- close the connections if we have enough connections in the pool
--
-- @return nil - close connection
--         IGNORE_RESULT - store connection in the pool
function disconnect_client()
local is_debug = proxy.global.config.rwsplit.is_debug
if is_debug then
print("[disconnect_client] " .. proxy.connection.client.src.name)
end
-- make sure we are disconnection from the connection
-- to move the connection into the pool
proxy.connection.backend_ndx = 0
end


重启下服务即可。下面的测试我就不写了哈。^_^

你可能感兴趣的:(过滤,MySQL主从同步,半同步,基于SSL安全复制)