前言
构建一个Mysql分布式集群需要对数据库的原理有较深的认识，而深奥复杂的数据库原理往往让很多读者中途放弃。本系列尝试用最简单的语言介绍Mysql分布式集群涉及到的原理，并结合实践的方式给读者介绍如何构建一个Mysql分布式集群。
目录
1.Mysql分布式集群（一）主从复制
2.Mysql分布式集群（二）主主复制和高可用
主主复制
主从复制的情况下，主节点只有一个，当主节点如果发生意外宕机则整个数据库系统就不在可用，因此需要对主节点做一个备份，一旦主节点发生问题，备份节点能立刻替换，为了保持备份节点和主节点同步，也为了保持Slave节点能同步备份节点，需要对主节点和备份节点做主主复制。
实验环境：
Master ip :192.168.179.132
Master backup ip: 192.168.179.134
Slave1 ip :192.168.179.133
Slave1 ip :192.168.179.131
操作系统：Ubuntu14.04
mysql版本：5.7.20
keepalived版本：keepalived-1.3.9
高可用结构
本文通过一个简单主主复制和两个主从复制完成一个简单的高可用例子，架构图如下所示。其中Master(ip：192.168.179.132)为主节点，Master backup (ip:192.168.179.134)为主节点的从节点，同时主节点也是该节点的从节点，二者互为主从。本文采用keepalived监听双主的状态，一旦主节点发生故障，则切换到备用节点上去。
高可用架构
主主复制和主从复制
主主复制采用主从复制的一样的策略，在配置好从节点之后，打开从节点的bin log日志，并在主节点端，指向从节点的bin log日志即可。
Master端配置
1.用户授权
mysql> grant all on *.* to 'master'@'192.168.179.%' identified by '123456';
Query OK, 0 rows affected, 1 warning (0.02 sec)
mysql> select user,host from mysql.user;
+---------------+---------------+
| user | host |
+---------------+---------------+
| master | 192.168.179.% |
| mysql.session | localhost |
| mysql.sys | localhost |
| root | localhost |
+---------------+---------------+
4 rows in set (0.30 sec)
mysql> show grants for 'master'@'192.168.179.%';
+---------------------------------------------------------+
| Grants for [email protected].% |
+---------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO 'master'@'192.168.179.%' |
+---------------------------------------------------------+
1 row in set (0.00 sec)
以上我们授予master用户所有库所有表的权限，在Master backup端远程登录Master的mysq。如果能登录则该步成功，如果出现下方错误，则参考这里中的方法解决
root@ubuntu:~# mysql -uslave -p -h192.168.179.128;
Enter password:
ERROR 2003 (HY000): Can't connect to MySQL server on '192.168.179.128' (111)
2.修改Mysql配置文件
root@ubuntu:~# vim /etc/mysql/mysql.conf.d/mysqld.cnf
[mysqld]
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
datadir = /var/lib/mysql
log-error = /var/log/mysql/error.log
# By default we only accept connections from localhost
#bind-address = 127.0.0.1
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
server-id=1
log-bin =mysql-bin
log-slave-updates=true
auto_increment_increment=2
auto_increment_offset=1
需要注意的是一下两点：
1.主主复制为了避免主键冲突，采用以下的方式的设置主键
auto_increment_increment=2 #每次以2为步长进行增长
auto_increment_offset=1 #从1开始
2.主主复制中，作为从的一段，需要配置如下变量。如果不配置的话，数据复制事件不会更新到bin log文件中，因此真正的Slave节点捕捉不到bin log日志的改变，会出现Master和Slave数据不一致的问题
log-slave-updates=true
Master Backup端配置
1.用户授权
mysql> grant all on *.* to 'master'@'192.168.179.%' identified by '123456';
Query OK, 0 rows affected, 1 warning (0.02 sec)
mysql> select user,host from mysql.user;
+---------------+---------------+
| user | host |
+---------------+---------------+
| master | 192.168.179.% |
| mysql.session | localhost |
| mysql.sys | localhost |
| root | localhost |
+---------------+---------------+
4 rows in set (0.30 sec)
mysql> show grants for 'master'@'192.168.179.%';
+---------------------------------------------------------+
| Grants for [email protected].% |
+---------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO 'master'@'192.168.179.%' |
+---------------------------------------------------------+
1 row in set (0.00 sec)
以上我们授予master用户所有库所有表的权限，在Master backup端远程登录Master的mysq。如果能登录则该步成功，如果出现下方错误，则参考这里中的方法解决
root@ubuntu:~# mysql -uslave -p -h192.168.179.128;
Enter password:
ERROR 2003 (HY000): Can't connect to MySQL server on '192.168.179.128' (111)
2.修改Mysql配置文件
root@ubuntu:~# vim /etc/mysql/mysql.conf.d/mysqld.cnf
[mysqld]
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
datadir = /var/lib/mysql
log-error = /var/log/mysql/error.log
# By default we only accept connections from localhost
#bind-address = 127.0.0.1
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
server-id=1
log-bin =mysql-bin
log-slave-updates=true
auto_increment_increment=2
auto_increment_offset=2
需要注意的是一下两点：
1.设置主键方式
auto_increment_increment=2 #每次以2为步长进行增长
auto_increment_offset=1 #从2开始
2.复制事件同步到bin log
log-slave-updates=true
从节点配置
1.Master端从配置
先在Master backup 端查看bin log日志状态
mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000001 | 154 | | | |
+------------------+----------+--------------+------------------+-------------------+
1 row in set (0.03 sec)
指向Master backup 的bin log
mysql> change master to master_host='192.168.179.134',
-> master_user='master',
-> master_password='123456',
-> master_log_file='mysql-bin.000001', master_log_pos=154;
Query OK, 0 rows affected, 2 warnings (0.34 sec)
mysql> start slave;
Query OK, 0 rows affected (0.15 sec)
mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.179.134
Master_User: master
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000001
Read_Master_Log_Pos: 154
Relay_Log_File: ubuntu-relay-bin.000002
Relay_Log_Pos: 320
Relay_Master_Log_File: mysql-bin.000001
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 154
Relay_Log_Space: 528
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 2
Master_UUID: e7c00798-b94e-11e7-8656-000c29f0d4cf
Master_Info_File: /var/lib/mysql/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set:
Auto_Position: 0
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)
可以发现已经配置成功.
2.Master backup 端
步骤同Master端配置（略）
3.Slave1端从配置
配置方法同Master端配置，只需要注意的是要指向Master的bin log（略）
4.Slave2端从配置
配置方法同Master端配置，只需要注意的是要指向Master backup的bin log（略）
此时配置工作已经完成，可以通过对Master表操作来测试，主主，主从复制是否成功。
Mysql高可用
Keepalived的作用是检测服务器的状态，如果有一台web服务器宕机，或工作出现故障，Keepalived将检测到，并将有故障的服务器从系统中剔除，同时使用其他服务器代替该服务器的工作，当服务器工作正常后Keepalived自动将服务器加入到服务器群中，这些工作全部自动完成，不需要人工干涉，需要人工做的只是修复故障的服务器，本文采用keepalived用于检测mysql的状态，以达到mysql高可用的状态。
Keepalived原理
Keepalived采用的是为多台机器配置同一个虚拟地址，该虚拟地址有且只有一个真实的节点，该节点宕机之后，该虚拟地址会会自动（根据一定的策略）映射到一下个节点上。
keepalived安装
以下操作在Master和Master backup节点上同时进行。
下载keepalived-1.3.9.tar.gz，上传到Master和Master backup上
解压缩后进入到 keepalived-1.3.9/目录中，由于keepalived采用的是源码安装，因此需要对它进行编译安装，并且需要依赖工具：
apt-get install libssl-dev
apt-get install openssl
apt-get install libpopt-dev
以上完成之后，可以进行如下操作：
root@ubuntu:/software/keepalived-1.3.9# ./configure
......
Keepalived configuration
------------------------
Keepalived version : 1.3.9
Compiler : gcc
Preprocessor flags :
Compiler flags : -Wall -Wunused -Wstrict-prototypes -Wextra -g -O2
Linker flags :
Extra Lib : -lcrypto -lssl
Use IPVS Framework : Yes
IPVS use libnl : No
IPVS syncd attributes : No
IPVS 64 bit stats : No
fwmark socket support : Yes
Use VRRP Framework : Yes
Use VRRP VMAC : Yes
Use VRRP authentication : Yes
With ip rules/routes : Yes
SNMP vrrp support : No
SNMP checker support : No
SNMP RFCv2 support : No
SNMP RFCv3 support : No
DBUS support : No
SHA1 support : No
Use Debug flags : No
Use Json output : No
Stacktrace support : No
Memory alloc check : No
libnl version : None
Use IPv4 devconf : No
Use libiptc : No
Use libipset : No
init type : upstart
Build genhash : Yes
Build documentation : No
root@ubuntu:/software/keepalived-1.3.9#make
root@ubuntu:/software/keepalived-1.3.9#make install
keepalived配置
1.Master 端
修改keepalived配置文件
root@ubuntu:~# vim /usr/local/etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs{
router_id LVS_MASTER
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.179.2/24 #虚拟主机ip
}
}
virtual_server 192.168.179.2 3306 {#设置虚拟ip地址和端口号
delay_loop 6
lb_algo rr
lb_kind NAT
persistence_timeout 50
protocol TCP #采用TCP协议
real_server 192.168.179.132 3306 {#检测的真实节点Ip和端口号
weight 1
notify_down /usr/local/etc/keepalived/mysql.sh #如果发现异常触发该脚本
TCP_CHECK{
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
2.Master backup 端
修改keepalived配置文件
root@ubuntu:~# vim /usr/local/etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs{
router_id LVS_MASTER
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.179.2/24 #虚拟主机ip
}
}
virtual_server 192.168.179.2 3306 {#设置虚拟ip地址和端口号
delay_loop 6
lb_algo rr
lb_kind NAT
persistence_timeout 50
protocol TCP #采用TCP协议
real_server 192.168.179.134 3306 {#检测的真实节点Ip和端口号
weight 1
notify_down /usr/local/etc/keepalived/mysql.sh #如果发现异常触发该脚本
TCP_CHECK{
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
以上的配置同Master中的配置，唯一不同的是real_server中的ip地址。如此，Master和Master backup上安装的keepalived 才能分别检测本机的Mysql服务器。另外，当Mysql 出现问题的时候会触发一个脚本，用于关掉本机的keepalived进程，之后才能进行切换到另外一个节点上去，该脚本如下：
root@ubuntu:~# vim /usr/local/etc/keepalived/mysql.sh
#!/bin/bash
pkill keepalived
在Master和Master backup上分别启动keepalived
#为keepalived脚本增加运行权限
root@ubuntu:/usr/local/sbin# chmod +x keepalived
#手工的制定配置文件启动
root@ubuntu:/usr/local/sbin# ./keepalived -D -f /usr/local/etc/keepalived/keepalived.conf
root@ubuntu:/usr/local/sbin# ps -aux |grep keepalived;
Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html
root 29710 0.4 0.0 13064 732 ? Ss 04:00 0:00 ./keepalived -D -f /usr/local/etc/keepalived/keepalived.conf
root 29711 0.5 0.1 21496 1224 ? S 04:00 0:00 ./keepalived -D -f /usr/local/etc/keepalived/keepalived.conf
root 29712 1.1 0.1 15160 1216 ? S 04:00 0:00 ./keepalived -D -f /usr/local/etc/keepalived/keepalived.conf
root 29714 6.0 0.0 10464 920 pts/2 S+ 04:00 0:00 grep --color=auto keepalived
至此，整个配置都已经完成。下边开始测试
Mysql高可用实验
1.查看keepalived虚拟的地址在哪一台机器上
Master端
root@ubuntu:/usr/local/sbin# ip a
1: lo: mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:f0:d4:cf brd ff:ff:ff:ff:ff:ff
inet 192.168.179.132/24 brd 192.168.179.255 scope global eth0
valid_lft forever preferred_lft forever
inet 192.168.179.2/24 scope global secondary eth0
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fef0:d4cf/64 scope link
valid_lft forever preferred_lft forever
Master backup端
root@ubuntu:/var/log# ip a
1: lo: mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:ba:4d:47 brd ff:ff:ff:ff:ff:ff
inet 192.168.179.134/24 brd 192.168.179.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:feba:4d47/64 scope link
valid_lft forever preferred_lft forever
可以发现只有Master 中eth0中增加了192.168.179.2这个虚拟ip.此时kill 掉master中keepalived线程会发现Master backup 中增加了一个虚拟ip. 在mysql为应用提供服务的时候，只需要提供虚拟ip即可，当双主中的节点中一个节点宕机后，另外一个节点可以继续提供服务。