一、MHA优缺点
优点：
1、 MHA自动化主服务器故障转移，快速将从服务器晋级为主服务器(通常在10-30s)，而不影响复制的一致性，不会有性能损耗，容易安装，不必更改现有的部署环境，适用于任何存储引擎。
2、 MHA提供在线主服务器切换，改变先正运行的主服务器到另外一台上，这个过程只需0.5-2s的时间，这个时间内数据无法写入。MHA Manager通过ssh连接mysql slave服务器。
3、 使用半同步复制，可以大大降低数据丢失的风险。MHA可以与半同步复制结合起来。如果只有一个slave已经收到了最新的二进制日志，MHA可以将最新的二进制日志应用于其他所有的slave服务器上，因此他们彼此保持一致性。
缺点：
1、 虽然MHA试图从宕机的主服务器上保存二进制日志，但也会有问题。例如，如果主服务器硬件故障或无法通过ssh访问，MHA没法保存二进制日志，只进行故障转移而丢失最新数据。
2、 当主DB故障，切换到另外的服务器上后，即使恢复了原来的主DB，也不能立即加入整套MHA系统中，得重新部署。而且当发生一次切换后，管理节点的监控进程就会自动退出，需要用脚本来自动启动。另外还得删除app1.failover.complete这个文件，否则新的主DB出现问题MHA就不会切换了。
MHA节点包含三个脚本，依赖perl模块；
save_binary_logs:保存和复制当掉的主服务器二进制日志；
apply_diff_relay_logs:识别差异的relay log事件，并应用于其他salve服务器；
purge_relay_logs:清除relay log文件；
1.安装epel yum源：rpm -Uvh http://dl.iuscommunity.org/pub/ius/stable/Redhat/6/x86_64/epel-release-6-5.noarch.rpm
yum -y install perl-DBD-MySQL perl-Config-Tiny perl-Log-Dispatch perl-Parallel-ForkManager perl-Time-HiRes
2.下载管理端：wget https://mysql-master-ha.googlecode.com/files/mha4mysql-manager-0.55-0.el6.noarch.rpm
下载节点控制端：wget https://mysql-master-ha.googlecode.com/files/mha4mysql-node-0.54-0.el6.noarch.rpm
每个节点安装(包括管理节点，否则报错)：rpm -ivh mha4mysql-node-0.54-0.el6.noarch.rpm
只在管理节点安装：rpm -ivh mha4mysql-manager-0.55-0.el6.noarch.rpm
3.配置管理节点的配置文件：
vim /etc/app.cnf
[server default]
user=mhauser
password=mhauser123
manager_workdir=/masterha/app1
manager_log=/masterha/app1/manager.log
remote_workdir=/masterha/app1
ssh_user=root
repl_user=rep
repl_password=rep123
ping_interval=1
[server1]
hostname=192.168.0.183
ssh_port=2208
master_binlog_dir=/data
no_master=1
[server2]
hostname=192.168.0.201
ssh_port=2208
master_binlog_dir=/data
candidate_master=1
[server3]
hostname=192.168.0.200
ssh_port=2208
master_binlog_dir=/data
candidate_master=1
4.将0.183 0.200 0.201 三台机器建立ssh无密钥登录：
分别每台执行：ssh-keygen -t rsa
分别在每台机器/root/.ssh目录下创建：authorized_keys
把本机的与其他的两台机器的/root/.ssh/id_rsa.pub内容全部复制到authorized_keys内，三台机器同样的操作
5.测试ssh无密钥登录是否成功
管理端：
[root@mongodb1 .ssh]# masterha_check_ssh --conf=/etc/app.cnf
Wed Aug 21 12:39:18 2013 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Wed Aug 21 12:39:18 2013 - [info] Reading application default configurations from /etc/app.cnf..
Wed Aug 21 12:39:18 2013 - [info] Reading server configurations from /etc/app.cnf..
Wed Aug 21 12:39:18 2013 - [info] Starting SSH connection tests..
Wed Aug 21 12:39:26 2013 - [debug]
Wed Aug 21 12:39:18 2013 - [debug] Connecting via SSH from [email protected](192.168.0.183:2208) to [email protected](192.168.0.201:2208)..
Address 192.168.0.201 maps to localhost, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!
Wed Aug 21 12:39:25 2013 - [debug] ok.
Wed Aug 21 12:39:25 2013 - [debug] Connecting via SSH from [email protected](192.168.0.183:2208) to [email protected](192.168.0.200:2208)..
Address 192.168.0.200 maps to localhost, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!
Wed Aug 21 12:39:26 2013 - [debug] ok.
Wed Aug 21 12:39:42 2013 - [debug]
Wed Aug 21 12:39:18 2013 - [debug] Connecting via SSH from [email protected](192.168.0.201:2208) to [email protected](192.168.0.183:2208)..
Address 192.168.0.201 maps to localhost, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!
Address 192.168.0.183 maps to localhost, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!
Wed Aug 21 12:39:28 2013 - [debug] ok.
Wed Aug 21 12:39:28 2013 - [debug] Connecting via SSH from [email protected](192.168.0.201:2208) to [email protected](192.168.0.200:2208)..
Address 192.168.0.201 maps to localhost, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!
Address 192.168.0.200 maps to localhost, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!
Wed Aug 21 12:39:42 2013 - [debug] ok.
Wed Aug 21 12:39:46 2013 - [debug]
Wed Aug 21 12:39:19 2013 - [debug] Connecting via SSH from [email protected](192.168.0.200:2208) to [email protected](192.168.0.183:2208)..
Address 192.168.0.200 maps to localhost, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!
Address 192.168.0.183 maps to localhost, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!
Wed Aug 21 12:39:32 2013 - [debug] ok.
Wed Aug 21 12:39:32 2013 - [debug] Connecting via SSH from [email protected](192.168.0.200:2208) to [email protected](192.168.0.201:2208)..
Address 192.168.0.200 maps to localhost, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!
Address 192.168.0.201 maps to localhost, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!
Wed Aug 21 12:39:46 2013 - [debug] ok.
Wed Aug 21 12:39:46 2013 - [info] All SSH connection tests passed successfully.
出现上述字样表示成功
6.安装mysql，并配置主从。
主：grant replication slave on *.* to 'slave'@'192.168.0.%' identified by 'jinslavecin' with grant option;
Flush privileges;
主备上面也要添加主从同步账户，并且
set read_only=1
set relay_log_purge=0
从添加主库信息
slave stop;
change master to master_host='192.168.0.200',master_user='slave',master_password='jinslavecin',master_port=3307,master_log_file='binlog3307.000005',master_log_pos=358;
set read_only=1
set relay_log_purge=0
7.添加管理账号，并验证同步
每台服务器添加管理账号：
grant all on *.* to XXXXX@'192.168.0.%' identified by 'XXXX';
Flush privileges;
管理端验证同步：
[root@devmsyql-yg MHA]# masterha_check_repl --conf=/etc/app3307.cnf
Thu Aug 22 17:02:33 2013 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Thu Aug 22 17:02:33 2013 - [info] Reading application default configurations from /etc/app3307.cnf..
Thu Aug 22 17:02:33 2013 - [info] Reading server configurations from /etc/app3307.cnf..
Thu Aug 22 17:02:33 2013 - [info] MHA::MasterMonitor version 0.55.
Thu Aug 22 17:02:35 2013 - [info] Dead Servers:
Thu Aug 22 17:02:35 2013 - [info] Alive Servers:
Thu Aug 22 17:02:35 2013 - [info] 192.168.0.200(192.168.0.200:3307)
Thu Aug 22 17:02:35 2013 - [info] 192.168.0.201(192.168.0.201:3307)
Thu Aug 22 17:02:35 2013 - [info] 192.168.0.183(192.168.0.183:3307)
Thu Aug 22 17:02:35 2013 - [info] Alive Slaves:
Thu Aug 22 17:02:35 2013 - [info] 192.168.0.201(192.168.0.201:3307) Version=5.5.27-log (oldest major version between slaves) log-bin:enabled
Thu Aug 22 17:02:35 2013 - [info] Replicating from 192.168.0.200(192.168.0.200:3307)
Thu Aug 22 17:02:35 2013 - [info] Primary candidate for the new Master (candidate_master is set)
Thu Aug 22 17:02:35 2013 - [info] 192.168.0.183(192.168.0.183:3307) Version=5.5.27-log (oldest major version between slaves) log-bin:enabled
Thu Aug 22 17:02:35 2013 - [info] Replicating from 192.168.0.200(192.168.0.200:3307)
Thu Aug 22 17:02:35 2013 - [info] Not candidate for the new Master (no_master is set)
Thu Aug 22 17:02:35 2013 - [info] Current Alive Master: 192.168.0.200(192.168.0.200:3307)
Thu Aug 22 17:02:35 2013 - [info] Checking slave configurations..
Thu Aug 22 17:02:35 2013 - [info] read_only=1 is not set on slave 192.168.0.201(192.168.0.201:3307).
Thu Aug 22 17:02:35 2013 - [warning] relay_log_purge=0 is not set on slave 192.168.0.201(192.168.0.201:3307).
Thu Aug 22 17:02:35 2013 - [info] read_only=1 is not set on slave 192.168.0.183(192.168.0.183:3307).
Thu Aug 22 17:02:35 2013 - [warning] relay_log_purge=0 is not set on slave 192.168.0.183(192.168.0.183:3307).
Thu Aug 22 17:02:35 2013 - [info] Checking replication filtering settings..
Thu Aug 22 17:02:35 2013 - [info] binlog_do_db= User0,User1, binlog_ignore_db=
Thu Aug 22 17:02:35 2013 - [info] Replication filtering check ok.
Thu Aug 22 17:02:35 2013 - [info] Starting SSH connection tests..
Thu Aug 22 17:03:18 2013 - [info] All SSH connection tests passed successfully.
Thu Aug 22 17:03:18 2013 - [info] Checking MHA Node version..
Thu Aug 22 17:03:34 2013 - [info] Version check ok.
Thu Aug 22 17:03:34 2013 - [info] Checking SSH publickey authentication settings on the current master..
Thu Aug 22 17:03:39 2013 - [warning] HealthCheck: Got timeout on checking SSH connection to 192.168.0.200! at /usr/share/perl5/vendor_perl/MHA/HealthCheck.pm line 298.
Thu Aug 22 17:03:39 2013 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers..
Thu Aug 22 17:03:39 2013 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user='mhauser' --slave_host=192.168.0.201 --slave_ip=192.168.0.201 --slave_port=3307 --workdir=/masterha/app1 --target_version=5.5.27-log --manager_version=0.55 --relay_log_info=/data/mysql/log3307/relaylog/relaylog3307 --relay_dir=/data/mysql/data3307/ --slave_pass=xxx
Thu Aug 22 17:03:39 2013 - [info] Connecting to [email protected](192.168.0.201:2208)..
Checking slave recovery environment settings..
Opening /data/mysql/log3307/relaylog/relaylog3307 ... ok.
Relay log found at /data/mysql/log3307/relaylog, up to relaylog3307.000004
Temporary relay log file is /data/mysql/log3307/relaylog/relaylog3307.000004
Testing mysql connection and privileges.. done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Thu Aug 22 17:03:45 2013 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user='mhauser' --slave_host=192.168.0.183 --slave_ip=192.168.0.183 --slave_port=3307 --workdir=/masterha/app1 --target_version=5.5.27-log --manager_version=0.55 --relay_log_info=/data/mysql/log2/relaylog/relaylog --relay_dir=/data/mysql/data2/ --slave_pass=xxx
Thu Aug 22 17:03:45 2013 - [info] Connecting to [email protected](192.168.0.183:2208)..
Checking slave recovery environment settings..
Opening /data/mysql/log2/relaylog/relaylog ... ok.
Relay log found at /data/mysql/log2/relaylog, up to relaylog.000020
Temporary relay log file is /data/mysql/log2/relaylog/relaylog.000020
Testing mysql connection and privileges.. done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Thu Aug 22 17:03:56 2013 - [info] Slaves settings check done.
Thu Aug 22 17:03:56 2013 - [info]
192.168.0.200 (current master)
+--192.168.0.201
+--192.168.0.183
Thu Aug 22 17:03:56 2013 - [info] Checking replication health on 192.168.0.201..
Thu Aug 22 17:03:56 2013 - [info] ok.
Thu Aug 22 17:03:56 2013 - [info] Checking replication health on 192.168.0.183..
Thu Aug 22 17:03:56 2013 - [info] ok.
Thu Aug 22 17:03:56 2013 - [warning] master_ip_failover_script is not defined.
Thu Aug 22 17:03:56 2013 - [warning] shutdown_script is not defined.
Thu Aug 22 17:03:56 2013 - [info] Got exit code 0 (Not master dead).
MySQL Replication Health is OK.
出现上述字样证明同步正常。
可能还会遇到如下错误:
Can’t exec “mysqlbinlog”: No such file or directory at /usr/share/perl5/vendor_perl/MHA/BinlogManager.pm line 99.
mysqlbinlog version not found!
解决办法将mysqlbinlog所在目录放到每台机器的PATH当中.
#vi ~/.bashrc或vi /etc/bashrc,然后在文件末尾添加
PATH="$PATH:/usr/local/mysql/bin"
export PATH
8.启动MHA管理：
nohup masterha_manager --conf=/etc/app3306.cnf /masterha/app1/manager.log 2>&1 &
masterha_stop --conf=/etc/app3306/cnf
9..中继日志文件清理
由于slave机器都设置relay_log_purge=0(禁用中继日志自动清除),因此必须定期对旧的中继日志文件进行清理.
每台slave设置cron job定期清除中继日志:
配置清除relay-log 的计划任务
00 00 * * * /usr/bin/purge_relay_logs --user=root --password=rootpass --disable_relay_log_purge >> /var/log/purge_relay_logs.log 2>&1
10.恢复原来的主服务器
–orig_master_is_new_slave切换时加上此参数是将原master变为slave节点，如果不加此参数，原来的master将不启动
–running_updates_limit=10000 切换时候选master如果有延迟的话，mha切换不能成功，加上此参数表示延迟在此时间范围内都可切换（单位为s），但是切换的时间长短是由recover时relay日志的大小决定
手动在线切换mha，切换时需要将在运行的mha停掉后才能切换。
在备库先执行DDL，一般先stop slave，一般不记录mysql日志，可以通过set SQL_LOG_BIN = 0实现。然后进行一次主备切换操作，再在原来的主库上执行DDL。这种方法适用于增减索引，如果是增加字段就需要额外注意
ps auxf|grep masterha_manager|grep -v grep|awk '{print $2}'|xargs kill
masterha_master_switch --master_state=alive --conf=/etc/app3306.cnf --orig_master_is_new_slave
11.mha上层可以通过keepalive部署VIP，程序连接数据库使用VIP，从而实现后台数据库的故障切换透明化.master_ip_failover脚本附件中奉上！
在发现主库down,根据输出日志信息，mha做一下操作
1：发现主库不可用，masterha_secondary_check 和save_binary_logs 两脚本检测从库是否可达，检测主库binlog 是否可用。并发出failover 信号。
2:检查主库server1，三次不可通后确认无法使用，确定server1 的mysql 不可用，但主机ssh 可达。
3:开始Dead Master Shutdown Phase 阶段，调用master_ip_failover 脚本.
4：开启master_recovry_phase，找到所有slaves 己读到的binlog 位置，若生产环境下从库读取不一致，则以最新的为准从dead_master 上获取最新的binlog。将dead_master存在但未同步到从上的binlog 保存下来
5：开始Determining New Master Phase 段段，寻找备用master，原则为设置candidate_master， 拥有最新的binlog。标记为no_master 的server永不会设为备主。然后开始failover，mysql 架构由120( 主库) ，121（备主）,122为从库。
6：开始New Master Diff Log Generation Phase 阶段，应用差异日志到新主库。确定新主当前的master_log_file,master_log_pos。
7：应用差异日志到所有从库，应用完成后，关闭新主server2 的read_only 选项，对应用可读可写。
8：在所有从库上执行change master。清除新主库的slave 信息，reset slave，完成failover。
9：最后调用send_report 脚本，发送报警短信或是邮件通知dba.
(12)主从配置keepalived ，主从的priority设置不同，权重大的为主状态
[root@centos1 keepalived-1.2.2]# more /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id mha
}
vrrp_script check_run {
script "/root/check_mysql.sh"
interval 1
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
track_script {
check_run
}
virtual_ipaddress {
192.168.8.123
}
}
[root@centos1 ~] vi check_mysql.sh
#!/bin/bash
MYSQL=/usr/local/mysql/bin/mysql
MYSQL_HOST=127.0.0.1
MYSQL_USER=root
MYSQL_PASSWORD=
CHECK_TIME=3
#mysql is working MYSQL_OK is 1 , mysql down MYSQL_OK is 0
MYSQL_OK=1
function check_mysql_helth (){
$MYSQL -h $MYSQL_HOST -u $MYSQL_USER -e "show status;" >/dev/null 2>&1
if [ $? = 0 ] ;then
MYSQL_OK=1
else
MYSQL_OK=0
fi
return $MYSQL_OK
}
while [ $CHECK_TIME -ne 0 ]
do
let "CHECK_TIME -= 1"
check_mysql_helth
if [ $MYSQL_OK = 1 ] ; then
CHECK_TIME=0
exit 0
fi
if [ $MYSQL_OK -eq 0 ] && [ $CHECK_TIME -eq 0 ]
then
pkill keepalived
exit 1
fi
sleep 1
done
此脚本不能检测MySQL 正常运行，不能正常，就关闭keepalived,释放虚拟IP.