安装MHA踩过的坑

MHA 0.58+Mysql5.7+redhat7.4 已测试通过。

之前在redhat6.10上安装，在家里电脑安装OK，一样的包，拿到公司安装，失败了。很郁闷。随后改成redhat7.4安装，安装成功，走了不少弯路。把坑写出来。

一些包的下载地方 ，在使用redhat6.10的时候，下载过，安装成功了，redhat7.4没有从这里下载直接光盘yum了。

http://rpm.pbone.net

-- 坑1 ，

[root@mysql1 bin]# masterha_check_ssh --conf=/usr/local/mha/mha.conf
Fri Apr 26 16:46:32 2019 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Fri Apr 26 16:46:32 2019 - [info] Reading application default configuration from /usr/local/mha/mha.conf..
Fri Apr 26 16:46:32 2019 - [info] Reading server configuration from /usr/local/mha/mha.conf..
Fri Apr 26 16:46:32 2019 - [info] Starting SSH connection tests..
Fri Apr 26 16:46:33 2019 - [error][/usr/local/share/perl5/MHA/SSHCheck.pm, ln63] 
Fri Apr 26 16:46:32 2019 - [debug]  Connecting via SSH from root@mysql1(192.168.54.230:22) to root@mysql2(192.168.54.231:22)..
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
Fri Apr 26 16:46:32 2019 - [error][/usr/local/share/perl5/MHA/SSHCheck.pm, ln111] SSH connection from root@mysql1(192.168.54.230:22) to root@mysql2(192.168.54.231:22) failed!
Fri Apr 26 16:46:35 2019 - [debug] 
Fri Apr 26 16:46:33 2019 - [debug]  Connecting via SSH from root@mysql2(192.168.54.231:22) to root@mysql1(192.168.54.230:22)..
Fri Apr 26 16:46:33 2019 - [debug]   ok.
Fri Apr 26 16:46:33 2019 - [debug]  Connecting via SSH from root@mysql2(192.168.54.231:22) to root@mysql3(192.168.54.232:22)..
Fri Apr 26 16:46:34 2019 - [debug]   ok.
Fri Apr 26 16:46:35 2019 - [debug] 
Fri Apr 26 16:46:33 2019 - [debug]  Connecting via SSH from root@mysql3(192.168.54.232:22) to root@mysql1(192.168.54.230:22)..
Fri Apr 26 16:46:34 2019 - [debug]   ok.
Fri Apr 26 16:46:34 2019 - [debug]  Connecting via SSH from root@mysql3(192.168.54.232:22) to root@mysql2(192.168.54.231:22)..
Fri Apr 26 16:46:34 2019 - [debug]   ok.
SSH Configuration Check Failed!
 at /usr/local/bin/masterha_check_ssh line 44.
[root@mysql1 bin]# 

原因： ssh互信配置的有问题。其实检查了下，ssh mysql1  ,ssh mysql2 ,ssh mysql3 都没有问题。发现之前配置的时候，是使用ssh-copy-id，随后老老实实配置，写到authorize文件里面，可以了。 感觉不可思议。可能自己对ssh这块了解的有点少。

-- 坑2

[root@mysql1 bin]# masterha_check_repl --conf=/usr/local/mha/mha.conf
Sun Apr 28 10:18:11 2019 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Sun Apr 28 10:18:11 2019 - [info] Reading application default configuration from /usr/local/mha/mha.conf..
Sun Apr 28 10:18:11 2019 - [info] Reading server configuration from /usr/local/mha/mha.conf..
Sun Apr 28 10:18:11 2019 - [info] MHA::MasterMonitor version 0.58.
Creating directory /usr/localmha.. done.
Sun Apr 28 10:18:11 2019 - [error][/usr/local/share/perl5/MHA/Server.pm, ln180] Got MySQL error when connecting mysql1(192.168.54.230:3306) :1045:Access denied for user 'root'@'mysql1' (using password: YES), but this is not a MySQL crash. Check MySQL server settings.
Sun Apr 28 10:18:11 2019 - [error][/usr/local/share/perl5/MHA/ServerManager.pm, ln301]  at /usr/local/share/perl5/MHA/ServerManager.pm line 297.
Sun Apr 28 10:18:11 2019 - [error][/usr/local/share/perl5/MHA/Server.pm, ln180] Got MySQL error when connecting mysql3(192.168.54.232:3306) :1045:Access denied for user 'root'@'mysql1' (using password: YES), but this is not a MySQL crash. Check MySQL server settings.
Sun Apr 28 10:18:11 2019 - [error][/usr/local/share/perl5/MHA/ServerManager.pm, ln301]  at /usr/local/share/perl5/MHA/ServerManager.pm line 297.
Sun Apr 28 10:18:11 2019 - [error][/usr/local/share/perl5/MHA/Server.pm, ln180] Got MySQL error when connecting mysql2(192.168.54.231:3306) :1045:Access denied for user 'root'@'mysql1' (using password: YES), but this is not a MySQL crash. Check MySQL server settings.
Sun Apr 28 10:18:11 2019 - [error][/usr/local/share/perl5/MHA/ServerManager.pm, ln301]  at /usr/local/share/perl5/MHA/ServerManager.pm line 297.
Sun Apr 28 10:18:12 2019 - [error][/usr/local/share/perl5/MHA/ServerManager.pm, ln309] Got fatal error, stopping operations
Sun Apr 28 10:18:12 2019 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations.  at /usr/local/share/perl5/MHA/MasterMonitor.pm line 329.
Sun Apr 28 10:18:12 2019 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Sun Apr 28 10:18:12 2019 - [info] Got exit code 1 (Not master dead).

MySQL Replication Health is NOT OK!

原因，这个原因应该比较好理解，就是没有权限访问mysql ，重新授权就可以了。

create user 'root'@'%' identified by 'mysql';
grant all on *.* to 'root'@'%';
flush privileges;

-- 坑3 ，这个主从上忘记设置 log-bin了， 设置后ok 。

[root@mysql1 mha]# masterha_check_repl --conf=/usr/local/mha/mha.conf
Sun Apr 28 11:16:54 2019 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Sun Apr 28 11:16:54 2019 - [info] Reading application default configuration from /usr/local/mha/mha.conf..
Sun Apr 28 11:16:54 2019 - [info] Reading server configuration from /usr/local/mha/mha.conf..
Sun Apr 28 11:16:54 2019 - [info] MHA::MasterMonitor version 0.58.
Sun Apr 28 11:16:55 2019 - [info] GTID failover mode = 0
Sun Apr 28 11:16:55 2019 - [info] Dead Servers:
Sun Apr 28 11:16:55 2019 - [info] Alive Servers:
Sun Apr 28 11:16:55 2019 - [info]   mysql1(192.168.54.230:3306)
Sun Apr 28 11:16:55 2019 - [info]   mysql2(192.168.54.231:3306)
Sun Apr 28 11:16:55 2019 - [info]   mysql3(192.168.54.232:3306)
Sun Apr 28 11:16:55 2019 - [info] Alive Slaves:
Sun Apr 28 11:16:55 2019 - [info]   mysql1(192.168.54.230:3306)  Version=5.7.25 (oldest major version between slaves) log-bin:disabled
Sun Apr 28 11:16:55 2019 - [info]     Replicating from 192.168.54.231(192.168.54.231:3306)
Sun Apr 28 11:16:55 2019 - [info]     Not candidate for the new Master (no_master is set)
Sun Apr 28 11:16:55 2019 - [info]   mysql3(192.168.54.232:3306)  Version=5.7.25 (oldest major version between slaves) log-bin:disabled
Sun Apr 28 11:16:55 2019 - [info]     Replicating from 192.168.54.231(192.168.54.231:3306)
Sun Apr 28 11:16:55 2019 - [info]     Primary candidate for the new Master (candidate_master is set)
Sun Apr 28 11:16:55 2019 - [info] Current Alive Master: mysql2(192.168.54.231:3306)
Sun Apr 28 11:16:55 2019 - [info] Checking slave configurations..
Sun Apr 28 11:16:55 2019 - [info]  read_only=1 is not set on slave mysql1(192.168.54.230:3306).
Sun Apr 28 11:16:55 2019 - [warning]  relay_log_purge=0 is not set on slave mysql1(192.168.54.230:3306).
Sun Apr 28 11:16:55 2019 - [warning]  log-bin is not set on slave mysql1(192.168.54.230:3306). This host cannot be a master.
Sun Apr 28 11:16:55 2019 - [info]  read_only=1 is not set on slave mysql3(192.168.54.232:3306).
Sun Apr 28 11:16:55 2019 - [warning]  relay_log_purge=0 is not set on slave mysql3(192.168.54.232:3306).
Sun Apr 28 11:16:55 2019 - [warning]  log-bin is not set on slave mysql3(192.168.54.232:3306). This host cannot be a master.
Sun Apr 28 11:16:55 2019 - [info] Checking replication filtering settings..
Sun Apr 28 11:16:55 2019 - [info]  binlog_do_db= , binlog_ignore_db= 
Sun Apr 28 11:16:55 2019 - [info]  Replication filtering check ok.
Sun Apr 28 11:16:55 2019 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln364] None of slaves can be master. Check failover configuration file or log-bin settings in my.cnf
Sun Apr 28 11:16:55 2019 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations.  at /usr/local/bin/masterha_check_repl line 48.
Sun Apr 28 11:16:55 2019 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Sun Apr 28 11:16:55 2019 - [info] Got exit code 1 (Not master dead).

MySQL Replication Health is NOT OK!

-- 坑4

Can't exec "mysqlbinlog": No such file or directory at /usr/local/share/perl5/MHA/BinlogManager.pm line 106.
mysqlbinlog version command failed with rc 1:0, please verify PATH, LD_LIBRARY_PATH, and client options
 at /usr/local/bin/apply_diff_relay_logs line 532.
Sun Apr 28 13:53:52 2019 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln208] Slaves settings check failed!
Sun Apr 28 13:53:52 2019 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln416] Slave configuration failed.
Sun Apr 28 13:53:52 2019 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations.  at /usr/local/bin/masterha_check_repl line 48.
Sun Apr 28 13:53:52 2019 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Sun Apr 28 13:53:52 2019 - [info] Got exit code 1 (Not master dead).

MySQL Replication Health is NOT OK!

原因，需要设置软连接

ln -s /mysql/mysql/bin/mysqlbinlog  /usr/local/bin/mysqlbinlog 

-- 坑5 ，同坑4，需要设置软连接

sh: mysql: command not found
mysql command failed with rc 127:0!

解决方法

ln -s /mysql/mysql/bin/mysql  /usr/local/bin/mysql

-- 坑6 ，在线切换有问题 。

root@mysql1 bin]# masterha_master_switch --conf=/usr/local/mha/mha.conf --master_state=alive --new_master_host=192.168.54.232 --orig_master_is_new_slave
Mon Apr 29 10:02:27 2019 - [info] MHA::MasterRotate version 0.58.
Mon Apr 29 10:02:27 2019 - [info] Starting online master switch..
Mon Apr 29 10:02:27 2019 - [info] 
Mon Apr 29 10:02:27 2019 - [info] * Phase 1: Configuration Check Phase..
Mon Apr 29 10:02:27 2019 - [info] 
Mon Apr 29 10:02:27 2019 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Mon Apr 29 10:02:27 2019 - [info] Reading application default configuration from /usr/local/mha/mha.conf..
Mon Apr 29 10:02:27 2019 - [info] Reading server configuration from /usr/local/mha/mha.conf..
Mon Apr 29 10:02:28 2019 - [info] GTID failover mode = 1
Mon Apr 29 10:02:28 2019 - [info] Current Alive Master: mysql2(192.168.54.231:3306)
Mon Apr 29 10:02:28 2019 - [info] Alive Slaves:
Mon Apr 29 10:02:28 2019 - [info]   mysql1(192.168.54.230:3306)  Version=5.7.25-log (oldest major version between slaves) log-bin:enabled
Mon Apr 29 10:02:28 2019 - [info]     GTID ON
Mon Apr 29 10:02:28 2019 - [info]     Replicating from 192.168.54.231(192.168.54.231:3306)
Mon Apr 29 10:02:28 2019 - [info]     Not candidate for the new Master (no_master is set)
Mon Apr 29 10:02:28 2019 - [info]   mysql3(192.168.54.232:3306)  Version=5.7.25-log (oldest major version between slaves) log-bin:enabled
Mon Apr 29 10:02:28 2019 - [info]     GTID ON
Mon Apr 29 10:02:28 2019 - [info]     Replicating from 192.168.54.231(192.168.54.231:3306)
Mon Apr 29 10:02:28 2019 - [info]     Primary candidate for the new Master (candidate_master is set)

It is better to execute FLUSH NO_WRITE_TO_BINLOG TABLES on the master before switching. Is it ok to execute on mysql2(192.168.54.231:3306)? (YES/no): yes
Mon Apr 29 10:02:46 2019 - [info] Executing FLUSH NO_WRITE_TO_BINLOG TABLES. This may take long time..
Mon Apr 29 10:02:46 2019 - [info]  ok.
Mon Apr 29 10:02:46 2019 - [info] Checking MHA is not monitoring or doing failover..
Mon Apr 29 10:02:46 2019 - [info] Checking replication health on mysql1..
Mon Apr 29 10:02:46 2019 - [info]  ok.
Mon Apr 29 10:02:46 2019 - [info] Checking replication health on mysql3..
Mon Apr 29 10:02:46 2019 - [info]  ok.
Mon Apr 29 10:02:46 2019 - [error][/usr/local/share/perl5/MHA/ServerManager.pm, ln1218] 192.168.54.232 is not alive!
Mon Apr 29 10:02:46 2019 - [error][/usr/local/share/perl5/MHA/MasterRotate.pm, ln233] Failed to get new master!
Mon Apr 29 10:02:46 2019 - [error][/usr/local/share/perl5/MHA/ManagerUtil.pm, ln177] Got ERROR:  at /usr/local/bin/masterha_master_switch line 53.

解决方法，将ip更换为主机名，再次切换OK 。

end