实现MySQL高可用之MHA过程错误记录集

MHA 集群是一套优秀的作为 MySQL 高可用性环境下故障切换和主从提升的高可用软件。目前在 MySQL 高可用方面是一个相对成熟的解决方案 ，在 MySQL 故障切换过程中，MHA 能做到在 0~30 秒之内自动完成数据库的故障切换操作并且在进行故障切换的过程中，MHA能在最大程度上保证数据的一致性，以达到真正意义上的高可用。

笔者在对MySQL集群搭建实现高可用解决方案时使用了MHA，在搭建过程中遇到一系列问题，通过本帖记录类遇到的问题以及解决办法，希望对读者有所帮助。

---

- 错误信息「Can’t locate Log/Dispatch.pm in @INC」

[root@mham mha4mysql-manager-0.56]# masterha_check_ssh --conf=/etc/mha_manager/app1.cnf
Can't locate Log/Dispatch.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /usr/local/share/perl5/MHA/SSHCheck.pm line 29.
BEGIN failed--compilation aborted at /usr/local/share/perl5/MHA/SSHCheck.pm line 29.
Compilation failed in require at /usr/local/bin/masterha_check_ssh line 25.
BEGIN failed--compilation aborted at /usr/local/bin/masterha_check_ssh line 25.
[root@mham mha4mysql-manager-0.56]# 

该错误提示是因为MHAManager主机的软件未安装完整，将所需软件安装:

[root@mham mha-soft-student]# ls
mha4mysql-manager-0.56                          perl-Mail-Sender-0.8.23-1.el7.noarch.rpm
mha4mysql-manager-0.56.tar.gz                   perl-Mail-Sendmail-0.79-21.el7.art.noarch.rpm
mha4mysql-node-0.56-0.el6.noarch.rpm            perl-MIME-Lite-3.030-1.el7.noarch.rpm
perl-Config-Tiny-2.14-7.el7.noarch.rpm          perl-MIME-Types-1.38-2.el7.noarch.rpm
perl-Email-Date-Format-1.002-15.el7.noarch.rpm  perl-Parallel-ForkManager-1.18-2.el7.noarch.rpm
perl-Log-Dispatch-2.41-1.el7.1.noarch.rpm
[root@mham mha-soft-student]# yum -y  install  perl-*.rpm
[root@mham mha-soft-student]# yum  -y  install   perl-DBD-mysql
[root@mham mha-soft-student]# rpm  -ivh  mha4mysql-node-0.56-0.el6.noarch.rpm

问题解决。

---

- 错误信息 「SSH Configuration Check Failed!」

Fri Jul 13 14:55:00 2018 - [error][/usr/local/share/perl5/MHA/SSHCheck.pm, ln111] SSH connection from [email protected](192.168.1.14:22) to [email protected](192.168.1.15:22) failed!
SSH Configuration Check Failed!
 at /usr/local/bin/masterha_check_ssh line 44.

集群中ssh免密登陆未设置好，仔细检查并重新设置好集群中的全部机器的ssh免密登录即可。

---

- 错误信息 「None of slaves can be master. Check failover configuration file or log-bin settings in my.cnf」

Fri Jul 13 15:00:14 2018 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln361] None of slaves can be master. Check failover configuration file or log-bin settings in my.cnf
Fri Jul 13 15:00:14 2018 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln424] Error happened on checking configurations.  at /usr/local/bin/masterha_check_repl line 48.
Fri Jul 13 15:00:14 2018 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln523] Error happened on monitoring servers.
Fri Jul 13 15:00:14 2018 - [info] Got exit code 1 (Not master dead).

MySQL Replication Health is NOT OK!

这个错误原因在与集群中的slave节点的数据库配置文件/ect/my.cnf没有设置log-bin参数，解决办法就是将所有slave节点的数据库配置文件加上log-bin=XXX参数，重启数据库服务即可。

---

- 错误信息 「mysql command failed with rc 1:0!」

Fri Jul 13 15:06:03 2018 - [info]   Connecting to [email protected](192.168.1.14:22).. 
  Checking slave recovery environment settings..
    Opening /var/lib/mysql/relay-log.info ... ok.
    Relay log found at /var/lib/mysql, up to mariadb-relay-bin.000006
    Temporary relay log file is /var/lib/mysql/mariadb-relay-bin.000006
    Testing mysql connection and privileges..ERROR 1045 (28000): Access denied for user 'root'@'node3' (using password: YES)
mysql command failed with rc 1:0!
 at /usr/bin/apply_diff_relay_logs line 375.
    main::check() called at /usr/bin/apply_diff_relay_logs line 497
    eval {...} called at /usr/bin/apply_diff_relay_logs line 475
    main::main() called at /usr/bin/apply_diff_relay_logs line 120
Fri Jul 13 15:06:04 2018 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln205] Slaves settings check failed!
Fri Jul 13 15:06:04 2018 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln413] Slave configuration failed.
Fri Jul 13 15:06:04 2018 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln424] Error happened on checking configurations.  at /usr/local/bin/masterha_check_repl line 48.
Fri Jul 13 15:06:04 2018 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln523] Error happened on monitoring servers.
Fri Jul 13 15:06:04 2018 - [info] Got exit code 1 (Not master dead).

MySQL Replication Health is NOT OK!

这是来自集群中单台节点配置检查的错误，从错误信息可以看到Access denied for user 'root'@'node3' (using password: YES)，问题就是说连接不上该节点的数据库。
解决过程：
1、检查note3的授权，从主数据库机连接node3时无问题

[root@node1 mha-soft-student]# mysql -hnode3 -uroot -p123456
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 11
Server version: 5.5.56-MariaDB MariaDB Server
……

2、在node3节点连接本机发现失败

[root@node3 ~]# mysql -h node3 -uroot -p123456
ERROR 1045 (28000): Access denied for user 'root'@'node3' (using password: YES)

3、使用root登陆本机数据库

[root@node3 ~]# mysql -uroot -p123456
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 14
Server version: 5.5.56-MariaDB MariaDB Server

Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

4、查询root用户表信息

MariaDB [(none)]> select user,host,password from mysql.user;
+----------+-----------+-------------------------------------------+
| user     | host      | password                                  |
+----------+-----------+-------------------------------------------+
| root     | localhost | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 |
| root     | node3     |                                           |
| root     | 127.0.0.1 |                                           |
| root     | ::1       |                                           |
|          | localhost |                                           |
|          | node3     |                                           |
| repluser | %         | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 |
| root     | %         | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 |
+----------+-----------+-------------------------------------------+
8 rows in set (0.00 sec)

查询结果的host字段中node3出现问题，没有password

5、删除node3、密码为空的记录

MariaDB [(none)]> delete from mysql.user where host='node3';
Query OK, 2 rows affected (0.00 sec)

MariaDB [(none)]> delete from mysql.user where password='';
Query OK, 3 rows affected (0.00 sec)

MariaDB [(none)]> select user,host,password from mysql.user;
+----------+-----------+-------------------------------------------+
| user     | host      | password                                  |
+----------+-----------+-------------------------------------------+
| root     | localhost | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 |
| repluser | %         | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 |
| root     | %         | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 |
+----------+-----------+-------------------------------------------+

3 rows in set (0.00 sec)

6、重新给root授权

MariaDB [(none)]> grant all privileges on *.* to root@'%' identified by '123456' with grant option;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> exit
Bye
[root@node3 ~]# mysql -h node3 -uroot -p123456
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 15
Server version: 5.5.56-MariaDB MariaDB Server

Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

问题解决。