Mariadb galera V10.1高可用配置

在openstack Mitaka中使用packstack安装时，默认会安装cloud repo中的mariadb-server，该版本是10.1的，高可用配置与5.5稍有不同。

安装包

[root@test3 my.cnf.d]# rpm -qa|grep mariadb 
mariadb-errmsg-10.1.12-4.el7.x86_64 
mariadb-galera-common-5.5.40-3.el7.x86_64 
mariadb-server-galera-10.1.12-4.el7.x86_64 
mariadb-libs-5.5.50-1.el7_2.x86_64 
mariadb-common-10.1.12-4.el7.x86_64 
mariadb-10.1.12-4.el7.x86_64 
mariadb-server-10.1.12-4.el7.x86_64 

高可用配置

1. 安装以上组件

2. 配置/etc/hosts

   [root@test3 my.cnf.d]# cat /etc/hosts
   127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
   ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
   
   192.168.91.130 test1
   192.168.91.131 test2
   192.168.91.132 test3

3. 配置mariadb(3台机器配置一样)

   [root@test3 my.cnf.d]# cat /etc/my.cnf.d/mariadb-server.cnf |grep -v ^#|grep -v ^$
   [server]
   [mysqld]
   datadir=/var/lib/mysql
   socket=/var/lib/mysql/mysql.sock
   log-error=/var/log/mariadb/mariadb.log
   pid-file=/var/run/mariadb/mariadb.pid
   [galera]
   wsrep_on=ON
   wsrep_provider=/usr/lib64/galera/libgalera_smm.so
   wsrep_cluster_address="gcomm://test1,test2,test3"
   default_storage_engine=InnoDB
   [embedded]
   [mariadb]
   [mariadb-10.1]

   [root@test3 my.cnf.d]# cat /etc/my.cnf.d/galera.cnf |grep -v ^#|grep -v ^$
   [mysqld]
   binlog_format=ROW
   default-storage-engine=innodb
   innodb_autoinc_lock_mode=2
   bind-address=0.0.0.0
   wsrep_provider=/usr/lib64/galera/libgalera_smm.so
   wsrep_cluster_name="my_wsrep_cluster"
   wsrep_cluster_address="gcomm://test1,test2,test3"
   wsrep_node_name=test3
   wsrep_node_address=test3
   wsrep_slave_threads=1
   wsrep_certify_nonPK=1
   wsrep_max_ws_rows=131072
   wsrep_max_ws_size=1073741824
   wsrep_debug=0
   wsrep_convert_LOCK_to_trx=0
   wsrep_retry_autocommit=1
   wsrep_auto_increment_control=1
   wsrep_drupal_282555_workaround=0
   wsrep_causal_reads=0
   wsrep_notify_cmd=
   wsrep_sst_method=rsync
   wsrep_sst_auth=root:

4. 启动主节点

   [root@ha1 ~]# /usr/libexec/mysqld --wsrep-new-cluster --user=root

   如果mariadb的log中出现关键字，则证明启动成功:

   [root@ha1 ~]# vi /var/log/mariadb/mariadb.log
   2016-09-04  8:34:31 140329951509248 [Note] WSREP: Synchronized with group, ready for connections

5. 启动其他节点

   [root@test2 mysql]# systemctl restart mariadb
   [root@test3 mysql]# systemctl restart mariadb

6. 确认服务
   在主节点上查看数据库关于wsrep的状态，确认所有节点都已经连接上：

   MariaDB [(none)]> show status like 'wsrep_cluster_size';
   +--------------------+-------+
   | Variable_name      | Value |
   +--------------------+-------+
   | wsrep_cluster_size | 3     |
   +--------------------+-------+

   在某节点数据库新建数据库，检查其他节点是否成功同步：

   [root@ha1 ~]# mysql -uroot
   MariaDB [(none)]> show databases;
   +--------------------+
   | Database           |
   +--------------------+
   | information_schema |
   | mysql              |
   | performance_schema |
   | test               |
   | test1              |
   +--------------------+
   5 rows in set (0.00 sec)
   
   MariaDB [(none)]> create database test2;
   Query OK, 1 row affected (0.01 sec)

   [root@test2 mysql]# mysql -uroot
   MariaDB [(none)]> show databases;
   +--------------------+
   | Database           |
   +--------------------+
   | information_schema |
   | mysql              |
   | performance_schema |
   | test               |
   | test1              |
   | test2              |
   +--------------------+
   6 rows in set (0.00 sec)

Trouble Shooting

1. 确认正确配置了firewalld或者将其关闭，确认正确配置SELinux或者将其关闭。

2. 确认开启了3306端口和4567端口（如果没有在配置文件中特别指定端口的话）。

   [root@controller01 ~]# netstat -apn|grep mysqld|more
   tcp        0      0 0.0.0.0:3306            0.0.0.0:*               LISTEN      81711/mysqld        
   tcp        0      0 0.0.0.0:4567            0.0.0.0:*               LISTEN      81711/mysqld  

3. 如果其他节点中以前启动过mariadb服务，则这里可能会报错:

   [root@test2 mysql]# systemctl status mariadb
   ● mariadb.service - MariaDB 10.1 database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; disabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Mon 2016-09-05 04:07:09 CST; 13min ago
   Process: 2426 ExecStopPost=/usr/libexec/mysql-wait-stop (code=exited, status=0/SUCCESS)
   Process: 2314 ExecStart=/usr/libexec/mysqld --basedir=/usr $MYSQLD_OPTS $_WSREP_NEW_CLUSTER (code=exited, status=1/FAILURE)
   Process: 2280 ExecStartPre=/usr/libexec/mysql-prepare-db-dir %n (code=exited, status=0/SUCCESS)
   Process: 2257 ExecStartPre=/usr/libexec/mysql-check-socket (code=exited, status=0/SUCCESS)
   Main PID: 2314 (code=exited, status=1/FAILURE)
   Status: "MariaDB server is down"   
   
   Sep 05 04:06:57 test2 rsyncd[2383]: rsync to rsync_sst/./test1 from test1 (192.168.91.130)
   Sep 05 04:06:57 test2 rsyncd[2383]: receiving file list
   Sep 05 04:06:57 test2 rsyncd[2385]: connect from test1 (192.168.91.130)
   Sep 05 04:06:57 test2 rsyncd[2385]: rsync to rsync_sst/ from test1 (192.168.91.130)
   Sep 05 04:06:57 test2 rsyncd[2385]: receiving file list
   Sep 05 04:07:09 test2 systemd[1]: mariadb.service: main process exited, code=exited, status=1/FAILURE
   Sep 05 04:07:09 test2 systemd[1]: Failed to start MariaDB 10.1 database server.
   Sep 05 04:07:09 test2 systemd[1]: Unit mariadb.service entered failed state.
   Sep 05 04:07:09 test2 systemd[1]: mariadb.service failed.
   Sep 05 04:09:55 test2 systemd[1]: Stopped MariaDB 10.1 database server.

   只需要删除数据库的结构目录，重启即可：

   [root@test2 mysql]# rm -rf /var/lib/mysql/*
   [root@test2 mysql]# systemctl restart mariadb

4. 确认所有节点中，/etc/my.cnf.d/目录下有无其他文件，如server.cnf等，这些文件是其他配置程序如puppet等生成的，在mariadb服务启动时会覆盖你的配置。
5. 如果觉得一切配置正确但是无法正常启动，重启机器可能会有帮助。在实践中，某些异常问题确实通过重启解决了。