初识mysql数据库-高可用

mysql主从复制及Gtid
www.mysql.com

实验环境：
master:172.25.40.1
slave:172.25.40.4
相同版本的mysql

安装数据库

tar xf mysql-5.7.11-1.el6.x86_64.rpm-bundle.tar

yum install -y mysql-community-client-5.7.11-1.el6.x86_64.rpm mysql-community-common-5.7.11-1.el6.x86_64.rpm mysql-community-libs-5.7.11-1.el6.x86_64.rpm mysql-community-libs-compat-5.7.11-1.el6.x86_64.rpm mysql-community-server-5.7.11-1.el6.x86_64.rpm

master:
修改mysql配置文件

vim /etc/my.cnf
    server-id=1                #必须不一致
    log-bin=mysql-bin         #开启二进制文件

启动数据库完成初始化

/etc/init.d/mysqld start
grep password /var/log/mysqld.log  #得到初始化密码
mysql_secure_installation   #初始化

这里注意初始化后密码有安全级别例如：Mysql+123

这里插入一个监控：

mv access.log access_$(date +%F -d -1day).log

进入数路库后创建一个供slave使用的用户及密码

grant replication slave on *.* to repl@'172.25.40.%' identified by 'Mysql+123';

此时查看master信息，并记下binlog文件地址和position

二进制文件如mysql-bin.000001查看使用命令：mysqlbinlog mysql-bin.000001

slave：
初始化后修改mysql配置文件

vim /etc/my.cnf
    server-id=4         #必须不一致

启动数据库并添加主库master信息：

change master to master_host='172.25.40.1', master_user='repl', master_password='Mysql+123', master_log_file='mysql-bin.000002', master_log_pos=1086;

启动slave
查看slave信息：

添加gtid模式
master slave配置文件加入：

gtid_mode=ON
enforce-gtid-consistency=true

stop slave
切换gtid模式

change master to master_host='172.25.40.1', master_user='repl', master_password='Mysql+123', master_auto_position=1;

start slave;

这里的信息进入mysql这个库才可以看到

并行复制
修改slave配置文件

slave:

master:

mysqlbinlog

半同步复制：
http://www.actionsky.com/mysql-57-semi-sync-intro/

首先主从数据库需加载PLUGIN插件

主：

mysql> INSTALL PLUGIN rpl_semi_sync_master SONAME 'semisync_master.so';

从：

mysql> INSTALL PLUGIN rpl_semi_sync_slave SONAME 'semisync_slave.so';

查看是否加载成功

mysql> show plugins;
或
mysql> SELECT PLUGIN_NAME, PLUGIN_STATUS FROM INFORMATION_SCHEMA.PLUGINS  WHERE PLUGIN_NAME LIKE '%semi%';

开启半同步：
主：

mysql> SET GLOBAL rpl_semi_sync_master_enabled = 1;

从：

mysql> SET GLOBAL rpl_semi_sync_slave_enabled = 1;

以上的启动方式是在命令行操作，也可写在配置文件中。

主：

plugin-load=rpl_semi_sync_master=semisync_master.so
rpl_semi_sync_master_enabled=1

从：

plugin-load=rpl_semi_sync_slave=semisync_slave.so
rpl_semi_sync_slave_enabled=1

重启从上的IO线程

mysql> STOP SLAVE IO_THREAD;

mysql> START SLAVE IO_THREAD;

查看半同步是否在运行

主：

mysql> show status like 'Rpl_semi_sync_master_status';

从：

mysql> show status like 'Rpl_semi_sync_slave_status';

做一个mysql代理 proxy
安装

tar zxf mysql-proxy-0.8.5-linux-el6-x86-64bit.tar.gz -C /usr/local

做链接：

ln -s /usr/local/mysql-proxy-0.8.5-linux-el6-x86-64bit/ /usr/local/mysql-proxy

写入环境变量：

vim ~/.bash_profile
    PATH=$PATH:$HOME/bin:/usr/local/mysql-proxy/bin
source ~/.bash_profile  #刷新

在mysql-proxy下创建目录conf写入其配置文件mysql-proxy.conf

vim /usr/local/mysql-proxy/conf/mysql-proxy.conf
    [mysql-proxy]
    daemon=true
    user=root
    pid-file=/usr/local/mysql-proxy/logs/mysql-proxy.pid
    log-file=/usr/local/mysql-proxy/logs/mysql-proxy.log
    log-level=info
    keepalive=true
    proxy-address=172.25.40.2:3306
    proxy-read-only-backend-addresses=172.25.40.4:3306
    proxy-backend-addresses=172.25.40.1:3306
    proxy-lua-script=/usr/local/mysql-proxy/share/doc/mysql-proxy/rw-splitting.lua
    plugins=proxy

修改脚本配置

vim /usr/local/mysql-proxy/share/doc/mysql-proxy/rw-splitting.lua

执行：

mysql-proxy --defaults-file=/usr/local/mysql-proxy/conf/mysql-proxy.conf 

此时可看到3306端口开启mysql-proxy进程存在

主从数据库 proxy分别安装lsof 可检测端口信息

主数据库：
添加一个代理使用的proxy用户给予一定的权限;

grant select, insert, update    on westos.* to proxy@'172.25.40.%' identified by 'Mysql+123';
flush privileges;      #刷新

物理机使用proxy登陆主从代理数据库mysql实现：

mysql -uproxy -p -h 172.25.40.1
mysql -uproxy -p -h 172.25.40.4
mysql -uproxy -p -h 172.25.40.2

搭配lsof软件可查看数据库登入情况

若可用proxy用户进行select，insert，update操作即实现成功

mycat
www.mycat.io

做一个线程的主从：

首先拷贝主数据库的信息：

mysqldump -p westos > test.sql

这里要注意下拷贝后的数据库：

生产环境需删除此行否则会造成数据的不同步

将此文件发送给从库slave2
slave2安装好数据库之后

mysqladmin -p create westos  #创建westos库
mysql -p westos < test.sql    #导入拷贝信息

重要的来了，这里slave1既做master的从数据库，又做slave2的主数据库

所以slave1配置文件写入：

log-bin=mysql-bin   #开启二进制文件
log_slave_updates=1  #重要

此时slave1上添加slave2使用的用户

grant replication slave on *.* to repl@'172.25.40.%' identified by 'Mysql+123';

slave2:

change master to master_host='172.25.40.4', master_user='repl', master_password='Mysql+123', master_auto_position=1;
start slave;

mysqldump -p mysql –tables user > user.sql 这个是拷贝表

MHA-数据库的高可用
https://www.cnblogs.com/gomysql/p/3675429.html

安装高可用管理软件
server2安装高可用管理和节点软件

yum install -y mha4mysql-manager-0.56-0.el6.noarch.rpm perl-* mha4mysql-node-0.56-0.el6.noarch.rpm

复制组master slave1 slave2 只安装node软件

yum install -y mha4mysql-node-0.56-0.el6.noarch.rpm

将之前的线程复制组改为一主多从
slave2：

change master to master_host='172.25.40.1', master_user='repl', master_password='Mysql+123', master_auto_position=1;
show slave status\G;  #查看slave状态

注意：这里做一个master slave1 slave2 的半同步 gtid的互相实现（安装plugin master 端和slave端 ），并统统开启

编写高可用的配置文件：

mkdir /etc/masterha/
vim app1.conf
[server default]
manager_workdir=/etc/masterha/app1.log
manager_log=/etc/masterha/manager.log
master_binlog_dir=/var/lib//mysql
#master_ip_failover_script= /usr/local/bin/master_ip_failover
#master_ip_online_change_script= /usr/local/bin/master_ip_online_change
password=Mysql+123
user=root
ping_interval=1
remote_workdir=/tmp
repl_password=Mysql+123
repl_user=repl
#report_script=/usr/local/send_report
#secondary_check_script= /usr/local/bin/masterha_secondary_check -s server03 -s server02            
#shutdown_script=""      
ssh_user=root

[server1]
hostname=172.25.40.1
port=3306

[server2]
hostname=172.25.40.4
port=3306
candidate_master=1
check_repl_delay=0

[server3]
hostname=172.25.40.3
port=3306

参照：

[server default]
manager_workdir=/var/log/masterha/app1.log              //设置manager的工作目录
manager_log=/var/log/masterha/app1/manager.log          //设置manager的日志
master_binlog_dir=/data/mysql                         //设置master 保存binlog的位置，以便MHA可以找到master的日志，我这里的也就是mysql的数据目录
master_ip_failover_script= /usr/local/bin/master_ip_failover    //设置自动failover时候的切换脚本
master_ip_online_change_script= /usr/local/bin/master_ip_online_change  //设置手动切换时候的切换脚本
password=123456         //设置mysql中root用户的密码，这个密码是前文中创建监控用户的那个密码
user=root               设置监控用户root
ping_interval=1         //设置监控主库，发送ping包的时间间隔，默认是3秒，尝试三次没有回应的时候自动进行railover
remote_workdir=/tmp     //设置远端mysql在发生切换时binlog的保存位置
repl_password=123456    //设置复制用户的密码
repl_user=repl          //设置复制环境中的复制用户名
report_script=/usr/local/send_report    //设置发生切换后发送的报警的脚本
secondary_check_script= /usr/local/bin/masterha_secondary_check -s server03 -s server02            
shutdown_script=""      //设置故障发生后关闭故障主机脚本（该脚本的主要作用是关闭主机放在发生脑裂,这里没有使用）
ssh_user=root           //设置ssh的登录用户名

[server1]
hostname=192.168.0.50
port=3306

[server2]
hostname=192.168.0.60
port=3306
candidate_master=1   //设置为候选master，如果设置该参数以后，发生主从切换以后将会将此从库提升为主库，即使这个主库不是集群中事件最新的slave
check_repl_delay=0   //默认情况下如果一个slave落后master 100M的relay logs的话，MHA将不会选择该slave作为一个新的master，因为对于这个slave的恢复需要花费很长时间，通过设置check_repl_delay=0,MHA触发切换在选择一个新的master的时候将会忽略复制延时，这个参数对于设置了candidate_master=1的主机非常有用，因为这个候选主在切换的过程中一定是新的master

[server3]
hostname=192.168.0.70
port=3306

检查SSH配置

masterha_check_ssh --conf=/etc/masterha/app1.conf

此时出现错误：

显示ssh连接出错
解决办法：
manger：

ssh-keygen   #添加密钥
cp id_rsa.pub authorized_keys  #做个备份

将密钥发送给master slave1 slave2

scp -r .ssh/ server1:
scp -r .ssh/ server4:
scp -r .ssh/ server3:

此时解决ssh问题：
再次，

masterha_check_ssh --conf=/etc/masterha/app1.conf

此时修改下mysql配置文件：如下分别为master slave1 slave2 配置文件

注意：核对slave2是否同步如用户

.检查整个复制环境状况

masterha_check_repl --conf=/etc/masterha/app1.conf

出现错误

解决办法：
master上授权root用户，并同步到slave

grant all on *.* to root@'172.25.40.%' identified by 'Mysql+123';

再次，

masterha_check_repl --conf=/etc/masterha/app1.conf

注：use mysql select * from user; 可查看mysql用户

开启MHA

masterha_manager --conf=/etc/masterha/app1.conf & (挂到后台)

检查MHA Manager的状态

masterha_check_status --conf=/etc/masterha/app1.conf

测试：
干掉master

kill -9 2978
kill -9 3272

删除：

此时查看状态：显示master迁移成功

查看slave2 的master库变为原slave1
此时恢复原master需要：

change master to master_host='172.25.40.4', master_user='repl', master_password='Mysql+123', master_auto_position=1;
start slave;

注：
此处有手动迁移master：

masterha_master_switch --conf=/etc/masterha/app1.conf --master_state=alive --new_master_host=172.25.40.1 --new_master_port=3306 --orig_master_is_new_slave  #迁移没有挂掉的数据库

masterha_master_switch --conf=/etc/masterha/app1.conf --master_state=dead --dead_master_host=172.25.40.4 --dead_master_port=3306 --new_master_host=172.25.40.1 --new_master_port=3306
 #手动迁移挂掉的数据库

至此，数据库实现基本高可用

使用vip进行高可用：

首先高可用配置文件：

vim /etc/masterma/app1.conf(添加)
    master_ip_failover_script= /usr/local/bin/master_ip_failover
    master_ip_online_change_script= /usr/local/bin/master_ip_online_change

删除app1.failover.complete 文件

将vip飘逸的脚本文件添加到指定路径/usr/local/bin
master_ip_failover

vim master_ip_failover
#!/usr/bin/env perl
use strict;
use warnings FATAL => 'all';
use Getopt::Long;

my (
    $command,          $ssh_user,        $orig_master_host, $orig_master_ip,
    $orig_master_port, $new_master_host, $new_master_ip,    $new_master_port
);

my $vip = '172.25.40.100/24';
my $ssh_start_vip = "/sbin/ip addr add $vip dev eth1";
my $ssh_stop_vip = "/sbin/ip addr del $vip dev eth1";

GetOptions(
    'command=s'          => \$command,
    'ssh_user=s'         => \$ssh_user,
    'orig_master_host=s' => \$orig_master_host,
    'orig_master_ip=s'   => \$orig_master_ip,
    'orig_master_port=i' => \$orig_master_port,
    'new_master_host=s'  => \$new_master_host,
    'new_master_ip=s'    => \$new_master_ip,
);

exit &main();

sub main {

    print "\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";

    if ( $command eq "stop" || $command eq "stopssh" ) {

        my $exit_code = 1;
        eval {
            print "Disabling the VIP on old master: $orig_master_host \n";
            &stop_vip();
            $exit_code = 0;
        };
        if ($@) {
            warn "Got Error: $@\n";
            exit $exit_code;
        }
exit $exit_code;
    }
    elsif ( $command eq "start" ) {

        my $exit_code = 10;
        eval {
            print "Enabling the VIP - $vip on the new master - $new_master_host \n";
            &start_vip();
            $exit_code = 0;
        };
        if ($@) {
            warn $@;
            exit $exit_code;
        }
        exit $exit_code;
    }
    elsif ( $command eq "status" ) {
        print "Checking the Status of the script.. OK \n";
        exit 0;
    }
}

sub start_vip() {
    `ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
sub stop_vip() {
     return 0  unless  ($ssh_user);
    `ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}

sub usage {
    print
    "Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
}                

master_ip_online_change

vim master_ip_online_change
#!/usr/bin/env perl
use strict;
use warnings FATAL =>'all';

use Getopt::Long;

my $vip = '172.25.40.100/24';
my $ssh_start_vip = "/sbin/ip addr add $vip dev eth1";
my $ssh_stop_vip = "/sbin/ip addr del $vip dev eth1";
my $exit_code = 0;

my (
  $command,              $orig_master_is_new_slave, $orig_master_host,
  $orig_master_ip,       $orig_master_port,         $orig_master_user,
  $orig_master_password, $orig_master_ssh_user,     $new_master_host,
  $new_master_ip,        $new_master_port,          $new_master_user,
  $new_master_password,  $new_master_ssh_user,
);
GetOptions(
  'command=s'                => \$command,
  'orig_master_is_new_slave' => \$orig_master_is_new_slave,
  'orig_master_host=s'       => \$orig_master_host,
  'orig_master_ip=s'         => \$orig_master_ip,
  'orig_master_port=i'       => \$orig_master_port,
  'orig_master_user=s'       => \$orig_master_user,
  'orig_master_password=s'   => \$orig_master_password,
  'orig_master_ssh_user=s'   => \$orig_master_ssh_user,
  'new_master_host=s'        => \$new_master_host,
  'new_master_ip=s'          => \$new_master_ip,
  'new_master_port=i'        => \$new_master_port,
  'new_master_user=s'        => \$new_master_user,
  'new_master_password=s'    => \$new_master_password,
  'new_master_ssh_user=s'    => \$new_master_ssh_user,
);


exit &main();

sub main {

#print "\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";  

if ( $command eq "stop" || $command eq "stopssh" ) {

# $orig_master_host, $orig_master_ip, $orig_master_port are passed.  
        # If you manage master ip address at global catalog database,  
        # invalidate orig_master_ip here.  
        my $exit_code = 1;
        eval {
            print "\n\n\n***************************************************************\n";
            print "Disabling the VIP - $vip on old master: $orig_master_host\n"; 
            print "***************************************************************\n\n\n\n";
&stop_vip();
            $exit_code = 0;
        };
        if ($@) {
            warn "Got Error: $@\n";
            exit $exit_code;
        }
        exit $exit_code;
}
elsif ( $command eq "start" ) {
# all arguments are passed.  
        # If you manage master ip address at global catalog database,  
        # activate new_master_ip here.  
        # You can also grant write access (create user, set read_only=0, etc) here.  
my $exit_code = 10;
        eval {
            print "\n\n\n***************************************************************\n";
            print "Enabling the VIP - $vip on new master: $new_master_host \n";  
            print "***************************************************************\n\n\n\n";
&start_vip();
            $exit_code = 0;
        };
        if ($@) {
            warn $@;
            exit $exit_code;
        }
        exit $exit_code;
}
elsif ( $command eq "status" ) {
        print "Checking the Status of the script.. OK \n";
        `ssh $orig_master_ssh_user\@$orig_master_host \" $ssh_start_vip \"`;
        exit 0;
}
else {
&usage();
        exit 1;
}
}

# A simple system call that enable the VIP on the new master  
sub start_vip() {
`ssh $new_master_ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
# A simple system call that disable the VIP on the old_master  
sub stop_vip() {
`ssh $orig_master_ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}

sub usage {
print
"Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
}

给予脚本x权限

启动mha

masterha_manager --conf=/etc/masterha/app1.conf &

手动给master添加vip

ip addr add 172.25.40.100/24 dev eth1

测试可否通过vip进行mysql的连接
物理机：

mysql -p -h 172.25.40.100

此时测试，干掉master实现备机上位master，并将vip实现飘逸

使用这条命令查看高可用及脚本状态

masterha_check_repl --conf=/etc/masterha/app1.conf

并同时检测vip连接数据库是否可用

高可用实现的同时添加mail服务：(使用163)

配置发送脚本到启动路径/usr/local/bin/

#!/usr/bin/perl

#  Copyright (C) 2011 DeNA Co.,Ltd.
#
#  This program is free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation; either version 2 of the License, or
#  (at your option) any later version.
#
#  This program is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#  GNU General Public License for more details.
#
#  You should have received a copy of the GNU General Public License
#   along with this program; if not, write to the Free Software
#  Foundation, Inc.,
#  51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA

## Note: This is a sample script and is not complete. Modify the script based on your environment.

use strict;
use warnings FATAL => 'all';
use Mail::Sender;
use Getopt::Long;

#new_master_host and new_slave_hosts are set only when recovering master succeeded
my ( $dead_master_host, $new_master_host, $new_slave_hosts, $subject, $body );
my $smtp='smtp.163.com';
my $mail_from='[email protected]';
my $mail_user='[email protected]';
my $mail_pass='password';
my $mail_to='[email protected]';
GetOptions(
  'orig_master_host=s' => \$dead_master_host,
  'new_master_host=s'  => \$new_master_host,
  'new_slave_hosts=s'  => \$new_slave_hosts,
  'subject=s'          => \$subject,
  'body=s'             => \$body,
);

mailToContacts($smtp,$mail_from,$mail_user,$mail_pass,$mail_to,$subject,$body);

sub mailToContacts {
    my ( $smtp, $mail_from, $user, $passwd, $mail_to, $subject, $msg ) = @_;
    open my $DEBUG, "> /tmp/monitormail.log"
        or die "Can't open the debug      file:$!\n";
    my $sender = new Mail::Sender {
        ctype       => 'text/plain; charset=utf-8',
        encoding    => 'utf-8',
        smtp        => $smtp,
        from        => $mail_from,
        auth        => 'LOGIN',
        TLS_allowed => '0',
        authid      => $user,
        authpwd     => $passwd,
        to          => $mail_to,
        subject     => $subject,
        debug       => $DEBUG
    };

    $sender->MailMsg(
        {   msg   => $msg,
            debug => $DEBUG
        }
    ) or print $Mail::Sender::Error;
    return 1;
}



# Do whatever you want here

exit 0;

给予执行权限

chmod +x send_report

修改高可用配置文件

vim /etc/masterha/app1.conf
report_script=/usr/local/bin/send_report

删除app1.failover.complete文件

rm -fr app1.failover.complete

开启mha：

masterha_manager --conf=/etc/masterha/app1.conf &

使虚拟机可以上网：

route add default gw 172.25.40.250

vim /etc/resolv.conf

测试：ping baidu.com

安装mailx

yum install -y mailx
mail

测试：
干掉master，备机上位
这里会发送mail 在/tmp/下

数据库缓存—基于数据库本身