keepalived+nginx+tomcat+redis+mysql主主架构部署

架构：
两台服务器，两个实体IP，一个虚拟IP。
如下图所示，服务器正常运行时，KEEPALIVED主保证的Nginx的HA，Nginx的主负责两台的Tomcat服务器负载均衡，Redis的主负责Tomcat1和2的会话共享，MySql主主同步保证了数据的安全及稳定性。
当主节点的一个或者多个服务无法正常运行，甚至节点宕机时，备节点能立即接续主节点来执行业务从而实现业务的不中断或短暂中断。
优势：适合硬件有限的小规模集群，且能保证服务器的稳定，高可用运行可持续发展来看，集群规模可升级-麻雀虽小五脏俱全。
劣势：对业务的处理能力有限，毕竟服务器仅两台。

一，环境：

Centos：7.4
Java：jdk1.8.0_192
Tomcat：apache-tomcat-9.0.13
Nginx：nginx-1.15.8
Keepalived ：Keepalivedv1.3.5
Redis：redis-5.0.3
Mysql：mysql-5.7.25 编译安装
Cmak：cmake-3.13.4
Boost：boost_1_59_0

1，需要Jar：

commons-pool2-2.4.2.jar
jedis-2.9.0.jar
tomcat-redis-session-manager.jar

2，依赖环境：

yum -y install gcc pcre-devel zlib-devel openssl-devel make gcc gcc-c++ ncurses-devel bison openssl-devel

3，服务器配置：

节点名称	IP	端口开放	VIP
Master	10.200.36.166	80，1777，6000，3306	10.200.36.168
Backuo	10.200.36.167	80，1777，6000，3306	10.200.36.168

二，部署前准备：
1，上传数据包（双节点都需要实施）
下载所需的包，网址：https://download.csdn.net/download/clownwl/11006534
因为CSDN上传大小限制，只能上传200M文件所以上个链接缺少了JDK，JDK：https://download.csdn.net/download/clownwl/11006810 也可去官网下载。
下载好以后，通过winSCP、Xftp等工具将包上传到/opt/soft目录下（opt /soft）
并赋予该目录下所有文件/文件夹可执行权限（chmod -R 755 *）

2，测试环境，先关闭防火墙和SELinux
关闭SELinux

#setenforce 0
#sed -i ‘s/SELINUX=enforcing/SELINUX=disabled/g’ /etc/selinux/config --CentOS7.4默认用的是firewalld防火墙
#systemctl stop firewalld.service　　--关闭firewalld

3，配置163yum源或者阿里yum源（双节点都需要安装）
去http://mirrors.163.com/.help/CentOS7-Base-163.repo 下载centos7的yum源。
备份原有yum源：

#mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup

更新生成新的缓存：

#yum clean all
#yum makecache

提前安装依赖包：

#yum -y install gcc pcre-devel zlib-devel openssl-devel make gcc gcc-c++ ncurses-devel bison openssl-devel

三，安装JDK（双节点都需要安装）
在安装JDK前，最好检测下是否安装了openjdk，若有，最好卸载。（rpm -qa | grep jdk）
1，安装JDK：

#cd /opt/soft
#tar -zxvf jdk-8u192-linux-x64.tar.gz -C /opt/
#vim /etc/profile.d/jdk.sh

输入以下内容：

export JAVA_HOME=/opt/jdk1.8.0_192
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH

检测：

#java -version

java version "1.8.0_192"
Java(TM) SE Runtime Environment (build 1.8.0_192-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.192-b12, mixed mode)

返回值若如上所示，说明安装成功

四，安装Nginx（双节点都需要安装）
1，安装nginx

#cd /opt/soft
#tar -xvf nginx-1.15.8.tar.gz
#cd nginx-1.15.8
#./configure
#make
#make install
#cd /usr/local/nginx/sbin
#./nginx -t 检查nginx是否安装成功 若出现以下内容则表示安装成功

nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok  
nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful 

2，配置nginx以实现负载均衡分发

#cd /usr/local/nginx/conf
#mv nginx.conf nginx.conf.bak --备份配置文件
#vim nginx.conf

主从节点配置文件一样：

user  root;
worker_processes  2;
error_log  /usr/local/nginx/logs/error.log  info;

#指定pid存放文件
pid        logs/nginx.pid;

events
{
    use epoll;            #epoll是多路复用IO(I/OMultiplexing)中的一种方式,但是仅用于linux2.6以上>内核,可以大大提高nginx的性能

    worker_connections  1024;    #单个后台worker process进程的最大并发链接数
}
http
{
upstream tomcat_pool
    {
     #server tomcat地址:端口号 weight表示权值，权值越大，被分配的几率越大;
     server 10.200.36.166:8080 weight=4 max_fails=2 fail_timeout=30s;
     server 10.200.36.167:8080 weight=4 max_fails=2 fail_timeout=30s;
    }
server {
        listen       80;
        #charset utf-8
        server_name 10.200.36.168;
location / {


        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header REMOTE-HOST $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://tomcat_pool;
        client_max_body_size 1000m;
        proxy_cookie_path /scientific  /;
        proxy_set_header Cookie $http_cookie;
 }

        error_page 404 /404.html;
            location = /40x.html {
        }

        error_page 500 502 503 504 /50x.html;
            location = /50x.html {
        }
    }
}

五，安装keepalived（双节点都需要安装）
1，安装Keepalived（两种安装方式）

第一种（推荐）：
#yum -y install keepalived
第二种：
#cd /opt/soft
#tar -zxvf keepalived-1.3.5.tar.gz
#cd keepalived-1.3.5
#./configure
#make
#make install

2，Keepalived主从节点配置
声明：以下两个配置文件中interface ens192这个参数，不同的节点用到的网络接口名会不同，具体请使用ifconfig查看当前IP对应的接口名。

#cd /etc/keepalived/
#vim /keepalived.conf

主节点配置文件：

! Configuration File for keepalived

global_defs {
   notification_email {
    [email protected]
   }
   notification_email_from   [email protected]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_script chk_nginx {
    #该脚本检测ngnix的运行状态，并在nginx进程不存在时尝
    #试重新启动ngnix，如果启动失败则停止keepalived，准备让其它机器接管。
    script "/opt/ha/check_nginx.sh"
    #每1s检测一次
    interval 1
    # 检测失败（脚本返回非0则优先级2）
    weight 2
}

vrrp_script chk_tomcat {
    #该脚本检测ngnix的运行状态，并在nginx进程不存在时尝
    #试重新启动ngnix，如果启动失败则停止keepalived，准备让其它机器接管。
    script "/opt/ha/check_tomcat.sh"
    #每1s检测一次
    interval 1
    # 检测失败（脚本返回非0则优先级2）
    weight 2
}

#vrrp_script check_gateway {
#    script "/opt/ha/check_gateway.sh"
#    interval 5
#    fall     4
#    rise     4
#    weight 3
#}

vrrp_instance VI_1 {
    #指定keepalived的角色，MASTER表示此主机是主服务器，BACKUP表示此主机是备用服务器
    state MASTER
    # 指定HA监测的网络接口
    interface ens192
    # 虚拟路由标识，要求主从节点一致
    virtual_router_id 55
    # 优先级，在同一vrrp_instance下要求主节点的优先级高于从节点
    priority 100
    # 设定主从节点之间同步检查的时间间隔，单位是秒
    advert_int 1
    authentication {
        # 设置验证类型，主要有PASS和AH两种
        auth_type PASS
        # 设置验证密码，在同一个vrrp_instance下，MASTER与BACKUP必须使用相同的密码才能正常通信
        auth_pass 1111
    }
    virtual_ipaddress {
        # 设置虚拟IP地址，可以设置多个虚拟IP地址，每行一个
        10.200.36.168
    }
    track_script {
        # 引用VRRP脚本，即在 vrrp_script 部分指定的名字。
        # 定期运行它们来改变优先级，并最终引发主备切换。
        chk_nginx
        chk_tomcat
        #chk_gateway
    }
}

#virtual_server 10.200.36.168 3306 {
#    delay_loop 6
#    lb_algo wrr          #lvs调度算法rr|wrr|lc|wlc|lblc|sh|dh
#    lb_kind DR           #负载均衡转发规则NAT|DR|RUN
#    persistence_timeout 50
#    protocol TCP
#
#    real_server 10.200.36.166 3306 {
#      weight 1
#      notify_down /opt/ha/mysql.sh
#      TCP_CHECK {
#      connect_timeout 10
#      nb_get_retry 3
#      delay_before_retry 3
#      connect_port 3306
#        }
#    }
#}

备节点配置文件：

! Configuration File for keepalived

global_defs {
   notification_email {
     [email protected]
   }
   notification_email_from [email protected]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_script chk_nginx {
    #该脚本检测ngnix的运行状态，并在nginx进程不存在时尝
    #试重新启动ngnix，如果启动失败则停止keepalived，准备让其它机器接管。
    script "/opt/ha/check_nginx.sh"
    #每1s检测一次
    interval 1
    # 检测失败（脚本返回非0则优先级2）
    weight 2
}

vrrp_script chk_tomcat {
    #该脚本检测ngnix的运行状态，并在nginx进程不存在时尝
    #试重新启动ngnix，如果启动失败则停止keepalived，准备让其它机器接管。
    script "/opt/ha/check_tomcat.sh"
    #每1s检测一次
    interval 1
    # 检测失败（脚本返回非0则优先级2）
    weight 2
}

#vrrp_script check_gateway {
#    script "/opt/ha/check_gateway.sh "
#    interval 5
#    fall     4
#    rise     4
#    weight 3
#}

vrrp_instance VI_1 {
    #指定keepalived的角色，MASTER表示此主机是主服务器，BACKUP表示此主机是备用服务器
    state BACKUP
    # 指定HA监测的网络接口
    interface ens192
    # 虚拟路由标识，要求主从节点一致
    virtual_router_id 55
    # 优先级，在同一vrrp_instance下要求主节点的优先级高于从节点
    priority 90
    # 设定主从节点之间同步检查的时间间隔，单位是秒
    advert_int 1
    authentication {
        # 设置验证类型，主要有PASS和AH两种
        auth_type PASS
        # 设置验证密码，在同一个vrrp_instance下，MASTER与BACKUP必须使用相同的密码才能正常通信
        auth_pass 1111
    }
    virtual_ipaddress {
        # 设置虚拟IP地址，可以设置多个虚拟IP地址，每行一个
        10.200.36.168
    }
    track_script {
        # 引用VRRP脚本，即在 vrrp_script 部分指定的名字。
        #定期运行它们来改变优先级，并最终引发主备切换。
        chk_nginx
        chk_tomcat
    }
}

#virtual_server 10.200.36.168 3306 {
#      delay_loop 2                 ##每隔2秒检查真实服务器状态
#      lb_algo wrr                  ##LVS算法，如果仅做高可用可不设
#      lb_kind DR                   ##LVS模式，如果仅做高可用可不设
#      persistence_timeout 60
#      protocol TCP
#
#      real_server 10.200.36.167 3306{
#      weight 3
#      notify_down /opt/ha/mysql.sh     ##如果mysql服务宕掉，执行的脚本
#      echo '3' >  /etc/keepalived/t.log
#      TCP_CHECK {
#      connect_timeout 10
#      nb_get_retry 3
#      delay_before_retry 3
#      connect_port 3306
#      }
#      }
#}

3，创建检测脚本（双节点都需要创建，可使用scp传输）

#mkdir -p /opt/ha/
#touch /opt/ha/check_nginx.sh
#vi /opt/ha/check_nginx.sh

---

#!/bin/sh
echo "1" >> /var/tmp/keepalived.log
Count1=`netstat -antp |grep -v grep |grep nginx |wc -l`
if [ $Count1 -eq 0 ]; then
    #/usr/local/nginx/sbin/nginx
    #sleep 2
        echo "2" >> /var/tmp/keepalived.log
    Count2=`netstat -antp |grep -v grep |grep nginx |wc -l`
    if [ $Count2 -eq 0 ]; then
                echo "nginx pid not found" >> /etc/keepalived/keepalived.log
        killall keepalived
    else
                echo "jinbulai" >> /var/tmp/keepalived.log
        exit 0
    fi
else
        echo "4" >> /var/tmp/keepalived.log
    exit 0
fi

赋予脚本可执行权限：

#chmod 755 /opt/ha/check_nginx.sh

六，部署Tomcat（双节点都需要操作）
1，创建用户tomcat

#useradd -d /home/tomcat -m tomcat
#passwd tomcat

根据提示设置tomcat用户密码即可，我这里设置成tomcat@123

2，部署Tomcat

#cd /opt/soft/
#tar -zxvf apache-tomcat-9.0.13.tar.gz -C /opt/　　--将soft下的tomcat解压到opt目录下
#cd /opt/
#chmod -R 755 tomcat:tomcat apache-tomcat-9.0.13/*　　--赋予tomcat用户目录下文件/文件夹可执行权限
#chown -R tomcat:tomcat apache-tomcat-9.0.13/*　　--将tomcat用户目录下文件/文件夹的用户和组改为tomcat

3，修改tomcat的内存分配

#cd /opt/apache-tomcat-9.0.13/bin
#vi .catalina.sh　　--添加以下内容

JAVA_OPTS="-Xms2048m -Xmx4096m -XX:PermSize=1024m -XX:MaxNewSize= 2048m  -XX:MaxPermSize=2048m"

4、修改tomcat的配置信息

#cd /opt/apache-tomcat-9.0.13/conf
#vi server.xml

添加jvmRoute信息，且主从节点不一致（例：主——tomcat1、从——tomcat2）
Mast：

Backup：

5，添加测试文件（双节点添加）

#cd /opt/apache-tomcat-9.0.13/webapps/
#mkdir test
#vi index.jsp

主机如下配置，从机需要将This is Tomcat Server 1改成This is Tomcat Server 2

<%--
  Created by IntelliJ IDEA.
  User: ccdk
  Date: 2018/4/27
  Time: 下午7:01
  To change this template use File | Settings | File Templates.
--%>
<%@ page contentType="text/html;charset=UTF-8" language="java" %>

  
    
  
  
  SessionID:<%=session.getId()%> 

  SessionIP:<%=request.getServerName()%> 

  SessionPort:<%=request.getServerPort()%>
  <%out.println("This is Tomcat Server 1");%> 
  

添加完以后要修改文件所属用户、组

cd /opt/apache-tomcat-9.0.13/webapps/
chown -R tomcat:tomcat test/index.jsp

七，安装Redis（双节点都需要安装配置）
1，安装Redis

#cd /opt/soft/
#mkdir -p /opt/redis
#tar -zxvf redis-5.0.3.tar.gz -C /opt/
#cd /opt/redis-5.0.3/
#make PREFIX=/opt/redis
#install
#cp /opt/redis-5.0.3/redis.conf /opt/redis/　　--将配置文件拷贝到/opt/redis目录下，方便启动

2，配置redis

#cd /opt/redis/
#vi redis.conf

主节点配置：

#bind 127.0.0.0  --把监听本地注释掉
port 6000  #设定监听端口
protected-mode no
pidfile "/var/run/redis_6000.pid"

从节点配置：

#bind 127.0.0.0
port 6000 #设定监听端口
protected-mode no
pidfile "/var/run/redis_6000.pid"
slaveof 10.200.36.166 6000

3，配置Tomcat连接Redis

这里需要先将jar包传到/opt/apache-tomcat-9.0.13/lib/

commons-pool2-2.4.2.jar
jedis-2.9.0.jar
tomcat-redis-session-manager.jar

接下来修改context.xml（主备机一样）：

#vi /opt/apache-tomcat-9.0.13/conf/context.xml

4，启动Redis，检测是否正常

#cd /opt/redis
#./bin/redis-server ./redis.conf &　　--启动

主机启动成功示例:

30472:C 08 Mar 2019 16:45:58.778 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
30472:C 08 Mar 2019 16:45:58.778 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=30472, just started
30472:C 08 Mar 2019 16:45:58.778 # Configuration loaded
30472:M 08 Mar 2019 16:45:58.779 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 5.0.3 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6000
 |    `-._   `._    /     _.-'    |     PID: 30472
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           http://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

30472:M 08 Mar 2019 16:45:58.780 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
30472:M 08 Mar 2019 16:45:58.780 # Server initialized
30472:M 08 Mar 2019 16:45:58.780 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
30472:M 08 Mar 2019 16:45:58.780 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
30472:M 08 Mar 2019 16:45:58.780 * DB loaded from disk: 0.000 seconds
30472:M 08 Mar 2019 16:45:58.780 * Ready to accept connections
30472:M 08 Mar 2019 16:45:58.782 * Replica 10.200.36.167:6000 asks for synchronization
30472:M 08 Mar 2019 16:45:58.782 * Partial resynchronization not accepted: Replication ID mismatch (Replica asked for 'cf37ef581b1ca03add649e6c8de36524a28acc02', my replication IDs are '007ccfcf163d9117ac34710be5d623e031d10779' and '0000000000000000000000000000000000000000')
30472:M 08 Mar 2019 16:45:58.782 * Starting BGSAVE for SYNC with target: disk
30472:M 08 Mar 2019 16:45:58.784 * Background saving started by pid 30476
30476:C 08 Mar 2019 16:45:58.787 * DB saved on disk
30476:C 08 Mar 2019 16:45:58.788 * RDB: 4 MB of memory used by copy-on-write
30472:M 08 Mar 2019 16:45:58.881 * Background saving terminated with success
30472:M 08 Mar 2019 16:45:58.881 * Synchronization with replica 10.200.36.167:6000 succeeded

从机启动成功示例：

12452:C 08 Mar 2019 16:45:58.747 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
12452:C 08 Mar 2019 16:45:58.747 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=12452, just started
12452:C 08 Mar 2019 16:45:58.747 # Configuration loaded
12452:S 08 Mar 2019 16:45:58.748 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 5.0.3 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6000
 |    `-._   `._    /     _.-'    |     PID: 12452
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           http://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

12452:S 08 Mar 2019 16:45:58.749 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
12452:S 08 Mar 2019 16:45:58.749 # Server initialized
12452:S 08 Mar 2019 16:45:58.749 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
12452:S 08 Mar 2019 16:45:58.749 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
12452:S 08 Mar 2019 16:45:58.749 * DB loaded from disk: 0.000 seconds
12452:S 08 Mar 2019 16:45:58.749 * Before turning into a replica, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
12452:S 08 Mar 2019 16:45:58.749 * Ready to accept connections
12452:S 08 Mar 2019 16:45:58.750 * Connecting to MASTER 10.200.36.166:6000
12452:S 08 Mar 2019 16:45:58.750 * MASTER <-> REPLICA sync started
12452:S 08 Mar 2019 16:45:58.750 * Non blocking connect for SYNC fired the event.
12452:S 08 Mar 2019 16:45:58.750 * Master replied to PING, replication can continue...
12452:S 08 Mar 2019 16:45:58.751 * Trying a partial resynchronization (request cf37ef581b1ca03add649e6c8de36524a28acc02:12329042).
12452:S 08 Mar 2019 16:45:58.754 * Full resync from master: 3a676503f92e3403f9d4a20097d8f16f7dfa2b5d:0
12452:S 08 Mar 2019 16:45:58.754 * Discarding previously cached master state.
12452:S 08 Mar 2019 16:45:58.850 * MASTER <-> REPLICA sync: receiving 175 bytes from master
12452:S 08 Mar 2019 16:45:58.851 * MASTER <-> REPLICA sync: Flushing old data
12452:S 08 Mar 2019 16:45:58.851 * MASTER <-> REPLICA sync: Loading DB in memory
12452:S 08 Mar 2019 16:45:58.851 * MASTER <-> REPLICA sync: Finished with success

5、配置Sentinel

#cd /opt/redis/
#vi sentinel.conf

主从节点一致：

#守护进程
daemonize yes
protected-mode no
#端口
port 1777
#运行日志
logfile "/opt/redis/sentinel.log"
pidfile "/opt/redis/sentinel.pid"
#配置监控主服务器,mymaster:定义主服务器名,10.200.36.166:主服务ip 6000:主服务端口 2:设置当有1个sentinel判断master故障后才真正认为master无法继续提供服务，即开始容灾措施
sentinel monitor mymaster 10.200.36.166 6000 1
#指定sentinel认定一个服务器断线的毫秒数,即一个sentinel认定服务SDOWN，在这个配置时间内需要获得指定个数的Sentinel判定ODWON,才开始failover
sentinel down-after-milliseconds mymaster 2000
#从节点数量为几后面就写几，我这里只有一个从节点，所以写1
sentinel parallel-syncs mymaster 1

6，启动Sentinel

#cd /opt/redis
#./bin/redis-sentinel ./sentinel.conf &　　--启动

7，检验

#./bin/redis-cli -h 10.200.36.166 -p 6000 # -h:ip地址,-p:访问端口

主节点:

10.200.36.166:6000> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=10.200.36.167,port=6000,state=online,offset=616,lag=1
master_replid:3a676503f92e3403f9d4a20097d8f16f7dfa2b5d
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:616
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:616

从节点:

10.200.36.167:6000> info replication
# Replication
role:slave
master_host:10.200.36.166
master_port:6000
master_link_status:up
master_last_io_seconds_ago:3
master_sync_in_progress:0
slave_repl_offset:616
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:3a676503f92e3403f9d4a20097d8f16f7dfa2b5d
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:616
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:616

sentinel使用./bin/redis-cli -h 10.200.36.166 -p1777访问

八，基础指令

1，启动顺序
Nginx -> Tomcat -> Keepalived -> Redis
2，指令

#cd /usr/local/nginx/sbin
#./nginx --启动nginx
#./nginx -s stop --关闭nginx
#./nginx -s reload --重启nginx

#su tomcat --切换成tomcat用户
#cd /opt/apache-tomcat-9.0.13/bin/
#./startup.sh --启动tomcat
#./shutdown.sh --关闭tomcat

#systemctl start keepalived.service --启动keepalived
#systemctl stop keepalived.service --关闭keepalived
#systemctl restart keepalived.service --重启keepalived

#cd /opt/redis
#./bin/redis-server ./redis.conf & --启动redis
#./bin/redis-cli -p 6000 shutdown --关闭redis
#./bin/redis-sentinel ./sentinel.conf & --启动sentinel
#./bin/redis-cli -p 1777 shutdown --关闭sentinel 注：redis有两种启动方式。 前台启动：./bin/redis-server ./redis.conf 后台启动：./bin/redis-server
./redis.conf &
还有种后台启动的方式是改redis.conf配置文件，将daemonize改为yes，这样直接使用./bin/redis-server
./redis.conf也是后台启动。

九，搭建MySql主主
1.安装MySQL服务（建议源码安装）
1.1 yum安装依赖包

yum -y install make gcc gcc-c++ ncurses-devel bison openssl-devel

1.2 添加MySQL所需要的用户和组，添加boost库Cmake编译安装时使用。

#groupadd -g 77 mysql
#adduser -u 77 -g mysql -s /sbin/nologin mysql
#cd /opt/soft/
#tar -zxvf boost_1_59_0.tar.gz -C /usr/local/boost
#cd /usr/local/boost/boost_1_59_0/
#./bootstrap.sh

1.3 创建mysql数据目录

#mkdir -p /optl/mysql/

1.4 解压编译安装cmake、MySQL

#cd /opt/soft/
#tar -zxvf cmake-3.13.4.tar.gz
#cd cmake-3.13.4/
#./bootstrap
#gmake
#make &make install

1.5安装MySql

#cd /opt/soft/
#tar -zxvf mysql-5.7.25.tar.gz
#cd mysql-5.7.25

#cmake . -DCMAKE_INSTALL_PREFIX=/usr/local/mysql \
--DDEFAULT_CHARSET=utf8 \
--DDEFAULT_COLLATION=utf8_general_ci \
--DENABLED_LOCAL_INFILE=ON \
--DWITH_INNOBASE_STORAGE_ENGINE=1 \
--DWITH_FEDERATED_STORAGE_ENGINE=1 \
--DWITH_BLACKHOLE_STORAGE_ENGINE=1 \
--DWITHOUT_EXAMPLE_STORAGE_ENGINE=1 \
--DWITH_PARTITION_STORAGE_ENGINE=1 \
--DWITH_PERFSCHEMA_STORAGE_ENGINE=1 \
--DCOMPILATION_COMMENT='Mysqlma' \
--DWITH_READLINE=ON \
--DWITH_BOOST=/usr/local/boost/ boost_1_59_0/ \
--DSYSCONFDIR=/data/mysqldata/3306

#make && make install

1.6 添加开机启动脚本

#cp support-files/mysql.server /etc/rc.d/init.d/mysqld

1.7 添加Master配置文件/etc/my.cnf

#vim /etc/my.cnf

[client]
port = 3306
socket = /tmp/mysql.sock

[mysqld]
basedir = /usr/local/mysql
port = 3306
socket = /tmp/mysql.sock
datadir = /usr/local/mysql/data
pid-file = /usr/local/mysql/data/mysql.pid
log-error = /usr/local/mysql/data/mysql.err

server-id = 1
auto_increment_offset = 1
auto_increment_increment = 2                                            #奇数ID

log-bin = mysql-bin                                                     #打开二进制功能,MASTER主服务器必须打开此项
binlog-format=ROW
#binlog-row-p_w_picpath=minimal
log-slave-updates=true
gtid-mode=on
enforce-gtid-consistency=true
master-info-repository=TABLE
relay-log-info-repository=TABLE
sync-master-info=1
slave-parallel-workers=0
sync_binlog=0
binlog-checksum=CRC32
master-verify-checksum=1
slave-sql-verify-checksum=1
binlog-rows-query-log_events=1
#expire_logs_days=5
max_binlog_size=1024M                                                   #binlog单文件最大值
							
replicate-ignore-db = mysql                                            #忽略不同步主从的数据库
replicate-ignore-db = information_schema
replicate-ignore-db = performance_schema
max_connections = 3000
max_connect_errors = 30

skip-character-set-client-handshake                                     #忽略应用程序想要设置的其他字符集
init-connect='SET NAMES utf8'                                           #连接时执行的SQL
character-set-server=utf8                                               #服务端默认字符集
wait_timeout=1800                                                       #请求的最大连接时间
interactive_timeout=1800                                                #和上一参数同时修改才会生效
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES                     #sql模式
max_allowed_packet = 10M
bulk_insert_buffer_size = 8M
query_cache_type = 1
query_cache_size = 128M
query_cache_limit = 4M
key_buffer_size = 256M
read_buffer_size = 16K

skip-name-resolve
slow_query_log=1
long_query_time = 6
slow_query_log_file=slow-query.log
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 16M

[mysql]
no-auto-rehash

[myisamchk]
key_buffer_size = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout

[mysqldump]
quick
max_allowed_packet = 16M

[mysqld_safe]

1.8 特别参数说明

log-slave-updates = true         #将复制事件写入binlog,一台服务器既做主库又做从库此选项必须要开启
#Master自增长ID auto_increment_offset = 1 auto_increment_increment = 2           #奇数ID
#Backup自增加ID auto_increment_offset = 2 auto_increment_increment = 2           #偶数ID

1.9 添加Backup配置文件/etc/my.cnf

#vim /etc/my.cnf

[client]
port = 3306
socket = /tmp/mysql.sock

[mysqld]
basedir = /usr/local/mysql
port = 3306
socket = /tmp/mysql.sock
datadir = /usr/local/mysql/data
pid-file = /usr/local/mysql/data/mysql.pid
log-error = /usr/local/mysql/data/mysql.err

server-id = 2
auto_increment_offset = 2
auto_increment_increment = 2                                            #偶数ID

log-bin = mysql-bin                                                     #打开二进制功能,MASTER主服务器必须打开此项
binlog-format=ROW
#binlog-row-p_w_picpath=minimal
log-slave-updates=true
gtid-mode=on
enforce-gtid-consistency=true
master-info-repository=TABLE
relay-log-info-repository=TABLE
sync-master-info=1
slave-parallel-workers=0
sync_binlog=0
binlog-checksum=CRC32
master-verify-checksum=1
slave-sql-verify-checksum=1
binlog-rows-query-log_events=1
#expire_logs_days=5
max_binlog_size=1024M                                                   #binlog单文件最大值

replicate-ignore-db = mysql                                             #忽略不同步主从的数据库
replicate-ignore-db = information_schema
replicate-ignore-db = performance_schema

max_connections = 3000
max_connect_errors = 30

skip-character-set-client-handshake                                     #忽略应用程序想要设置的其他字符集
init-connect='SET NAMES utf8'                                           #连接时执行的SQL
character-set-server=utf8                                               #服务端默认字符集
wait_timeout=1800                                                       #请求的最大连接时间
interactive_timeout=1800                                                #和上一参数同时修改才会生效
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES                     #sql模式
max_allowed_packet = 10M
bulk_insert_buffer_size = 8M
query_cache_type = 1
query_cache_size = 128M
query_cache_limit = 4M
key_buffer_size = 256M
read_buffer_size = 16K

skip-name-resolve
slow_query_log=1
long_query_time = 6
slow_query_log_file=slow-query.log
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 16M

[mysql]
no-auto-rehash

[myisamchk]
key_buffer_size = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout

[mysqldump]
quick
max_allowed_packet = 16M

[mysqld_safe]

1.10 初始化MySQL

#cd /usr/local/mysql

#./bin/mysqld \
--initialize-insecure \
--user=mysql \
--basedir=/usr/local/mysql \
--datadir=/usr/local/mysql/data

1.11 为启动脚本赋予可执行权限并启动MySQL

#chmod +x /etc/rc.d/init.d/mysqld
#/etc/init.d/mysqld start

2. 配置主从同步
2.1 添加主从同步账户
Mast上：

#mysql> grant replication slaveon . to ‘root’@‘10.200.36.167’ identified by ‘123456’;
#mysql> flush privileges;

Backup上：

#mysql> grant replication slave on . to ‘root’@‘10.200.36.166’ identified by ‘123456’;
#mysql> flush privileges;

2.2 查看主库的状态
Master上：

#mysql> show master status;

mysql> show master status;
+------------------+----------+--------------+------------------+----------------------------------------------------------------------------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set                                                                      |
+------------------+----------+--------------+------------------+----------------------------------------------------------------------------------------+
| mysql-bin.000005 |     5094 |              |                  | 9be18158-3fc0-11e9-b3bc-00505694ed12:1-14,
ae807b12-3fbf-11e9-bd3d-00505694bdbe:1-1620 |
+------------------+----------+--------------+------------------+----------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

Backup上：

#mysql> show master status;

mysql> show master status;
+------------------+----------+--------------+------------------+----------------------------------------------------------------------------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set                                                                      |
+------------------+----------+--------------+------------------+----------------------------------------------------------------------------------------+
| mysql-bin.000005 |     5094 |              |                  | 9be18158-3fc0-11e9-b3bc-00505694ed12:1-14,
ae807b12-3fbf-11e9-bd3d-00505694bdbe:1-1620 |
+------------------+----------+--------------+------------------+----------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

2.3 配置同步信息：
Master上：

#mysql>change master to master_host=‘10.200.36.167’,master_port=3306,master_user=‘root’,master_password=‘123456’,master_log_file=‘mysql-bin.000003’,master_log_pos=5094;
#mysql> start slave;
#mysql> show slave status\G;

显示有如下状态则正常：

mysql> show slave status\G;
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 10.200.36.167
                  Master_User: root
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000005
          Read_Master_Log_Pos: 5094
               Relay_Log_File: localhost-relay-bin.000006
                Relay_Log_Pos: 2408
        Relay_Master_Log_File: mysql-bin.000005
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB:
          Replicate_Ignore_DB: mysql,information_schema,performance_schema,test,zabbix
           Replicate_Do_Table:
       Replicate_Ignore_Table:
      Replicate_Wild_Do_Table:
  Replicate_Wild_Ignore_Table:
                   Last_Errno: 0
                   Last_Error:
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 5094
              Relay_Log_Space: 2659
              Until_Condition: None
               Until_Log_File:
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File:
           Master_SSL_CA_Path:
              Master_SSL_Cert:
            Master_SSL_Cipher:
               Master_SSL_Key:
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error:
               Last_SQL_Errno: 0
               Last_SQL_Error:
  Replicate_Ignore_Server_Ids:
             Master_Server_Id: 2
                  Master_UUID: 9be18158-3fc0-11e9-b3bc-00505694ed12
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind:
      Last_IO_Error_Timestamp:
     Last_SQL_Error_Timestamp:
               Master_SSL_Crl:
           Master_SSL_Crlpath:
           Retrieved_Gtid_Set: 9be18158-3fc0-11e9-b3bc-00505694ed12:1-14
            Executed_Gtid_Set: 9be18158-3fc0-11e9-b3bc-00505694ed12:1-14,
ae807b12-3fbf-11e9-bd3d-00505694bdbe:1-1620
                Auto_Position: 1
         Replicate_Rewrite_DB:
                 Channel_Name:
           Master_TLS_Version:
1 row in set (0.00 sec)

Backup上：

#mysql>change master to master_host=‘10.200.36.166’,master_port=3306,master_user=‘root’,master_password=‘123.com123.COM’,master_log_file=‘mysql-bin.000003’,master_log_pos=5094;
start slave;
mysql> show slave status\G;

显示有如下状态则正常：

mysql> show slave status\G;
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 10.200.36.166
                  Master_User: root
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000005
          Read_Master_Log_Pos: 5094
               Relay_Log_File: localhost-relay-bin.000006
                Relay_Log_Pos: 3315
        Relay_Master_Log_File: mysql-bin.000005
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB:
          Replicate_Ignore_DB: mysql,information_schema,performance_schema,test,zabbix
           Replicate_Do_Table:
       Replicate_Ignore_Table:
      Replicate_Wild_Do_Table:
  Replicate_Wild_Ignore_Table:
                   Last_Errno: 0
                   Last_Error:
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 5094
              Relay_Log_Space: 3582
              Until_Condition: None
               Until_Log_File:
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File:
           Master_SSL_CA_Path:
              Master_SSL_Cert:
            Master_SSL_Cipher:
               Master_SSL_Key:
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error:
               Last_SQL_Errno: 0
               Last_SQL_Error:
  Replicate_Ignore_Server_Ids:
             Master_Server_Id: 1
                  Master_UUID: ae807b12-3fbf-11e9-bd3d-00505694bdbe
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind:
      Last_IO_Error_Timestamp:
     Last_SQL_Error_Timestamp:
               Master_SSL_Crl:
           Master_SSL_Crlpath:
           Retrieved_Gtid_Set: ae807b12-3fbf-11e9-bd3d-00505694bdbe:1-3:5-1620
            Executed_Gtid_Set: 9be18158-3fc0-11e9-b3bc-00505694ed12:1-14,
ae807b12-3fbf-11e9-bd3d-00505694bdbe:1-1620
                Auto_Position: 1
         Replicate_Rewrite_DB:
                 Channel_Name:
           Master_TLS_Version:
1 row in set (0.00 sec)

3.测试主从同步
3.1 在Master上创建一个数据库测试同步效果

#mysql> show databases;
#mysql> create database test01;
#mysql> show databases;

3.2 到masterB查看是否已经同步创建数据库

#mysql> show databases;

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| mytest             |
| sys                |
| test01             |
+--------------------+
5 rows in set (0.01 sec)

4. 开启MySQL5.7的GTID功能
Master和Backup分别执行如下命令：

#mysql> stop slave;
Query OK, 0 rows affected (0.00 sec)
#mysql> change master to MASTER_AUTO_POSITION=1;
Query OK, 0 rows affected (0.01 sec)
#mysql> start slave;
Query OK, 0 rows affected (0.00 sec)

5．MySql更改密码并添加远程登录功能：
查看数据库随机密码：

#cat /var/log/mysqld.log | grep ‘A temporary password’

更改数据库密码：

mysqladmin -u root -p password “abc123”

授权远程登录：

mysql> grant all privileges on . to ‘root’@’%’ identified by ‘abc123’ with grant option;
mysql>FLUSH PRIVILEGES;

6，以上就是keepalived+nginx+tomcat+redis+mysql主主架构部署完成。
可以拿测试的mode上去进行测试，nginx的负载，session会话共享，keepalived的VIP的迁移，Mysql的主主同步。
在keepalived的配置上可以添加多个脚本判断，请自行添加。
写的第一篇博客，纯手动码字，谢谢观赏。
如有不足，请各位大佬指出。