Keepalived+Nginx+Tomcat+Redis搭建高可用服务

Keepalived+Nginx+Tomcat+Redis搭建高可用服务

Keepalived作用是检测服务器的状态，Nginx的作用是负载均衡，Tomcat提供web应用，redis提供session共享。

一、环境条件

名称	IP	端口	系统	版本	备注
Keepalived+Nginx	192.168.10.176	8080	Centos6.10	Keepalived v1.2.13+nginx/1.18.0	MASTER
Keepalived+Nginx	192.168.10.177	8080	Centos6.10	Keepalived v1.2.13+nginx/1.18.0	BACKUP
Tomcat+Redis	192.168.10.178	8086/6379	Centos6.10	Tomcat/7.0.53+Redis v3.2.12	Web Server1
Tomcat+Redis	192.168.10.179	8086/6379	Centos6.10	Tomcat/7.0.53+Redis v3.2.12	Web Server2
VIP	192.168.10.110	8080	Centos6.10		虚拟飘移IP

服务器ssh免密码登录配置

ssh-keygen
ssh-copy-id -i ~root/.ssh/id_rsa.pub 192.168.10.176
ssh-copy-id -i ~root/.ssh/id_rsa.pub 192.168.10.177
ssh-copy-id -i ~root/.ssh/id_rsa.pub 192.168.10.178
ssh-copy-id -i ~root/.ssh/id_rsa.pub 192.168.10.179

二、Tomcat服务器配置

1.JDK安装
安装jdk、查看jdk版本

[root@server1 ~]# yum -y install java-1.7.0-openjdk
[root@server1 ~]# java -version

2.SERVER1服务器配置
Tomcat部署比较简单，修改端口启动可以正常访问，开放WEB端口，修改server.xml如下

[root@server1 conf]# vim server.xml
# 多个Tomcat在同台服务器上需要修改
<Server port="8005" shutdown="SHUTDOWN">
# 修改访问端口
<Connector port="8086" protocol="HTTP/1.1"
           connectionTimeout="20000"
           redirectPort="8443" />
# 多个Tomcat在同台服务器上需要修改
<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
# 配置Engine节点的路由信息，session复制和负载均衡需要用
# 添加jvmRoute="TOMCAT01"、每个tomcat都需要修改
<Engine name="Catalina" defaultHost="localhost" jvmRoute="TOMCAT01">
# 去掉Cluster注释
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>

在访问的项目index.jsp页面加上以下代码

# 添加以下内容
<%HttpSession session=request.getSession();%>
SESSION ID: <%=session.getId()%><BR>
PORT: <%=request.getServerPort()%><BR>
<%out.println("IP:192.168.10.178");%>
# SERVICE2修改成IP:192.168.10.179

3.SERVER2服务器配置
将配置好的tomcat发送到server2，修改jvmRoute=“Tomcat02”

[root@server1 home]# scp -r apache-tomcat-7.0.53 192.168.10.179:/home/

Tomcat没有重启命令，每次修改后还需要关闭再启动，太麻烦，我将关闭和启动整合一下，变成重启，下面是重启脚本，将脚本保存到tomcat/bin/目录，重启直接执行./restart.sh

#!/bin/bash
# -*- coding: UTF-8 -*-
# Filename: restart.sh
# Description: 用于Tomcat重新启动
# Author: SERE

LOGSDIR=../logs
./shutdown.sh &> /dev/null
echo -e "\033[32mINFO: Tomcat close to finish\033[0m"
echo -e "\033[32mINFO: Now restart tomcat\033[0m"
nohup ./startup.sh &> /dev/null
sleep 1
tail -f $LOGSDIR/catalina.out

4.测试web应用
分别启动tomcat进行访问

三、Nginx服务器配置

1.Nginx安装
Nginx yum 源地址：
http://nginx.org/packages/centos/

[root@master ~]# yum -y install nginx
# nginx -t ：测试配置文件是否有语法错误
[root@master ~]# nginx -v

2.Master服务器配置
Nginx转发配置

[root@master ~]# vim /etc/nginx/conf.d/cluster.conf
# 添加以下内容
upstream application {
    server 192.168.10.178:8086 weight=1;
    server 192.168.10.179:8086 weight=1;
    }
server {
    listen      8080;
    # BACKUP改为：192.168.10.177
    server_name  192.168.10.176;
    location / {
        root  /var/www/html;
        index  index.php index.html index.htm;
        proxy_pass http://application;
    }
}

测试配置文件是否有语法错误

[root@master ~]# nginx -t
[root@master ~]# service nginx restart

测试负载nginx转发web应用
http://192.168.10.176:8080/application/

3.Backup服务器配置
将nginx转发配置文件发送到backup服务器

[root@master ~]# scp -r /etc/yum.repos.d/nginx.repo 192.168.10.177:/etc/yum.repos.d/
[root@master ~]# scp -r /etc/nginx/conf.d/cluster.conf 192.168.10.177:/etc/nginx/conf.d/
[root@backup ~]# nginx -t
[root@backup ~]# service nginx restart

4.测试负载nginx转发web应用
http://192.168.10.177:8080/application/

四、Redis服务器配置

1.Redis安装
上传jar包到tomcat/lib/目录

[root@server1 ~]# yum -y install redis
[root@server1 ~]# vim /etc/redis.conf
# 关闭protected-mode模式
protected-mode no
# 监听地址，端口为默认6379,允许监听所有的
bind 0.0.0.0

下载地址：
https://github.com/jcoleman/tomcat-redis-session-manager
https://download.csdn.net/download/SERE0211/12527300
找不到私信我

commons-pool2-2.2
jedis-2.5.2
tomcat-redis-session-manager-2.0.0

2.SERVER1服务器配置
将jar发送server2服务器

[root@server1 lib]# scp -r commons-pool2-2.2.jar jedis-2.5.2.jar tomcat-redis-session-manager-2.0.0.jar 192.168.10.179:/home/tomcat/lib/

修改conf/context.xml

[root@server1 lib]# vim ../conf/context.xml
# 添加以下内容
    <WatchedResource>WEB-INF/web.xml</WatchedResource>
    <WatchedResource>${catalina.base}/conf/web.xml</WatchedResource>
    <Valve className="com.orangefunction.tomcat.redissessions.RedisSessionHandlerValve" />
    <Manager className="com.orangefunction.tomcat.redissessions.RedisSessionManager"
     host="192.168.10.178"
     port="6379"
     database="0"
     maxInactiveInterval="60"/>
# 一般是从23行开始添加

如果是redis集群

[root@server1 lib]# vim ../conf/context.xml
# 添加以下内容
    <Valve className="com.orangefunction.tomcat.redissessions.RedisSessionHandlerValve" />
    <Manager className="com.orangefunction.tomcat.redissessions.RedisSessionManager"
     sentinels="192.168.10.178:26379,192.168.10.179:26379" />
     # 这里是redis哨兵的地址,多个sentinels集群就写多个ip地址
     # 这里是sentinel的master名称，一定不要写错
     sentinelMaster="mymaster"
     # 还有认证密码
     password="123456"
     maxInactiveInterval="60"

配置redis sentinel(哨兵)

[root@server1 ~]# vim /etc/redis-sentinel.conf
# 添加以下内容
port 26379
dir /tmp
sentinel monitor mymaster 192.168.10.178 6379 1
sentinel down-after-milliseconds mymaster 30000
sentinel auth-pass mymaster 123456
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000
logfile /var/log/redis/sentinel.log

重启redis

[root@server1 ~]# service redis restart
[root@server1 ~]# tail -f /var/log/redis/redis.log

重启redis-sentinel(哨兵)

[root@server1 ~]# service redis-sentinel restart
[root@server1 ~]# tail -f /var/log/redis/sentinel.log 

3.SERVER2服务器配置
将context.xml发送server2服务器

[root@server1 bin]# scp ../conf/context.xml 192.168.10.179:/home/apache-tomcat-7.0.53/conf/

将配置redis sentinel(哨兵)发到server2服务器

[root@server1 bin]# scp /etc/redis-sentinel.conf 192.168.10.179:/etc/redis-sentinel.conf

修改从节点，添加slaveof，修改slave-read-only yes

[root@server2 ~]# vim /etc/redis.conf
slaveof 192.168.10.178 6379
# slave-read-only no
# 如果slave-read-only no，context.xml文件中host可以修改host="192.168.10.179"

#重启tomcat，redis也需要启动，否则访问报错

[root@server2 ~]# service redis restart

4.测试SESSION会话共享
在server1插入数据

[root@server1 ~]# redis-cli -h 192.168.10.178
192.168.10.178:6379> set name SERE

在server2提取数据

[root@server2 ~]# redis-cli -h 192.168.10.179
192.168.10.179:6379> get name

五、Keepalived服务器配置

1.Keepalived安装

[root@master ~]# yum -y install  keepalived

2.Master服务器配置

[root@master ~]# vim /etc/keepalived/keepalived.conf
# 添加以下内容
! Configuration File for keepalived
global_defs {

    router_id hostname #机器标识
}

vrrp_instance VI_1 {
    state MASTER  #主备状态
    interface eth0  #监听网卡
    virtual_router_id 51  #虚拟路由id
    mcast_src_ip 192.168.10.176  #本机IP地址
    priority 100  #权重
    nopreempt  #设置不抢占
    advert_int 1  #同步检查时间设置
    authentication {  #认证配置
        auth_type PASS  #认证方式
        auth_pass 1111  #认证密码
    }
    track_script {
        chk_nginx  #执行Nginx监控的服务#
    }
    virtual_ipaddress { #虚拟IP
        192.168.10.110  #虚拟IP池MASTER&&BACKUP, 两个设置必须一样，虚拟ip，可以定义多个
    }
}

将master配置好的keepalived.conf发送到backup服务器

[root@master ~] service keepalived restart
[root@master ~] scp /etc/keepalived/keepalived.conf 192.168.10.177:/etc/keepalived/keepalived.conf

3.Backup服务器配置

[root@master ~]# vim /etc/keepalived/keepalived.conf
# 添加以下内容
! Configuration File for keepalived
global_defs {

    router_id backup #机器标识
}

vrrp_instance VI_1 {
    state MASTER  #主备状态
    interface eth0  #监听网卡
    virtual_router_id 51  #虚拟路由id
    mcast_src_ip 192.168.10.177  #本机IP地址
    priority 100  #权重
    nopreempt  #设置不抢占
    advert_int 1  #同步检查时间设置
    authentication {  #认证配置
        auth_type PASS  #认证方式
        auth_pass 1111  #认证密码
    }
    track_script {
        chk_nginx  #执行Nginx监控的服务#
    }
    virtual_ipaddress { #虚拟IP
        192.168.10.110  #虚拟IP池MASTER&&BACKUP, 两个设置必须一样，虚拟ip，可以定义多个
    }
}

# 我将（# priority 100）master和backup设置的是一样的，当一个节点故障，另个节点自动变成主节点

4.测试高可用服务
http://192.168.10.110:8080/application

常用命令：
查看本地IP地址：ip addr

五.参考文档

nginx官网：
https://nginx.org/
redis官网：
https://redis.io/
keepalived官网：
https://www.keepalived.org/