2019-03-10:keepalived+nginx+tomcat实现双主热备动静分离负载均衡

一、安装nginx
1.安装make:

# yum -y install gcc automake autoconf libtool make

2.安装g++:

# yum install gcc gcc-c++

3.安装PCRE库
我安装在/usr/local/src,可以自己定义,到ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/选择你要安装的源码包,我选的最新的pcre-8.43.tar.gz

cd /usr/local/src
wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.43.tar.gz

下载完成

tar -zxvf pcre-8.43.tar.gz
cd pcre-8.43
./configure
make
make install

4.安装zlib库(同上)

cd /usr/local/src

wget http://zlib.net/zlib-1.2.11.tar.gz
tar -zxvf zlib-1.2.11.tar.gz
cd zlib-1.2.11
./configure
make
make install

5.安装openssl(个别系统默认没有安装)
openssl version -a 查看是否已安装

如果没有安装,执行:

cd /usr/local/src
wget https://www.openssl.org/source/openssl-1.0.1t.tar.gz
tar -zxvf openssl-1.0.1t.tar.gz
cd openssl-1.0.1
./configure
make
make install

6.安装nginx

cd /usr/local/src
wget http://nginx.org/download/nginx-1.16.0.tar.gz
tar -zxvf nginx-1.16.0.tar.gz
cd nginx-1.16.0
./configure
make
make install

注:教程说这一步有可能会报错,我没有遇到,如果遇到了执行下面:

ubuntu下

apt-get install openssl
apt-get install libssl-dev

centos下

yum -y install openssl openssl-devel

7.启动nginx
如果需要修改端口,/usr/local/nginx/conf/nginx.conf

查看80端口是否占用:netstat -ano|grep 80
启动:/usr/local/nginx/sbin/nginx
查看是否启动 :ps -A | grep nginx 或 netstat -anp | grep :80

访问:crul 192.168.75.128

如果远程访问不到,远程tenlet 192.168.75.218 80查看端口是否正常连接,如连接失败执行:
开启防火墙端口:firewall-cmd --zone=public --add-port=80/tcp --permanent
防火墙重新加载:firewall-cmd --reload
检查开放端口:firewall-cmd --permanent --zone=public --list-ports

8.配置开机启动

 vim /lib/systemd/system/nginx.service

[Unit]
Description=nginx service
After=network.target 

[Service] 
Type=forking 
ExecStart=/usr/local/nginx/sbin/nginx
ExecReload=/usr/local/nginx/sbin/nginx -s reload
ExecStop=/usr/local/nginx/sbin/nginx -s quit
PrivateTmp=true 

[Install] 
WantedBy=multi-user.target

[Install] 
WantedBy=multi-user.target

启动:systemctl start nginx
设置开机启动:systemctl enable nginx

如果报错执行:

ln -s ‘/etc/systemd/system/nginx.service’ ‘/etc/systemd/system/multi-user.target.wants/nginx.service’


或:Centos7使用yum安装:
1、添加源
  默认情况Centos7中无Nginx的源,最近发现Nginx官网提供了Centos的源地址。因此可以如下执行命令添加源:

# rpm -Uvh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm

2、安装Nginx
  通过yum search nginx看看是否已经添加源成功。如果成功则执行下列命令安装Nginx。

# yum install -y nginx

3、启动Nginx并设置开机自动运行

# systemctl start nginx.service
# systemctl enable nginx.service

查看运行nginx目录: ps -ef | grep nginx

# config: /etc/nginx/nginx.conf     安装目录
# config: /etc/sysconfig/nginx
# pidfile: /var/run/nginx.pid

二、Noggin+tomcat实现动静分离和多个tomcat负载均衡
1.tomcat配置
修改多个tomcat bin目录下Catalina.sh文件中TITLE的值:按顺序app01、app02

  1. Nginx.conf配置:
#配置开始-----------
#user  nobody;
#有1个工作子进程,可以自行修改,但太大无益,因为争夺CPU,一般设置为CPU数*核数
worker_processes  1;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;

events {
#单个后台worker process进程的最大并发链接数
#最大连接数 = worker_processes * worker_connections
#这里指一个子进程最大允许连接1024
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on;

    # 使用默认策略,轮询
    upstream webserver{
        # 下面介绍几种负载均衡策略,其中轮询、weight、ip_hash是nginx内置的,可以直接使用。fair和url_hash需要第三方支持才可以使用。
        # 1、轮询(默认):每个请求按时间顺序逐一分配到不同的后端服务器,如果后端服务器down掉,能自动剔除。
        server 192.168.1.14:8081;
        server 192.168.1.14:8080;

        # 2、weight:指定权重,按照权重进行请求的分配。wight和访问比例成正比,适合后端服务器性能不均的情况。
        # 下面的配置就会经常访问8288的服务。如果后端服务器8288 down掉,能够立刻切换到8299或者8290。如果8288再次启动,则又能回到原有的权重配置上。8288可以继续提供服务。
        # server localhost:8288 weight=10;
        # server localhost:8289 weight=1;
        # server localhost:8290 weight=1;

        # 3、ip_hash:每个请求按照ip的hash结果进行分配,这样的话每个访客固定请求一个后端服务器,可以解决session没共享的问题。
        # 如果8288 down掉,则依然可以访问,可能会缓存8289或者8290。如果8288启动,则会从8289或8290切换到8288。
        # ip_hash; 
        # server localhost:8288;
        # server localhost:8289;
        # server localhost:8290;

        # 4、fair(第三方):后端服务器响应时间短的优先分配。
        # fair; 
        # server localhost:8288;
        # server localhost:8289;
        # server localhost:8290;

        # 5、url_hash(第三方):按访问的url的hash结果来分配请求,这样相同url会分配到相同的后端服务器。适合后端服务器有缓存的情况。
        # hash $request_uri; 
        # hash_method crc32; 
        # server localhost:8288;
        # server localhost:8289;
        # server localhost:8290;
    }

    server {
        listen       80;
        server_name  192.168.75.128;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;

#定位,把特殊的文件再次定位,如image目录单独处理或html单独处理

    location ~ .*\.(html|htm|gif|jpg|jpeg|bmp|png|ico|txt|js|css)$ {
        root   html;
        index  index.html index.htm;
    }

        location ~ .*\.(jsp|do|action)$ {  #动态
           proxy_pass http://webserver;
        }

        #error_page  404              /404.html;

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

        # proxy the PHP scripts to Apache listening on 127.0.0.1:80
        #
        #location ~ \.php$ {
        #    proxy_pass   http://127.0.0.1;
        #}
        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        #
        #location ~ \.php$ {
        #    root           html;
        #    fastcgi_pass   127.0.0.1:9000;
        #    fastcgi_index  index.php;
        #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
        #    include        fastcgi_params;
        #}

        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        #
        #location ~ /\.ht {
        #    deny  all;
        #}
    }

    # another virtual host using mix of IP-, name-, and port-based configuration
    #
    #server {
    #    listen       8000;
    #    listen       somename:8080;
    #    server_name  somename  alias  another.alias;

    #    location / {
    #        root   html;
    #        index  index.html index.htm;
    #    }
    #}

    # HTTPS server
    #
    #server {
    #    listen       443 ssl;
    #    server_name  localhost;

    #    ssl_certificate      cert.pem;
    #    ssl_certificate_key  cert.key;

    #    ssl_session_cache    shared:SSL:1m;
    #    ssl_session_timeout  5m;

    #    ssl_ciphers  HIGH:!aNULL:!MD5;
    #    ssl_prefer_server_ciphers  on;

    #    location / {
    #        root   html;
    #        index  index.html index.htm;
    #    }
    #}

}
#配置结束-----------------------------------------

三、keepalived+nginx+tomcat虚拟IP负载均衡

keepalived安装
1:安装环境

yum -y install kernel-devel*
yum -y install openssl-*
yum -y install popt-devel
yum -y install lrzsz
yum -y install openssh-clients
yum -y install libnl libnl-devel popt
Yum -y install libnfnetlink-devel

2.安装keepalived

cd /usr/local/src
wget http://www.keepalived.org/software/keepalived-1.4.2.tar.gz

解压:
tar -zxvf keepalived-1.4.2.tar.gz
进入文件夹:
cd keepalived-1.4.2
执行配置命令
./configure --prefix=/usr/local/keepalived
编译
make
安装
make install
至此安装成功

3.执行拷贝文件:

# cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
# cp /usr/local/src/keepalived-1.4.2/keepalived/etc/init.d/keepalived /etc/init.d
# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig
# mkdir -p /etc/keepalived
# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived

添加可执行权限chmod +x /etc/init.d/keepalived

加入开机启动
chkconfig --add keepalived

添加时必须保证/etc/init.d/keepalived存在

chkconfig keepalived on

启动keepalived

启动:service keepalived start
停止:service keepalived stop
重启:service keepalived restart

5.防火墙添加arrp组播规则
1> iptables

shell> vi /etc/sysconfig/iptables
-A INPUT -p vrrp -d 224.0.0.18/32 -j ACCEPT

2> firewall

firewall-cmd --direct --permanent --add-rule ipv4 filter INPUT 0 --in-interface enp4s0 --destination 224.0.0.18 --protocol vrrp -j ACCEPT
firewall-cmd --reload

4、关闭selinux

# vi /etc/sysconfig/selinux
#修改:
SELINUX=disabled
#setenforce 0

6.配置主备模式:

主nginx
修改主nginx下/etc/keepalived/keepalived.conf文件

! Configuration File for keepalived
global_defs {
    ## keepalived 自带的邮件提醒需要开启 sendmail 服务。 建议用独立的监控或第三方 SMTP
    router_id liuyazhuang133 ## 标识本节点的字条串,通常为 hostname
} 
## keepalived 会定时执行脚本并对脚本执行的结果进行分析,动态调整 vrrp_instance 的优先级。如果脚本执行结果为 0,并且 weight 配置的值大于 0,则优先级相应的增加。如果脚本执行结果非 0,并且 weight配置的值小于 0,则优先级相应的减少。其他情况,维持原本配置的优先级,即配置文件中 priority 对应的值。
vrrp_script chk_nginx {
    script "/etc/keepalived/nginx_check.sh" ## 检测 nginx 状态的脚本路径
    interval 2 ## 检测时间间隔
    weight -20 ## 如果条件成立,权重-20
}
## 定义虚拟路由, VI_1 为虚拟路由的标示符,自己定义名称
vrrp_instance VI_1 {
    state MASTER ## 主节点为 MASTER, 对应的备份节点为 BACKUP
    interface eth1 ## 绑定虚拟 IP 的网络接口,与本机 IP 地址所在的网络接口相同, 我的是 eth1
    virtual_router_id 33 ## 虚拟路由的 ID 号, 两个节点设置必须一样, 可选 IP 最后一段使用, 相同的 VRID 为一个组,他将决定多播的 MAC 地址
    mcast_src_ip 192.168.136.13 ## 本机 IP 地址
    priority 100 ## 节点优先级, 值范围 0-254, MASTER 要比 BACKUP 高
    nopreempt ## 优先级高的设置 nopreempt 解决异常恢复后再次抢占的问题
    advert_int 1 ## 组播信息发送间隔,两个节点设置必须一样, 默认 1s
    ## 设置验证信息,两个节点必须一致
    authentication {
        auth_type PASS
        auth_pass 1111 ## 真实生产,按需求对应该过来
    }
    ## 将 track_script 块加入 instance 配置块
    track_script {
        chk_nginx ## 执行 Nginx 监控的服务
    } #
    # 虚拟 IP 池, 两个节点设置必须一样
    virtual_ipaddress {
        192.168.136.10 ## 虚拟 ip,可以定义多个
    }
}


备nginx
修改备nginx下/etc/keepalived/keepalived.conf文件
配置备nginx时需要注意:需要修改state为BACKUP , priority比MASTER低,virtual_router_id和master的值一致

! Configuration File for keepalived
global_defs {
    router_id liuyazhuang133
}
vrrp_script chk_nginx {
    script "/etc/keepalived/nginx_check.sh"
    interval 2
    weight -20
}
vrrp_instance VI_1 {
    state BACKUP
    interface eth1
    virtual_router_id 33
    mcast_src_ip 192.168.136.14
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    track_script {
        chk_nginx
    }
    virtual_ipaddress {
        192.168.136.10
    }
}

监控机脚本:

#!/bin/bash
A=`ps -C nginx --no-header | wc -l`
if [ $A -eq 0 ];then
/opt/nginx/sbin/nginx #尝试重新启动nginx
sleep 2 #睡眠2秒
if [ `ps -C nginx --no-header | wc -l` -eq 0 ];then
killall keepalived #启动失败,将keepalived服务杀死。将vip漂移到其它备份节点
fi
fi

文件授权:chmod 777 /etc/keepalived/nginx_check.sh
启动keepalived:

# service keepalived start
# ps -ef | grep keepalived

注:打开日志观察: tail -f /var/log/messages分析故障问题

7.配置双主模式:
主机1

global_defs {
router_id nginx_01 #标识本节点的名称,通常为hostname
}

vrrp_script chk_nginx {
script "/etc/keepalived/nginx_check.sh"
interval 2
weight -20
}

vrrp_instance VI_1 {
state BACKUP
interface enp0s3
virtual_router_id 51
mcast_src_ip 192.168.1.201
priority 100
advert_int 1
nopreempt
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.1.210
}

track_script {
chk_nginx # nginx存活状态检测脚本
}
}

主机2

global_defs {
router_id nginx_02
}

vrrp_script chk_nginx {
script "/etc/keepalived/nginx_check.sh"
interval 2
weight -20
}

vrrp_instance VI_1 {
state BACKUP
interface enp0s3
virtual_router_id 51
mcast_src_ip 192.168.1.202
priority 90
advert_int 1
nopreempt
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.1.210
}
track_script {
chk_nginx
}
}

和非抢占模式的配置相比,只改了两个地方:
1> 在vrrp_instance块下两个节点各增加了nopreempt指令,表示不争抢vip
2> 节点的state都为BACKUP
两个keepalived节点都启动后,默认都是BACKUP状态,双方在发送组播信息后,会根据优先级来选举一个MASTER出来。由于两者都配置了nopreempt,所以MASTER从故障中恢复后,不会抢占vip。这样会避免VIP切换可能造成的服务延迟。

我上传到csdn的每一步有截图:https://download.csdn.net/download/gaoruiinga/11319190

你可能感兴趣的:(2019-03-10:keepalived+nginx+tomcat实现双主热备动静分离负载均衡)