nginx+keepalived 部署web高可用

1.准备两台在同一个局域网的Linux服务器

主机器ip：12.16.119.213

备用机器ip：12.16.119.214

2.两台服务器先都搭建好nginx服务器

1.两台服务器都安装keepalived

yum install keepalived -y  # 先安装好nginx后再安装 keepalived

keepalived 配置文件路径：/etc/keepalived/keepalived.conf

cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak

2.两台服务器都修改keepalived配置

keepalived配置文件路劲：/etc/keepalived/keepalived.conf

主机器:12.16.119.213

! Configuration File for keepalived

global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   # vrrp_strict 
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

#VIP1
vrrp_instance VI_1 {
    state MASTER #
    interface ens160 #
    virtual_router_id 75 #
    priority 100  #
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress { 
        12.16.119.200 #
        12.16.119.20 #
		12.16.110.200 #
    }
}

修改前可以备份默认的配置：cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak

备用机器:12.16.119.214

! Configuration File for keepalived

global_defs {
   notification_email {
     acassen@firewall.loc # 接收邮件的邮箱
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc # 发邮件的邮箱
   smtp_server 192.168.200.1 # 发邮件的邮箱地址
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   # vrrp_strict # 存在导致虚拟ip ping 不通
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

#VIP1
vrrp_instance VI_1 {  # 实例名称主备保持一致
    state BACKUP # 主备配置为 MASTER ---- BACKUP
    interface ens160 # ifconfig查看自己的网卡名称修改为它
    virtual_router_id 75 # vrid 路由标识符，主备保持一致 0 - 255， 默认51
    priority 90 # 优先级值越大越高 MASTER 100----BACKUP 90 
    advert_int 1
    authentication { # 认证，默认即可，主备保持一致
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress { # vip，主备一致，可以有多个虚拟vip
        12.16.119.200 # 可以
        12.16.119.20 # 可以
		12.16.110.200 # 可以
    }
}

其实还可以继续添加备用机器:12.16.119.xxx

! Configuration File for keepalived

global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   # vrrp_strict 
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

#VIP1
vrrp_instance VI_1 {
    state BACKUP #
    interface ens160 #
    virtual_router_id 75 #
    priority 80  #
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress { 
        12.16.119.200 #
        12.16.119.20 #
		12.16.110.200 #
    }
}

如果 ifconfig 指令不能用可以执行 yum install net-tools 安装

3.两台服务器都启动keepalived

此外主机器上的防火墙和selinux必须关闭，否则keepalived无法拉起nginx。

临时关闭防火墙：systemctl stop firewalld

临时关闭selinux：setenforce 0

1.先启动两台服务器的nginx

/usr/local/webserver/nginx/sbin/nginx  # 启动nginx的命令

2.再启动两台服务器的 keepalived

service keepalived start

[root@fxjc log]#  service keepalived start
Redirecting to /bin/systemctl start keepalived.service
[root@fxjc log]#

2.两台服务器启动后，用 service keepalived status 看到 keepalived 的状态是正常的。

service keepalived status

[root@fxjc log]#  service keepalived status
Redirecting to /bin/systemctl status keepalived.service
● keepalived.service - LVS and VRRP High Availability Monitor
   Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
   Active: active (running) since Thu 2020-05-07 16:53:58 CST; 18s ago
  Process: 887 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 888 (keepalived)
   CGroup: /system.slice/keepalived.service
           ├─888 /usr/sbin/keepalived -D
           ├─889 /usr/sbin/keepalived -D
           └─890 /usr/sbin/keepalived -D

May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.110.200
May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.119.20
May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.119.200
May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.110.200
May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.119.20
May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.119.200
May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.110.200
May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.119.20
May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.119.200
May 07 16:54:05 fxjc Keepalived_vrrp[890]: Sending gratuitous ARP on ens160 for 12.16.110.200
[root@fxjc log]# 

3.查看启动的情况

ip addr

[root@fxjc log]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:50:56:8b:71:fa brd ff:ff:ff:ff:ff:ff
    inet 12.16.119.213/16 brd 12.16.255.255 scope global noprefixroute ens160
       valid_lft forever preferred_lft forever
       
    inet 12.16.119.20/32 scope global ens160
       valid_lft forever preferred_lft forever
    inet 12.16.119.200/32 scope global ens160
       valid_lft forever preferred_lft forever
    inet 12.16.110.200/32 scope global ens160
       valid_lft forever preferred_lft forever
       
    inet6 fe80::250:56ff:fe8b:71fa/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
[root@fxjc log]# 
[root@fxjc log]# 

4.启动失败查看keepalived的日志文件

keepalived 的日志文件记录在/var/log/messages文件内

5.停止服务

[root@fxjc log]#  service keepalived stop
Redirecting to /bin/systemctl stop keepalived.service
[root@fxjc log]#

4.测试高可用

浏览器输入下面的地址都可以访问

注意nginx里面提前准备了pro项目，项目中只有一个index.html的文件

http://12.16.119.20/pro/index.html
http://12.16.110.200/pro/index.html
http://12.16.119.200/pro/index.html

5.仅停止主机器的keepalived

1.在主机器中执行

[root@fxjc log]#  service keepalived stop
[root@fxjc log]#

2.在主机中查看虚拟ip （没有）

[root@fxjc log]#  service keepalived stop
Redirecting to /bin/systemctl stop keepalived.service
[root@fxjc log]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:50:56:8b:71:fa brd ff:ff:ff:ff:ff:ff
    inet 12.16.119.213/16 brd 12.16.255.255 scope global noprefixroute ens160
       valid_lft forever preferred_lft forever
    inet6 fe80::250:56ff:fe8b:71fa/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
[root@fxjc log]# 

3.查看备用机器的虚拟ip （有）

[root@ocr bin]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:50:56:8b:6c:ba brd ff:ff:ff:ff:ff:ff
    inet 12.16.119.214/16 brd 12.16.255.255 scope global noprefixroute ens160
       valid_lft forever preferred_lft forever

    inet 12.16.119.20/32 scope global ens160
       valid_lft forever preferred_lft forever
    inet 12.16.119.200/32 scope global ens160
       valid_lft forever preferred_lft forever
    inet 12.16.110.200/32 scope global ens160
       valid_lft forever preferred_lft forever

    inet6 fe80::924c:47f0:3651:2820/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:f4:98:50:1d brd ff:ff:ff:ff:ff:ff
    inet 192.168.128.1/24 brd 192.168.128.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:f4ff:fe98:501d/64 scope link 
       valid_lft forever preferred_lft forever
[root@ocr bin]# 

4.刷新网页发现，展示的内容也是备用机器中的网站

5.如果主机器重新启动keepalived, 那么虚拟ip会自动指向主机器

6.仅停止主机器的nginx

1.在主机器中执行

[root@fxjc log]# /usr/local/webserver/nginx/sbin/nginx -s stop

2.在主机中查看虚拟ip （有）

[root@fxjc log]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:50:56:8b:71:fa brd ff:ff:ff:ff:ff:ff
    inet 12.16.119.213/16 brd 12.16.255.255 scope global noprefixroute ens160
       valid_lft forever preferred_lft forever
    inet 12.16.119.20/32 scope global ens160
       valid_lft forever preferred_lft forever
    inet 12.16.119.200/32 scope global ens160
       valid_lft forever preferred_lft forever
    inet 12.16.110.200/32 scope global ens160
       valid_lft forever preferred_lft forever
    inet6 fe80::250:56ff:fe8b:71fa/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
[root@fxjc log]# 

3.查看备用机器的虚拟ip （没有生成）

[root@ocr bin]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:50:56:8b:6c:ba brd ff:ff:ff:ff:ff:ff
    inet 12.16.119.214/16 brd 12.16.255.255 scope global noprefixroute ens160
       valid_lft forever preferred_lft forever
    inet6 fe80::924c:47f0:3651:2820/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:f4:98:50:1d brd ff:ff:ff:ff:ff:ff
    inet 192.168.128.1/24 brd 192.168.128.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:f4ff:fe98:501d/64 scope link 
       valid_lft forever preferred_lft forever

4.刷新网页发现，网页无法访问了

5.如果主机器重新启动nginx，网页回复正常

建议nginx和keepalived同生死

7.监听nginx服务器的进程

主机器：建议 nginx 和 keepalived 同生死，如果监听到nginx挂了，那么也要把keepalived停止

1.主机器的的 keepalived.conf

! Configuration File for keepalived

global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   # vrrp_strict 
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_script chk_nginx
{
	script "/etc/keepalived/nginx_check.sh"
    #每1秒检测一次nginx的运行状态
	interval 1
    #失败一次，将自己的优先级调整为-2
	weigth -2
}

#VIP1
vrrp_instance VI_1 {
    state MASTER #
    interface ens160 #
    virtual_router_id 75 #
    priority 100  #
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress { 
        12.16.119.200 #
        12.16.119.20 #
		12.16.110.200 #
    }

    #nginx存活状态检测脚本
	track_script 
	{
		chk_nginx
	}

}

2.其中调用的 /etc/keepalived/nginx_check.sh 脚本内容为：

注意授权： chmod a+x nginx_check.sh 给这个脚本设置所有用户执行权限，不然这个脚本会调用失败

#!/usr/bin/bash
# 下面获取指令的返回值，不能用“” 或者 ‘’ 号
A=`ps -C nginx -no-header |wc -l`
# 下面[] 前后必须要给个空格 -eq 是等于
if [ $A -eq 1 ]
then
  echo 'nginx has stop, stop keepalived too'
  pkill keepalived
else
  echo 'nginx run'
fi

3.测试

启动主机器的nginx 和 keepalived

 /usr/local/webserver/nginx/sbin/nginx
 service keepalived start

停止nginx服务器

 /usr/local/webserver/nginx/sbin/nginx -s stop

查看keepalived，发现keepalived也停止了

service keepalived status

7.发生故障邮箱通知

1.安装依赖linux发送邮件的依赖包

 yum -y install perl-CPAN  # 按钮perl命令工具
> sudo perl -MCPAN -e shell # 进入capn
capn > install Net::SMTP_auth # 安装Net::SMTP_auth

2.在主机器下新建 /etc/keepalived/send_mail.sh 脚本

注意授权： chmod a+x nginx_check.sh 给这个脚本设置所有用户执行权限，不然这个脚本会调用失败

#!/usr/bin/perl -w
use Net::SMTP_auth;
use strict;
#smtp服务器
my $mailhost = 'smtp.qq.com';
#发送邮件的邮箱
my $mailfrom = '[email protected]';
#接收邮件的邮箱
my @mailto   = ('[email protected]');
#邮件主题
my $subject  = 'keepalived up on backup';
#邮件正文
my $text = "正文\n nginx-1服务器宕机！！nginx-2变为master！！！";
#发送邮件的用户名
my $user   = '[email protected]';
#发送邮件的邮箱授权的密码
my $passwd = 'rxrgeedpitcbhxeb';
&SendMail();
##############################  
# Send notice mail  
##############################  
sub SendMail() {
    my $smtp = Net::SMTP_auth->new( $mailhost, Timeout => 120, Debug => 1 )
      or die "Error.\n";
    $smtp->auth( 'LOGIN', $user, $passwd );
    foreach my $mailto (@mailto) {
        $smtp->mail($mailfrom);
        $smtp->to($mailto);
        $smtp->data();
        $smtp->datasend("To: $mailto\n");
        $smtp->datasend("From:$mailfrom\n");
        $smtp->datasend("Subject: $subject\n");
        $smtp->datasend("\n");
        $smtp->datasend("$text\n\n");
        $smtp->dataend();
    }
    $smtp->quit;
}

测试发送邮件的脚本，如果能发送邮件代表脚本没问题

[root@fxjc keepalived]# cd /etc/keepalived

[root@fxjc keepalived]# ./send_mail.sh

修改主机器的 keepalived.conf，如果主机 keepalived 停止了就发执行发起邮箱的脚本

! Configuration File for keepalived

global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   # vrrp_strict 
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}
vrrp_script chk_nginx
{
	script "/etc/keepalived/nginx_check.sh"
	interval 1
	weigth -2
}
#VIP1
vrrp_instance VI_1 {
    state MASTER #
    interface ens160 #
    virtual_router_id 75 #
    priority 100  #
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress { 
        12.16.119.200 #
        12.16.119.20 #
		12.16.110.200 #
    }

    # 当该服务器挂掉以后会执行该脚本
	notify_master /etc/keepalived/send_mail.sh

	track_script 
	{
		chk_nginx
	}
}

测试

启动主机器的nginx 和 keepalived

 /usr/local/webserver/nginx/sbin/nginx
 service keepalived start

停止 keepalived 服务器, 这个时候就会自动发送邮箱提醒该主机器发生了故障

service keepalived stop

参考文章：

https://www.cnblogs.com/zhangxingeng/p/10721083.html#auto_id_5

https://www.cnblogs.com/mzhaox/p/11181943.html

https://blog.csdn.net/weixin_33834137/article/details/92422554

https://blog.csdn.net/zhou16333/article/details/98179341

https://segmentfault.com/q/1010000009646984

https://blog.csdn.net/wzyzzu/article/details/50787042

https://www.cnblogs.com/John-2011/p/7838794.html