1、双主模型ipvs(keepalived) 高可用集群
2、双主模型nginx(keepalived)高可用集群
1、准备五个虚拟机(centos7)；
note1、note2、note3、note4、client
2、配置静态ip地址，分别为；
note1;172.16.253.10
note2;172.16.253.11
note3;172.16.253.13
note4;172.16.253.14
3、将防火墙及selinux关闭；
#systemctl stop firewalld
#systemctl disable firewalld
#systemctl is-enabled firewalld
#setenforce 0
#vim /etc/selinux/config
SELINUX=permissive
#getenforce
4、配置yum源；
5、安装keepalived、ipvsadm、nginx、httpd；
6、同步时间；
vim /etc/chrony
server 172.16.0.1 iburst
systemctl start chronyd.service
chronyc sources---查看同步状态
7、设置/etc/hosts文件，做ip地址解析；
ipvs(keepalived)
1、note1、note2，设置单主模式keepalived
配置/etc/keepalived/keepalived.conf文件；
设置全局配置、vrrp_instance虚拟路由器配置
[root@note1 ~]# cat keepalived.conf.shuangzhu
! Configuration File for keepalived
global_defs {
notification_email {
[email protected]
}
notification_email_from [email protected]
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id note1
vrrp_mcast_group4 224.12.0.18
}
vrrp_instance VI_1 {
state MASTER
interface ens34
virtual_router_id 11
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass yKt4PsOZ
}
virtual_ipaddress {
172.16.253.101/16 dev ens34
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
将note1的此配置文件拷贝到note2上，将配置文件MASTER改为BACKUP(主用改为备用)、将prioirty权重改为95；
[root@note1 ~]#systemctl start keepalived
#ip a l
#mail
[root@note2 ~]#systemctl start keepalived
#ip a l
#mail
将note1主用keepalived停用，查看note2备用keepalived是否升为主用；
2、note1、note2，设置双主模式keepalived；
#yum install keepalived
在/etc/keepalived/目录下创建一个脚本,并调用，生成邮件；
#!/bin/bash
contact='root@localhost'
notify() {
local mailsubject="$(hostname) to be $1, vip floating"
local mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"
echo "$mailbody" | mail -s "$mailsubject" $contact
}
case $1 in
master)
notify master
;;
backup)
notify backup
;;
fault)
notify fault
;;
*)
echo "Usage: $(basename $0) {master|backup|fault}"
exit 1
;;
esac
将原有的配置文件中的vrrp_instance配置段各复制一份，进行修改。如果主备模式三台可以复制三分，并进行设置；
1、将虚拟路由器名称VI_1改为VI_2。
2、设置虚拟路由器id，11改为12。
3、更改身份验证auto_PASS。
4、更改主备设置及权重。
note1配置文件；
vrrp_instance VI_1 {
state MASTER
interface ens34
virtual_router_id 11
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass yKt4PsOZ
}
virtual_ipaddress {
172.16.253.101/16 dev ens34
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
vrrp_instance VI_2 {
state BACKUP
interface ens34
virtual_router_id 12
priority 95
advert_int 1
authentication {
auth_type PASS
auth_pass yKt5PsOZ
}
virtual_ipaddress {
172.16.253.102/16 dev ens34
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
#双主模式；有两个虚拟路由器VI_1和VI_2，两个虚拟路由器有各自的ip地址；
*note1; VI_1 为MASTER，VI_2为BACKUP
*note2; VI_2 为MASTER，VI_1为BACKUP
测试；tcp -nn -l ens34 host 244.12.0.18
systemctl stop keepalived
tcp -nn -l ens34 host 244.12.0.18
3、note3、note4，设置网页index.html文件；
#yum install nginx
1、将默认网页文件内容删除重新编辑；
note3；vim /usr/share/nginx/html/index.html
RS1
note4；vim /usr.share/nginx/html/index.html
RS2
4、note3、note4；将虚拟路由器ip添加到回环网卡lo上；
note3;
编辑一个脚本并执行；
vim setrs.sh
#!/bin/bash
vip=172.16.253.101
mask=255.255.255.255
iface="lo:0"
case $1 in
start)
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig $iface $vip netmask $mask broadcast $vip up
route add -host $vip dev $iface
;;
stop)
ifconfig $iface down
echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
;;
*)
echo "Usage: $(basename $0) start|stop"
exit 1
;;
esac
检查语法；bash -n setrs.sh
检查语法并执行；bash -x setrs.sh start
执行成功后将虚拟路由器ip添加到lo网卡上;
将脚本ip改为172.16.253.102，在lo网卡上添加第二个虚拟路由器ip;
note4;
根据note3步骤将虚拟路由器ip添加到note4后端服务器lo网卡上;
5、note1、note2，设置ipvs相关的vs、rs---地址；
1、编辑/etc/keepalived/keepalived.conf文件，设置virtual_server 的vs及rs的地址；
virtual_server 172.16.253.102 80 {
delay_loop 6
lb_algo rr
lb_kind DR
nat_mask 255.255.0.0
protocol TCP
sorry_server 127.0.0.1 80
real_server 172.16.253.13 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 172.16.253.14 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
note1;在各自的虚拟路由器段，如VI_1、VI_2的vrrp配置段下添加virtual_server段，注意虚拟服务器地址，分别为；172.16.253.101,172.16.253.102.
note2;根据note1配置。
6、note1,note2;在配置文件/etc/keepalived/keepaived.conf，virtual-server板块中 sorry_server错误界面选项；
virtual_server 172.16.253.101 80 {
delay_loop 6
lb_algo rr
lb_kind DR
nat_mask 255.255.0.0
protocol TCP
sorry_server 127.0.0.1 80
real_server 172.16.253.13 80 {
weight 1
HTTP_GET {
安装ngnix或httpd,将默认网页文件改为自己所需要的内容；
[root@note1 ~]# vim /usr/share/nginx/html/index.html
sorry from Director1
note1,note2都设置此项；
7、client测试高可用性；
访问ip地址；172.16.253.101或172.16.253.101
[root@xjcentos7 ~]# for i in {1..10};do curl http://172.16.253.101;done
RS1
RS2
RS1
RS2
RS1
RS2
RS1
RS2
RS1
RS2
当将两台后端主机RS服务器停掉，在访问172.16.253.101或172.16.253.102；
[root@xjcentos7 ~]# for i in {1..10};do curl http://172.16.253.101;done
sorry from Director1
sorry from Director1
sorry from Director1
sorry from Director1
sorry from Director1
sorry from Director1
sorry from Director1
sorry from Director1
sorry from Director1
sorry from Director1
[root@xjcentos7 ~]# for i in {1..10};do curl http://172.16.253.102;done
sorry from Director2
sorry from Director2
sorry from Director2
sorry from Director2
sorry from Director2
sorry from Director2
sorry from Director2
sorry from Director2
sorry from Director2
sorry from Director2
nginx（keepalived）
1、实现单主模型下nginx（keepalived）调用后端RS服务器；
1、启用nginx调用功能，编写配置文件/etc/nginx/nginx.conf；
note1,note2设置配置文件；
upstream websrvs {
server 172.16.253.13:80
server 172.16.253.14:80
}
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
root /usr/share/nginx/html;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
location / {
proxy_pass http://websrvs;
}
设置完成后，访问172.16.253.10测试一下；
[root@note1 ~]# curl http://172.16.253.10
RS1
[root@note1 ~]# curl http://172.16.253.10
RS2
2、在/etc/keepalived/keepalived.conf文件中调用命令段，实现降权、升权的功能，主备切换的功能；
设置调用脚本或程需命令选项；
查看示例；
[root@note1 ~]# grep -i vrrp_script /usr/share/doc/keepalived-1.2.13/samples/*
/usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.vrrp.localcheck:vrrp_script chk_sshd {
/usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.vrrp.localcheck:vrrp_script chk_haproxy {
/usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.vrrp.localcheck:vrrp_script chk_http_port {
/usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.vrrp.localcheck:vrrp_script chk_https_port {
/usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.vrrp.localcheck:vrrp_script chk_smtp_port {
[root@note1 ~]# cat /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.vrrp.localcheck
! Configuration File for keepalived
vrrp_script chk_sshd {
script "killall -0 sshd" # cheaper than pidof
interval 2 # check every 2 seconds
weight -4 # default prio: -4 if KO
fall 2 # require 2 failures for KO
rise 2 # require 2 successes for OK
}
设置手动制造故障及设置调用nginx脚本；
global_defs {
notification_email {
[email protected]
}
notification_email_from [email protected]
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id note1
vrrp_mcast_group4 224.12.0.18
}
vrrp_script chk_down {
script "[[ -f /etc/keepalived/down ]]" && exit 1 || exit 0"
interval 1
weight -10
fall 2
rise 2
}
vrrp_script chk_nginx {
skript "killall -0 nginx"
interval 2
weight -10
fall 2
rise 2
}
vrrp_instance VI_1 {
state MASTER
interface ens34
virtual_router_id 11
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass yKt4PsOZ
}
virtual_ipaddress {
172.16.253.101/16 dev ens34
}
track_script{
chk_nginx
chk_down
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
3、将配置文件复制到note2上并进行修改；
将MASTER改为BACKUP,单主模式配置完成。
进行访问测试；
[root@xjcentos7 ~]# for i in {1..10};do curl http://172.16.253.101;done
RS1
RS2
RS1
RS2
[root@note1 ~]# cd /etc/keepalived/
[root@note1 keepalived]# touch down
[root@note1 ~]# tcpdump -i ens34 -nn host 224.12.0.18
11:01:41.580252 IP 172.16.253.10 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 100, authtype simple, intvl 1s, length 20
11:01:41.580881 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 95, authtype simple, intvl 1s, length 20
11:01:41.581866 IP 172.16.253.10 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 100, authtype simple, intvl 1s, length 20
11:01:42.583792 IP 172.16.253.10 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 100, authtype simple, intvl 1s, length 20
11:01:43.587375 IP 172.16.253.10 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 100, authtype
2、实现双主模式下nginx ( keepalived ) 调用后端RS服务器；
1、note1; 在配置文件/etc/keepalived/keepalived.conf中, 添加复制一份
virtual_server区域内容，并进行修改。MASTER,BACKUP、权、验证码、虚拟路由器ip(vip)
vrrp_instance VI_2 {
state BACKUP
interface ens34
virtual_router_id 11
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass yKt5PsOZ
}
virtual_ipaddress {
172.16.253.102/16 dev ens34
}
track_script{
chk_nginx
chk_down
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
note2；按note1的步骤配置、/etc/keepalived/keepalived.conf文件；
2、cilent进行测试，也可在note1和note2端使用touch down来进行检测；
[root@xjcentos7 ~]# for i in {1..10};do curl http://172.16.253.101;done
RS1
RS2
RS1
RS2
[root@xjcentos7 ~]# for i in {1..10};do curl http://172.16.253.102;done
RS1
RS2
RS1
RS2
[root@note1 keepalived]# touch down
[root@note1 ~]# tcpdump -i ens34 -nn host 224.12.0.18
11:21:53.974338 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 12, prio 100, authtype simple, intvl 1s, length 20
11:21:53.976207 IP 172.16.253.10 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 100, authtype simple, intvl 1s, length 20
11:21:54.976431 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 12, prio 100, authtype simple, intvl 1s, length 20
11:21:54.978223 IP 172.16.253.10 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 100, authtype simple, intvl 1s, length 20
11:21:55.978463 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 12, prio 100, authtype simple, intvl 1s, length 20
11:21:55.980380 IP 172.16.253.10 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 100, authtype simple, intvl 1s, length 20
11:21:56.980566 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 12, prio 100, authtype simple, intvl 1s, length 20
11:21:56.982511 IP 172.16.253.10 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 90, authtype simple, intvl 1s, length 20
11:21:56.986256 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 95, authtype simple, intvl 1s, length 20
11:21:56.987810 IP 172.16.253.10 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 90, authtype simple, intvl 1s, length 20
11:21:56.995422 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 95, authtype simple, intvl 1s, length 20
11:21:57.982627 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 12, prio 100, authtype simple, intvl 1s, length 20
11:21:57.998655 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 95, authtype simple, intvl 1s, length 20
11:21:58.984764 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 12, prio 100, authtype simple, intvl 1s, length 20
11:21:59.000745 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 11, prio 95, authtype simple, intvl 1s, length 20
11:21:59.986896 IP 172.16.253.11 > 224.12.0.18: VRRPv2, Advertisement, vrid 12, prio 100, authtype simple, intvl 1s, length 20
3、测试故障修复抢占模式；
note1；/etc/keepalived/keepalied.conf目录中的调用了notify脚本文件，在脚本文件中添加一项内容；systemctl start nginx
在notify backup下添加一行；systemctl start nginx