LVS+keepalive+nginx集群环境搭建
LVS+keepalive+nginx集群环境搭建
安装nginx
我们在106、107两台机器上安装nginx
nginx安装(使用yum安装)步骤：
-
新增nginx官网的yum安装源
sudo rpm -Uvh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm -
安装nginx
yum install -y nginx -
启动Nginx并设置开机自动运行
systemctl start nginx.service systemctl enable nginx.service -
安装完成成，在测试之前需要确认是否开着防火墙，如果开着防火墙则关闭防火墙或者开放80端口
-
方式一：关闭防火墙
#1、先检查防火墙是否开着 systemctl status firewalld.service #2、如果开着则先 停掉防火墙 systemctl stop firewalld.service #3、将防火墙服务禁止掉 systemctl disable firewalld.service -
方式二：开放端口
firewall-cmd --zone=public --add-port=80/tcp --permanent firewall-cmd --reload
-
安装keepalived与配置
-
安装keepalived（使用yum安装）
yum install -y curl gcc openssl-devel libnl3-devel net-snmp-devel yum install -y keepalived -
启动keepalived并设置成开启启动
#启动keepalived systemctl start keepalived #加入开机启动keepalived systemctl enable keepalived #重新启动keepalived systemctl restart keepalived #查看keepalived状态 systemctl status keepalived -
配置keepalived
#到keepalived安装目录并修改配置文件 cd /etc/keepalived #找到 keepalived.conf配置文件并修改配置文件，将配置修改成如下：keepalived.conf
! Configuration File for keepalived global_defs { #指定router_id的名字 router_id LVS_104 } vrrp_instance VI_1 { #指定当前节点为MASTER还BACKUP state MASTER #指定网卡的名称，大家根据自己的网卡名称来设置 interface enp0s8 # 指定虚拟路由ID名称 virtual_router_id 104 # 指定当前节点的优先级 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } # 配置虚拟IP virtual_ipaddress { 192.168.56.204 } } #配置虚拟IP与真实IP的转发规则 virtual_server 192.168.56.204 80 { #健康检查时间，单位：秒 delay_loop 6 #配置负载均衡算法，默认是轮询 lb_algo rr #配置LVS的模式 NAT/TUN/DR lb_kind DR #设置持久化时间，默认50秒 persistence_timeout 50 #默认的协议 protocol TCP #配置真实服务器的地址IP real_server 192.168.56.107 80 { #设置每台机器的权重配比 weight 1 #设置健康检查 TCP_CHECK { #连接端口 80 connect_port 80 #设置检查的超时时间 connect_timeout 2 #重试的次数 5次 nb_get_retry 5 #间隔时间 3S delay_before_retry 3 } } real_server 192.168.56.106 80 { #设置每台机器的权重配比 weight 1 #设置健康检查 TCP_CHECK { #连接端口 80 connect_port 80 #设置检查的超时时间 connect_timeout 2 #重试的次数 5次 nb_get_retry 5 #间隔时间 3S delay_before_retry 3 } } } -
安装ipvsadm工具（ipvsadm是linux下的LVS虚拟服务器的管理工具），安装该工具方便查看LVS相关配置
yum install ipvsadm -
利用ipvsadm工具来检查我们上面的配置是否正确
[root@localhost keepalived]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.56.204:80 rr persistent 50 -> 192.168.56.106:80 Route 1 2 0 -> 192.168.56.107:80 Route 1 0 0通过以上执行结果我们可以看得到我们刚配置的内容已经生效了
配置Nginx服务器
-
配置虚拟ip
cd /etc/sysconfig/network-scripts cp ifcfg-lo ifcfg-lo:1 vi ifcfg-lo:1将ifcfg-lo:1改成：
DEVICE=lo IPADDR=192.168.56.204 NETMASK=255.255.255.255 NETWORK=127.0.0.0 # If you're having problems with gated making 127.0.0.0/8 a martian, # you can change this to something else (255.255.255.255, for example) BROADCAST=127.255.255.255 ONBOOT=yes NAME=loopback重启网络服务：
service network restart -
修改/etc/sysctl.conf配置文件，在尾部加入以下相关配置：
net.ipv4.conf.all.arp_ignore = 1 net.ipv4.conf.default.arp_ignore = 1 net.ipv4.conf.lo.arp_ignore = 1 net.ipv4.conf.all.arp_announce = 2 net.ipv4.conf.default.arp_announce = 2 net.ipv4.conf.lo.arp_announce = 2 -
重新刷新sysctl.conf，
sysctl -p，这个时候我们可以看到如下结果：[root@localhost network-scripts]# sysctl -p net.ipv4.conf.all.arp_ignore = 1 net.ipv4.conf.default.arp_ignore = 1 net.ipv4.conf.lo.arp_ignore = 1 net.ipv4.conf.all.arp_announce = 2 net.ipv4.conf.default.arp_announce = 2 net.ipv4.conf.lo.arp_announce = 2 -
主机添加虚拟IP的路由
#添加路由，将192.168.56.204这个虚拟ip请求进来的请求委托与lo这个网卡来处理 route add -host 192.168.56.204 dev lo这个时候我们可以使用
route -n来验证是否添加成功：[root@localhost network-scripts]# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.52.1 0.0.0.0 UG 100 0 0 enp0s3 192.168.52.0 0.0.0.0 255.255.255.0 U 100 0 0 enp0s3 192.168.56.0 0.0.0.0 255.255.255.0 U 101 0 0 enp0s8 192.168.56.204 0.0.0.0 255.255.255.255 UH 0 0 0 lo这个是否我们可以看到已经新增了一条
192.168.56.204的路由 -
将路由配置成永久生效
#我们只需将添加路由的命令添加的/etc/rc.local即可实现服务器重启的时候动态添加路由信息 echo "route add -host 192.168.56.204 dev lo" >>/etc/rc.local
测试
-
正常测试
这样我们就可以通过访问http://192.168.56.204/来进行测试了，为了方便测试我们将nginx欢迎页面加上对应的ip信息，访问结果如下：
-
停掉107的nginx测试
从以上测试结果我们可以看得访问到了107这台的nginx上，我们这个时候将107的nginx停掉
nginx -s stop，然后再访问试下以效果
-
验证keepalived主备切换的效果
目前我们的keepalived的主是104这台，我们可以通过
ip addr查看，通过下面的执行结果我们可以看到其中enp0s8网卡多了一个204的虚拟ip[root@localhost ~]# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 08:00:27:b1:28:14 brd ff:ff:ff:ff:ff:ff inet 192.168.52.5/24 brd 192.168.52.255 scope global noprefixroute dynamic enp0s3 valid_lft 583sec preferred_lft 583sec inet6 fe80::a00:27ff:feb1:2814/64 scope link valid_lft forever preferred_lft forever 3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 08:00:27:5f:62:20 brd ff:ff:ff:ff:ff:ff inet 192.168.56.104/24 brd 192.168.56.255 scope global noprefixroute enp0s8 valid_lft forever preferred_lft forever inet 192.168.56.204/32 scope global enp0s8 valid_lft forever preferred_lft forever inet6 fe80::a00:27ff:fe5f:6220/64 scope link valid_lft forever preferred_lft forever这个时候我们将104的keepalived服务停掉
systemctl stop keepalived，在看下104,105两台服务器的变化是怎么样104机器的
ip addr结果：[root@localhost ~]# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 08:00:27:b1:28:14 brd ff:ff:ff:ff:ff:ff inet 192.168.52.5/24 brd 192.168.52.255 scope global noprefixroute dynamic enp0s3 valid_lft 375sec preferred_lft 375sec inet6 fe80::a00:27ff:feb1:2814/64 scope link valid_lft forever preferred_lft forever 3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 08:00:27:5f:62:20 brd ff:ff:ff:ff:ff:ff inet 192.168.56.104/24 brd 192.168.56.255 scope global noprefixroute enp0s8 valid_lft forever preferred_lft forever inet6 fe80::a00:27ff:fe5f:6220/64 scope link valid_lft forever preferred_lft forever105机器的
ip addr结果：[root@localhost ~]# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 08:00:27:ba:2f:c2 brd ff:ff:ff:ff:ff:ff inet 192.168.52.8/24 brd 192.168.52.255 scope global noprefixroute dynamic enp0s3 valid_lft 348sec preferred_lft 348sec inet6 fe80::a00:27ff:feba:2fc2/64 scope link valid_lft forever preferred_lft forever 3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 08:00:27:a4:7f:f4 brd ff:ff:ff:ff:ff:ff inet 192.168.56.105/24 brd 192.168.56.255 scope global noprefixroute enp0s8 valid_lft forever preferred_lft forever inet 192.168.56.204/32 scope global enp0s8 valid_lft forever preferred_lft forever inet6 fe80::a00:27ff:fea4:7ff4/64 scope link valid_lft forever preferred_lft forever从上面的执行结果我们可以看得到，在mater挂掉的情况下，keepalived会自动的切换到backup上。
如果这个时候我们再把mater(104)启动起来
systemctl start keepalived，我会发现，虚拟ip有会挂回到104上面来：[root@localhost ~]# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 08:00:27:b1:28:14 brd ff:ff:ff:ff:ff:ff inet 192.168.52.5/24 brd 192.168.52.255 scope global noprefixroute dynamic enp0s3 valid_lft 482sec preferred_lft 482sec inet6 fe80::a00:27ff:feb1:2814/64 scope link valid_lft forever preferred_lft forever 3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 08:00:27:5f:62:20 brd ff:ff:ff:ff:ff:ff inet 192.168.56.104/24 brd 192.168.56.255 scope global noprefixroute enp0s8 valid_lft forever preferred_lft forever inet 192.168.56.204/32 scope global enp0s8 valid_lft forever preferred_lft forever inet6 fe80::a00:27ff:fe5f:6220/64 scope link valid_lft forever preferred_lft forever
-