LVS+Keepalived+httpd安装及配置
LVS+Keepalived+httpd安装及配置
目录
1 环境... 1
2 安装LVS软件（分别在LVS-Master和LVS-Backup中）... 1
1.1.查看是否有IPVS模块... 1
1.2.安装IPVS管理工具... 1
3 安装keepalived（分别在LVS-Master和LVS-Backup中）... 1
3.1 安装keepalived并配置... 1
3.2.启动keepalived. 1
4 在LVS-Backup上安装LVS、keepalived并配置... 1
4.1 安装LVS、keepalived. 1
4.2 配置LVS-Backup的keepalived. 1
4.3启动LVS-Backup的keeplived服务，并检查是否生效... 1
5 测试VIP，检查是否能主备切换。... 1
5.1 查看主VIP. 1
5.2 查看备VIP. 1
5.3 日志查看主备切换... 1
6 配置WEB服务器，即real server节点1和2. 1
6.1 配置web2. 1
6.2 配置web4. 1
7 负载均衡以及高可用测试... 1
7.1 负载均衡测试... 1
7.2 高可用测试... 1
8 其他常用操作... 1
8.1 关闭LVS. 1
8.2 系统虚拟IP管理：... 1
8.3 查看LVS的状态... 1
1 环境
操作系统Centos 6.4 X86_64
LVS-Master:192.168.1.19
LVS-Backup:192.168.1.18
VIP:192.168.1.10
RS01:192.168.1.18
RS02:192.168.1.17
2 安装LVS软件（分别在LVS-Master和LVS-Backup中）
1.1.查看是否有IPVS模块
LVS是Linux标准内核的一部分,直接被编译在内核中!
[root@LVS-Master ~]# modprobe -l | grep ipvs #使用modprobe命令查看
1.2.安装IPVS管理工具
[root@LVS-Master ~]# yum install ipvsadm
3 安装keepalived（分别在LVS-Master和LVS-Backup中）
3.1 安装keepalived并配置
[root@LVS-Master ~]# yum install keepalived
[root@LVS-Master ~]# vi/etc/keepalived/keepalived.conf
vi keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
}
notification_email_from [email protected]
smtp_server 10.0.0.1
smtp_connect_timeout30
router_idLVS_7
}
vrrp_instance VI_1 {
state MASTER
interfaceeth0
virtual_router_id 55
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.1.10/24
}
}
virtual_server192.168.1.10 80 {
delay_loop6
lb_algowrr
lb_kindDR
nat_mask255.255.255.0
persistence_timeout300
protocolTCP
# ipvsadm -A -t192.168.1.10:80 -s wrr -p 20
real_server 192.168.1.19 80 {
weight1
TCP_CHECK{
connect_timeout8
nb_get_retry3
delay_before_retry3
connect_port80
}
}
real_server 192.168.1.18 80 {
weight1
TCP_CHECK{
connect_timeout8
nb_get_retry3
delay_before_retry3
connect_port80
}
}
}
#ipvsadm -a -t 192.168.1.10:80 -r 192.168.1.18:80 -g -w 1
#ipvsadm -a -t 192.168.1.10:80 -r 192.168.1.19:80 -g -w 1
3.2.启动keepalived
设置keepalived服务自启动，检查是否生效
[root@LVS-Master ~]# /etc/init.d/keepalived start
Starting keepalived: [ OK ]
[root@LVS-Master ~]# chkconfig keepalived on
[root@LVS-Master keepalived]# ipvsadm -L -n #查看ipvs规则是否生效
IPVirtual Server version 1.2.1 (size=4096)
ProtLocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.2.33:80 wlc persistent 50
-> 192.168.2.29:80 Route 100 0 0
-> 192.168.2.30:80 Route 100 0 0
4 在LVS-Backup上安装LVS、keepalived并配置
4.1 安装LVS、keepalived
[root@LVS-Backup ~]# yum install ipvsadm
[root@LVS-Backup ~]# yum install keepalived
4.2 配置LVS-Backup的keepalived
先从LVS－Master把修改过的keepalived配置文件复制过来。
[root@LVS-Master keepalived]# scp [email protected]:/etc/keepalived/
[root@LVS-Backup ~]# vi/etc/keepalived/keepalived.conf
只修改state为BACKUP和priority为90，MASTER为主机服务器，BACKUP为备用服务器。
state BACKUP
priority 90
vi keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
}
notification_email_from [email protected]
smtp_server 10.0.0.1
smtp_connect_timeout 30
router_idLVS_7
}
vrrp_instanceVI_1 {
state BACKUP
interfaceeth0
virtual_router_id 55
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.1.10/24
}
}
virtual_server192.168.1.10 80 {
delay_loop6
lb_algowrr
lb_kindDR
nat_mask255.255.255.0
persistence_timeout300
protocolTCP
#ipvsadm -A -t 192.168.1.10:80 -s wrr -p 20
real_server 192.168.1.18 80 {
weight1
TCP_CHECK{
connect_timeout8
nb_get_retry3
delay_before_retry3
connect_port80
}
}
real_server 192.168.1.17 80 {
weight1
TCP_CHECK{
connect_timeout8
nb_get_retry3
delay_before_retry3
connect_port80
}
}
}
#ipvsadm -a -t 192.168.1.10:80 -r 192.168.1.18:80-g -w 1
#ipvsadm -a -t 192.168.1.10:80 -r 192.168.1.17:80-g -w 1
4.3启动LVS-Backup的keeplived服务，并检查是否生效
[root@LVS-Backup ~]# /etc/init.d/keepalived start
[root@LVS-Backup ~]# chkconfig keepalived on
[root@LVS-Backup keepalived]# ipvsadm -L -n #查看ipvs规则是否生效
IPVirtual Server version 1.2.1 (size=4096)
ProtLocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.2.33:80 wlc persistent 50
-> 192.168.2.29:80 Route 100 0 0
-> 192.168.2.30:80 Route 100 0 0
5 测试VIP，检查是否能主备切换。
5.1 查看主VIP
启动keeplived后，先用ip addr显示主备服务器的VIP情况，当主备服务器同时启用keepalived时，只有主服务器拥有VIP地址，备服务器没有。
[root@LVS-Master~]# ip addr
1: lo:
link/loopback 00:00:00:00:00:00 brd00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:
link/ether 00:50:56:a6:00:13 brdff:ff:ff:ff:ff:ff
inet 192.168.2.32/24 brd 192.168.2.255scope global eth0
inet 192.168.2.33/32 scope global eth0 #可以看到主服务器拥有192.168.2.33这个VIP地址
inet6 fe80::250:56ff:fea6:13/64 scope link
valid_lft forever preferred_lft forever
[root@LVS-Backup~]# ip addr
1: lo:
link/loopback 00:00:00:00:00:00 brd00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:
link/ether 08:00:27:77:d3:82 brdff:ff:ff:ff:ff:ff
inet 192.168.2.31/24 brd 192.168.2.255scope global eth0 #可以到看备服务器没有192.168.2.33这个VIP地址
inet6 fe80::a00:27ff:fe77:d382/64 scopelink
valid_lft forever preferred_lft forever
5.2 查看备VIP
当停止主服务器的keepalived服务，再查看下主备服务器VIP地址，发现VIP地址已从主服务器转移到了备服务器；
[root@LVS-Master ~]# service keepalived stop
Stopping keepalived: [ OK ]
[root@LVS-Master keepalived]# ip addr
[root@mobankeepalived]# /etc/init.d/keepalived stop
Stopping keepalived: [ OK ]
[root@mobankeepalived]# ip add
1: lo:
link/loopback 00:00:00:00:00:00 brd00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:
link/ether 00:0c:29:48:9b:89 brdff:ff:ff:ff:ff:ff
inet 192.168.1.19/24 brd 192.168.1.255scope global eth0 #主服务器keepalived停止后，VIP地址将移除
inet6 fe80::20c:29ff:fe48:9b89/64 scopelink
valid_lft forever preferred_lft forever
[root@mobankeepalived]#
[root@LVS-Backup ~]# ip addr
[root@rs01keepalived]# ip add
1: lo:
link/loopback 00:00:00:00:00:00 brd00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:
link/ether 00:0c:29:7b:a7:5e brdff:ff:ff:ff:ff:ff
inet 192.168.1.18/24 brd 192.168.1.255scope global eth0
inet 192.168.1.10/24 scope global secondaryeth0 #可以看到VIP地址已转移到备服务器，继续提供服务
inet6 fe80::20c:29ff:fe7b:a75e/64 scopelink
valid_lft forever preferred_lft forever
[root@rs01keepalived]#
5.3 日志查看主备切换
1) 当主服务器停止keepalived后，主备服务器相关日志
[root@LVS-Master~]# service keepalived stop
Stopping keepalived: [ OK ]
主服务器日志
Sep 2 17:39:13 LVS-Master kernel: IPVS:__ip_vs_del_service: enter
Sep 2 17:39:13 LVS-Master Keepalived[13192]:Stopping Keepalived v1.2.7 (02/21,2013) #主服务器停掉后
Sep 2 17:39:13 LVS-Master Keepalived_vrrp[13195]:VRRP_Instance(VI_1) sending 0 priority
Sep 2 17:39:13 LVS-Master Keepalived_vrrp[13195]:VRRP_Instance(VI_1) removing protocol VIPs.
备服务器日志
Sep 2 17:39:14 LVS-Backup Keepalived_vrrp[979]:VRRP_Instance(VI_1) Transition to MASTER STATE
Sep 2 17:39:15 LVS-Backup Keepalived_vrrp[979]:VRRP_Instance(VI_1) Entering MASTER STATE #备转为MASTER，并设置VIP地址为192.168.2.33
Sep 2 17:39:15 LVS-Backup Keepalived_vrrp[979]:VRRP_Instance(VI_1) setting protocol VIPs.
Sep 2 17:39:15 LVS-Backup Keepalived_vrrp[979]:VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.2.33
Sep 2 17:39:15 LVS-BackupKeepalived_healthcheckers[978]: Netlink reflector reports IP 192.168.2.33 added
Sep 2 17:39:20 LVS-Backup Keepalived_vrrp[979]:VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.2.33
2) 当主服务器重启keepalived后，主备服务器相关日志
[root@LVS-Master~]# service keepalived start
Starting keepalived: [ OK ]
主服务器日志
Sep 2 17:41:28 LVS-Master Keepalived_vrrp[13231]:VRRP_Instance(VI_1) Transition to MASTER STATE
Sep 2 17:41:29 LVS-Master Keepalived_vrrp[13231]:VRRP_Instance(VI_1) Entering MASTER STATE #主服务器重启后，从新接手MASTER角色，并拿回VIP地址192.168.2.33
Sep 2 17:41:29 LVS-Master Keepalived_vrrp[13231]:VRRP_Instance(VI_1) setting protocol VIPs.
Sep 2 17:41:29 LVS-MasterKeepalived_healthcheckers[13230]: Netlink reflector reports IP 192.168.2.33added
Sep 2 17:41:29 LVS-Master Keepalived_vrrp[13231]:VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.2.33
备服务器日志
Sep 2 17:41:28 LVS-Backup Keepalived_vrrp[979]:VRRP_Instance(VI_1) Received higher prio advert
Sep 2 17:41:28 LVS-Backup Keepalived_vrrp[979]:VRRP_Instance(VI_1) Entering BACKUP STATE #备服务器从MASTER又变回了BACKUP，并移除VIP地址192.168.2.33
Sep 2 17:41:28 LVS-Backup Keepalived_vrrp[979]:VRRP_Instance(VI_1) removing protocol VIPs.
Sep 2 17:41:28 LVS-BackupKeepalived_healthcheckers[978]: Netlink reflector reports IP 192.168.2.33removed
6 配置WEB服务器，即realserver节点1和2
由于采用的是DR方式调度，Real_Server会以LVS的VIP来直接回复Client，所以需要在Real_Server的lo上开启LVS的VIP来与Client建立通信
6.1 配置web2
[root@web2~]# vi /etc/init.d/lvsrs ＃编辑一个lvsrs脚本，并放到/etc/init.d/下
#!/bin/bash
#description: start Real Server
VIP=192.168.1.10
./etc/rc.d/init.d/functions
case"$1" in
start)
echo " Start LVS of Real Server"
/sbin/ifconfig lo:0 $VIP broadcast $VIPnetmask 255.255.255.255 up
/sbin/route add -host $VIP dev lo:0
echo "1">/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2">/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1">/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2">/proc/sys/net/ipv4/conf/all/arp_announce
;;
stop)
/sbin/ifconfig lo:0 down
echo "close LVS Directorserver"
echo "0">/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0">/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0">/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0">/proc/sys/net/ipv4/conf/all/arp_announce
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
[root@web2~]# chmod 755 /etc/init.d/lvsrs #给新增lvsrs脚本一个755权限
[root@web2~]# service lvsrs start #启动lvsrs服务，我环境启动时出错，给一个functions 755权限，按各自环境操作
/etc/init.d/lvsrs: line 7:./etc/rc.d/init.d/functions: Permission denied
Start LVS ofReal Server
[[email protected]]# chmod 755 /etc/rc.d/init.d/functions #给functions 755权限
[root@web2~]# service lvsrs start #启动lvsrs服务
Start LVS ofReal Server
[root@web2~]# ifconfig #查看已有一个VIP地址
lo:0 Link encap:Local Loopback
inet addr:192.168.1.10 Mask:255.255.255.255
UP LOOPBACK RUNNING MTU:16436 Metric:1
[root@web2~]# echo "service lvsrs start" >> /etc/rc.local #把lvsrs服务添加到开机启动，否则下次启动电脑后，还需手动启动。
[root@web2~]# yum install httpd
[root@web2~]# vi /var/www/html/index.html
Welcome to web2(192.168.2.29)
[root@web2~]# service httpd start
6.2 配置web4
配置同上
[root@web4~]# scp [email protected]:/etc/init.d/lvsrs /etc/init.d/
[root@web4~]# chmod 755 /etc/rc.d/init.d/functions
[root@web4~]# service lvsrs start
Start LVS ofReal Server
[root@web4~]# echo "service lvsrs start" >> /etc/rc.local
[root@web4~]# ifconfig
lo:0 Link encap:Local Loopback
inet addr:192.168.1.10 Mask:255.255.255.255
UP LOOPBACK RUNNING MTU:16436 Metric:1
[root@web4~]# yum install httpd
[root@web4~]# vi /var/www/html/index.html
Welcome to web4(192.168.2.30)
[root@web4~]# service httpd start
7 负载均衡以及高可用测试
7.1 负载均衡测试
（1）两台WEB服务器都工作正常，访问VIP地址时，可以被负载到两台WEB服务器上。
（2）停掉一台WEB服务器的http服务后，访问VIP地址时，被负载到正常的WEB服务器上。
7.2 高可用测试
（1）两台DR的keepalived服务器正常时，VIP地址在主服务器，提供正常服务。
（2）当关闭主服务器的keepalived后，VIP地址自动转移到备服务器上，当主服务器重启keepalived服务后，VIP自动转移回主服务器，备服务器继续备用，实现了高可用负载均衡。
8 其他常用操作
8.1 关闭LVS
(1)在real服务器上执行
#service lvsrs stop
(2)LVS服务器上执行
killall-9 keepalived;ipvsadm -C
8.2 系统虚拟IP管理：
查看虚拟IP地址
# ip address list
删除虚拟IP地址
# ip addr del 192.168.1.205 dev eth0
[root@mobankeepalived]# ip add
1: lo:
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet127.0.0.1/8 scope host lo
inet6::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:
link/ether 00:0c:29:48:9b:89 brd ff:ff:ff:ff:ff:ff
inet192.168.1.19/24 brd 192.168.1.255 scope global eth0
inet192.168.1.10/24 scope global secondary eth0
inet6fe80::20c:29ff:fe48:9b89/64 scope link
valid_lft forever preferred_lft forever
[root@moban keepalived]#
8.3 查看LVS的状态
#ipvsadm -ln
ipvsadm -L-n
[root@mobankeepalived]# ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
->RemoteAddress:Port ForwardWeight ActiveConn InActConn
TCP 192.168.1.10:80 wrr persistent 300
->192.168.1.17:80 Route 1 0 0
->192.168.1.18:80 Route 1 0 0
[root@moban keepalived]#