本文主要讲解LVS结合keepalived和dns实现高可用,主要实现调度器和RealServer的高可用。主调度器如果出现宕机等情况,利用 keepalived的心跳监测会自动把VIP和router转移到备调度器,当一台RealServer出现故障的时候自动从LVS负载中剔除,恢复之 后自动加入。以下为本次实验架构图
系统环境:
具体的IP配置,按照图示配置即可,这里不再说明。
iptables -t nat -A PREROUTING -p udp --dport 53 -j DNAT --to 192.168.0.10 iptables -t nat -A PREROUTING -p tcp --dport 80 -j DNAT --to 192.168.0.10
RealServer的配置基本类似,web和dns配置这里不作说明,以RealServer1为例
apt-get install arptables
arptables -A INPUT -d 192.168.0.10 -j DROP arptables -A OUTPUT -s 192.168.0.10 -j mangle --mangle-ip-s 192.168.1.11
关于arptables的用法可以参见 http://kb.linuxvirtualserver.org/wiki/Using_arptables_to_disable_ARP
这里以主调度器为例:
apt-get install keepalived apt-get install ipvsadm cp /usr/share/doc/keepalived/samples/keepalived.conf.sample /etc/keepalived/keepalived.conf
作为对比配置中同时加入了http的配置
cat /etc/keepalived/keepalived.conf vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.0.10/24 } virtual_routes { via 192.168.0.100 dev eth0 } } virtual_server 192.168.0.10 80 { delay_loop 6 lb_algo rr lb_kind DR protocol TCP real_server 192.168.0.11 80 { weight 1 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 192.168.0.12 80 { weight 1 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } } virtual_server 192.168.0.10 53 { delay_loop 6 lb_algo rr lb_kind DR protocol UDP real_server 192.168.0.11 53 { weight 1 MISC_CHECK { connect_timeout 3 misc_path "/etc/keepalived/dnscheck.sh -h 192.168.0.11" } } real_server 192.168.0.12 53 { weight 1 MISC_CHECK { connect_timeout 3 misc_path "/etc/keepalived/dnscheck.sh -h 192.168.0.12" } }
注:如果同时有http和dns服务,一定要把http配置放在dns配置之前,笔者在测试过程中一开始是把dns配置写在http配置之前,导致lvs配置失败!
如果细心的话,你会发现keepalived配置文件中http和dns的检查方式是不同的。keepalived的健康检查方式 有:HTTP_GET|SSL_GET|TCP_CHECK|SMTP_CHECK|MISC_CHECK,但是自带没有UDP相关的检查方式,所以需要 自己编写DNS检测脚本,使用MISC_CHECK外部调用脚本,编写DNS脚本赋予执行权限存放在/etc/keepalived/下,内容如下:
cat /etc/keepalived/dnscheck.sh #!/bin/bash # dns zone文件中加入了AaBbCcDdEeFf的txt记录 domain="txt.qq.com" [ $# -le 1 ] && { echo "usage: ${0} -h <ip>"; exit 126; } while getopts "h:" OPT;do case $OPT in h)host=$OPTARG;; *)echo "usage: $0 -h <ip>" && exit 1;; esac done dig @${host} txt ${domain} +time=1 | grep "\<AaBbCcDdEeFf\>" >/dev/null exit $?
完成主调度器配置之后,相应的在备调度器安装keepalived和ipvsadm,然后把主调度器keepalived配置文件、检测脚本拷贝到备调度器,修改以下两个位置:
分别启动主备keepalived
service keepalived start
检查是否配置成功
ipvsadm -L -n IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.0.10:80 rr -> 192.168.0.11:80 Route 1 0 0 -> 192.168.0.12:80 Route 1 0 0 UDP 192.168.0.10:53 rr -> 192.168.0.11:53 Route 1 0 0 -> 192.168.0.12:53 Route 1 0 0 ip addr show dev eth0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:7d:f7:5a brd ff:ff:ff:ff:ff:ff inet 192.168.0.20/24 brd 192.168.0.255 scope global eth0 inet 192.168.0.10/24 scope global secondary eth0 inet6 fe80::20c:29ff:fe7d:f75a/64 scope link valid_lft forever preferred_lft forever
如果vip和lvs规则都已加载,表示配置已经成功,备采用类似方法查看,备调度器只有lvs规则而没有vip
以上配置完成之后就可以在客户端测试,测试效果省略,以下为测试方法
1、正常测试服务是否正常
dns:
dig @10.2.0.110 domain.com
web:
elinks --dump 10.2.0.110
2、测试完成之后,停掉一台RealServer机器,主DR调度器上使用如下方法监控
watch -n 1 ipvsadm -L -n
观看lvs规则是否自动剔除停掉的RealServer,再打开停掉的RealServer,就会发现主调度器自动会加入恢复的RealServer
3、停掉主调度器,如果一切正常,备调度器就会接管vip替代主调度器的位置
参考文档:https://gist.github.com/antonlindstrom/1099202#file-dnscheck-L11