Heartbeat+Ldirectord+LVS+httpd集群部署
一、硬件环境
4台虚拟机在同一网段
操作系统：centos6.3
关闭系统不必要的服务脚本
#!/bin/bash services=`chkconfig --list|cut -f1|cut -d" " -f1` for ser in $services do if [ "$ser" == "network" ] || [ "$ser" == "rsyslog" ] || [ "$ser" == "sshd" ] || [ "$ser" == "crond" ] || [ "$ser" == "atd" ]; then chkconfig "$ser" on else chkconfig "$ser" off fi done reboot
二、ip地址规划
master 172.30.82.45 slave 172.30.82.58 node1 172.30.82.3 node2 172.30.82.11 VIP 172.30.82.61
三、注意：
1、设置各个节点间的时间同步
ntpdate 172.30.82.254 &>/dev/null
2、基于hosts文件实现能够互相用主机名访问,修改/etc/hosts文件
3、使用uname -n执行结果要和主机名相同
4、确保ldirectord服务关闭开机启动
chkconfig ldirectord off
5、关闭selinux
setenfroce 0
四、安装相关软件
heartbeat heartbeat-libs 下载地址： http://dl.fedoraproject.org/pub/epel/6/x86_64/repoview/letter_h.group.html libdnet 下载地址： http://dl.fedoraproject.org/pub/epel/6/x86_64/repoview/letter_l.group.html ldirectord 下载地址： http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/x86_64/ yum install -y perl-MailTools perl-Compress-Zlib perl-HTML-Parser perl-HTML-Tagset perl-TimeDate perl-libwww-perl yum --nogpgcheck localinstall heartbeat-3.0.4-2.el6.x86_64.rpm heartbeat-libs-3.0.4-2.el6.x86_64.rpm libdnet-1.12-6.el6.x86_64.rpm ldirectord-3.9.6-0rc1.1.1.x86_64.rpm
五、配置director节点的高可用
1、拷贝配置文件
cp /usr/share/doc/heartbeat-3.0.4/{authkeys，ha.cf，haresources} /etc/ha.d
cp /usr/share/doc/ldirectord-3.9.6/ldirectord.cf /etc/ha.d/
2、修改authkeys
auth 1
1 md5 eb09b7d8e7117191cb9e6e68054ff83b
eb09b7d8e7117191cb9e6e68054ff83b 为md5加密的随机数可以通过取得：
dd if=/dev/random bs=10 count=1 |md5sum
3、修改heartbeat主配置文件ha.cf
logfile /var/log/heartbeat.log # 日志文件的路径
#logfacility local0
keepalive 2 # 心跳检测的平率为2秒一次
deadtime 10 # 如果30秒内没有相应心跳就做切换
warntime 10 # 指明心跳的延时为10秒做警告记录一条日志，没有其他动作
initdead 40 # 这个值设置成deadtime的两倍就可以了
# baud 115200 19200 # 串口速率
# serial /dev/ttyS0 # 串口检测心跳
udpport 694 # 数据通信的端口
bcast eth0 # eth0为心跳工作接口
ucast eth1 10.0.0.2 # 检测对端心跳接口的地址 (HA备节点的eth1地址)
auto_failback on # 当主HA节点恢复后，自动切换
node master # 主HA节点计算机名
node slave # 备HA节点计算机名
ping 172.30.82.254 # 检测网络是否正常，可以设置成网关地址
4、修改集群资源配置文件haresources，添加：
master 172.30.82.61/24/eth0/172.30.82.255 ldirectord
master 指定集群的主节点
172.30.82.61 指定集群的vip
/24子网掩码
/eth0指定vip绑定的网卡，若未指定且服务器有多块网卡，则系统默认绑定到与vip在同一网段的网卡上
/172.30.82.255 网络广播地址，可以不指定
ldirectord定义启动ldirectord资源，注意资源之间的空格
5、修改ldirectord配置文件ldirectord.cf
checktimeout=3 # 检测超时
checkinterval=1 # 检测间隔
autoreload=yes # 从新载入客户机
logfile="/var/log/ldirectord.log" # 日志路径
logfile="local0"
quiescent=no # realserver 宕机后从lvs列表中删除，恢复后自动添加进列表
virtual=192.168.2.200:80 # 监听VIP地址80端口
real=192.168.2.203:80 gate # 真机IP地址和端口 路由模式
real=192.168.2.204:80 gate
fallback=127.0.0.1:80 gate # 如果real节点都宕机，则回切到环回地址
service=http # 服务是http
request=".text.html" # 保存在real的web根目录并且可以访问,通过它来判断real是否存活
receive="OK" # 检测文件内容
scheduler=rr # 调度算法
protocol=tcp # 检测协议
checktype=negotiate # 检测类型
checkport=80 # 检测端口
6、复制配置文件到备用节点：
scp -P authkeys haresources ha.cf ldirectord.cf slave:/etc/ha.d/
六、DR模型下配置realserver脚本：
#!/bin/bash
VIP=172.30.82.61
host=`/bin/hostname`
case "$1" in
start)
# Start LVS-DR real server on this machine.
/sbin/ifconfig lo down
/sbin/ifconfig lo up
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
/sbin/ifconfig lo:0 $VIP netmask 255.255.255.255 up
/sbin/route add -host $VIP dev lo:0
;;
stop)
# Stop LVS-DR real server loopback device(s).
/sbin/ifconfig lo:0 down
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
;;
status)
# Status of LVS-DR real server.
islothere=`/sbin/ifconfig lo:0 | grep $VIP`
isrothere=`netstat -rn | grep "lo" | grep $VIP`
if [ ! "$islothere" -o ! "$isrothere" ];then
# Either the route or the lo:0 device
# not found.
echo "LVS-DR real server is stopped."
else
echo "LVS-DR real server is running."
fi
;;
*)
# Invalid entry.
echo "$0: Usage: $0 {start|status|stop}"
exit 1
;;
esac
七、real上安装httpd服务并添加测试页面
1、node1 yum install -y httpd echo "Welcome to realserver 1" >/var/www/html/index.html echo "OK" >/var/www/html/.text.html service httpd start 2、node2 yum install -y httpd echo "Welcome to realserver 2" >/var/www/html/index.html echo "OK" >/var/www/html/.text.html service httpd start
八、开启并测试高可用集群服务
1、在master上执行 service heartbeat start ssh slave 'service heaertbeat start' 2、集群资源运行状态测试 a、客户端访问http://172.30.82.61 master 上执行 [root@master log]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 172.30.82.61:80 rr -> 172.30.82.3:80 Route 1 0 11 -> 172.30.82.11:80 Route 1 1 11 b、在slave上执行 [root@slave log]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn 说明集群资源只运行在master上 3、集群资源转移测试 a、master上执行 service heartbeat stop [root@master log]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn b、在slave上执行 [root@slave log]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 172.30.82.61:80 rr -> 172.30.82.3:80 Route 1 1 17 -> 172.30.82.11:80 Route 1 0 18 说明集群资源转移成功 c、master上执行,测试集群服务是否会返回到主节点上 service heartbeat start d、后端服务故障检测node1上执行 service httpd stop 查看master集群服务 [root@master log]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 172.30.82.61:80 rr -> 172.30.82.11:80 Route 1 0 0 恢复node1服务 service httpd start [root@master log]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 172.30.82.61:80 rr -> 172.30.82.11:80 Route 1 0 0 -> 172.30.82.3:80 Route 1 0 0
报错处理：
heartbeat: [56411]: ERROR: glib: Error binding socket (Permission denied). Retrying setenforce 0 关闭selinux