高性能集群软件-----Keepalived 安装配置
keepalived介绍：
Keepalived是Linux下轻量级的高可用的解决方案。Keepalived主要是通过虚拟路由冗余来实现高可用功能，具有部署和使用非常简单，只需一个配置文件即可。它是根据TCP/IP参考模型的第三，第四和第五层交换机检测到每个服务接点的状态，如果某个服务节点出现异常，或者出现故障，keepalived将检测到，并将出现故障的服务节点从集群中剔除，而在故障节点恢复后，keepalived又可以自动将此服务节点重新加入服务器集群中，这些工作全部自动完成，不需要人工干涉，需要人工完成的只是修复出现故障的服务节点。（2台服务器运行Keepalived，一台为主服务器（MASTER），一台为备份服（BACKUP），但是对外表现为一个虚拟IP，主服务器会发送特定的消息给备份服务器，当备份服务器收不到这个消息的时候，即主服务器宕机的时候， 备份服务器就会接管虚拟IP，继续提供服务，从而保证了高可用性）。keepalived采用的是vrrp来实现的。
安装和配置：
1，编译安装：
[root@localhost install]#cd /tmp/install
[root@localhost install]# yum install gcc gcc-c++ openssl* kernel-devel net-snmp* libnl* -y
[root@localhost install]#wget http://www.keepalived.org/software/keepalived-1.2.19.tar.gz
###或者直接在官网上下载
[root@localhost install]#tar zxvf keepalived-1.2.19.tar.gz
[root@localhost install]#cd keepalived-1.2.19
[root@localhost install]#./configure --prefix=/usr/local/keepalived
[root@localhost install]#make && make install
2,配置：
（1）环境说明
| hostname |
PI | 安装软件 | OS |
| Master |
192.168.122.120 | nginx（已安装).keepalived | Centos6.5 |
| backup | 192.168.122.121 | nginx（已安装).keepalived | Centos6.5 |
| vip（虚拟IP） |
192.168.122.130 |
备注：依次在192.168.122.120和192.168.122.121进行编译安装如上边的步骤,安装完成之后，会在/usr/local下生成bin、etc、sbin三个文件夹，主要的配置文件在etc目录下，执行文件在sbin目录下
（2）依次在120和121上操作：
(a)####keepalive启动时默认会去读/etc/keepalived/keepalived.conf配置文件，但编译安转后，没有自动创建这个文件，而是放到了/usr/local/etc/keepalived/keepalived.conf
为了以后配置方便，我们手动将/usr/local/etc/keepalived/keepalived.conf放到/etc/keepalived/下
[root@localhost install]#mkdir /etc/keepalived/ [root@localhost install]#cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/ [root@localhost install]#cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/keepalived
(b)#####建立服务启动脚本
[root@localhost install]#cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/ [root@localhost install]#chmod +x /etc/init.d/keepalived
(c)####更改启动脚本中keepalive启动参数文件的位置，脚本文件路径是/etc/sysconfig/keepalived（如果是通过yum安装，则这个位置就是正确的），而实际文件是在/usr/local/keepalived/etc/sysconfig/keepalived
#sed -i 's#/etc/sysconfig/keepalived#/usr/local/etc/sysconfig/keepalived#g' /etc/init.d/keepalived
[root@localhost install]#vi /etc/init.d/keepalived
#!/bin/sh
#
# Startup script for the Keepalived daemon
#
# processname: keepalived
# pidfile: /var/run/keepalived.pid
# config: /etc/keepalived/keepalived.conf
# chkconfig: - 21 79
# description: Start and stop Keepalived
# Source function library
. /etc/rc.d/init.d/functions
# Source configuration file (we set KEEPALIVED_OPTIONS there)
. /usr/local/keepalived/etc/sysconfig/keepalived
RETVAL=0
prog="keepalived"
start() {
echo -n $"Starting $prog: "
daemon keepalived ${KEEPALIVED_OPTIONS}
RETVAL=$?
echo
[ $RETVAL -eq 0 ] && touch /var/lock/subsys/$prog
}
"/etc/init.d/keepalived" 74L, 1329C
（d)给/usr/local/keepalived/sbin/keepalived做个软连接，否则会报错
[root@localhost sbin]# /etc/init.d/keepalived start Starting keepalived: /bin/bash: keepalived: command not found [FAILED]
[root@localhost sbin]# ln -s /usr/local/keepalived/sbin/keepalived /usr/bin/ [root@localhost sbin]# /etc/init.d/keepalived start Starting keepalived: [ OK ] [root@localhost sbin]#
（3)配置keepalived.conf 主要配置三个地方：state MASTER ,priority 99 和virtual_ipaddress {192.168.122.130}
root@localhost sbin]# vi /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs { #全局设置
router_id LVS_DEVEL #仅仅是个机器标识，可自定义
}
vrrp_script check_http { #首先在vrrp_script区域定义脚本名字和脚本执行的间隔和脚本执行的优先级变更
script "/root/chk_httpd.sh" #然后在实例(vrrp_instance)里面引用，有点类似脚本里面的函数引用一样：先定义，后引用函数名
interval 2 #脚本执行间隔
weight 10
}
vrrp_instance VI_1 { #定义一个vrrp实例名，可自定义
state MASTER #设定初始状态，可以是MASTER或BACKUP，不过当其他节点keepalived启动时会将priority比较大的节点选举为MASTER，因此该项其实没有实质用途。
interface eth0 #具有固定ip的网卡，用来接收和发送vrrp包。如果没有设定mcast_src_ip发送多播数据包的地址，那么将使用这个网卡的ip来发送。
virtual_router_id 51 #取值在0-255之间，用来区分多个instance的VRRP组播。相同的VRID为一个组，他将决定多播的MAC地址
priority 99 #设置本节点的优先级，优先级高的为master
advert_int 1 #组播信息发送间隔，两个节点设置必须一样，默认为1秒
authentication { #设置节点间验证信息，所以所有节点必须一致
auth_type PASS
auth_pass 1111
}
track_script { #引用上面定义的检测脚本
check_http
weight 20
}
virtual_ipaddress { #设置vip，如果不指定网卡，那么默认绑定到interface指定的网卡，可以通过添加dev ethx的方式，绑定到指定的网卡。
192.168.122.130
}
}
同理，在121上进行操作：
vi /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id LVS_DEVEL
}
vrrp_script check_http {
script "/root/chk_httpd.sh"
interval 2
weight 10
}
vrrp_instance VI_1 {
state BACKUP#此处为BACKUP
interface eth0
virtual_router_id 51
priority 96#此处比MASTER要小
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
track_script {
check_http
weight 20
}
virtual_ipaddress {
192.168.122.130
}
}
(4）120和121同时启动keepalived
[root@hadoop1 ~]# /etc/init.d/keepalived start Starting keepalived: [ OK ] [root@hadoop1 ~]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 52:54:00:05:7b:31 brd ff:ff:ff:ff:ff:ff inet 192.168.122.120/24 brd 192.168.122.255 scope global eth0 inet 192.168.122.130/32 scope global eth0 inet6 fe80::5054:ff:fe05:7b31/64 scope link valid_lft forever preferred_lft forever
[root@hadoop2 ~]# /etc/init.d/keepalived start Starting keepalived: [ OK ] [root@hadoop2 ~]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 52:54:00:c6:65:80 brd ff:ff:ff:ff:ff:ff inet 192.168.122.121/24 brd 192.168.122.255 scope global eth0 inet6 fe80::5054:ff:fec6:6580/64 scope link valid_lft forever preferred_lft forever
打开浏览器进行查看：
(5)验证实验结果：
实验一：关闭120的keepalived服务看是否切换：
[root@hadoop1 ~]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 52:54:00:05:7b:31 brd ff:ff:ff:ff:ff:ff inet 192.168.122.120/24 brd 192.168.122.255 scope global eth0 inet6 fe80::5054:ff:fe05:7b31/64 scope link valid_lft forever preferred_lft forever
在查看下121的IP地址：
[root@hadoop2 ~]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 52:54:00:c6:65:80 brd ff:ff:ff:ff:ff:ff inet 192.168.122.121/24 brd 192.168.122.255 scope global eth0 inet 192.168.122.130/32 scope global eth0 inet6 fe80::5054:ff:fec6:6580/64 scope link valid_lft forever preferred_lft forever [root@hadoop2 ~]#
已成功切换，在打开浏览器：
实验二：重新启动120,查看下虚拟ip130是否切换过来：
[root@hadoop1 ~]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 52:54:00:05:7b:31 brd ff:ff:ff:ff:ff:ff inet 192.168.122.120/24 brd 192.168.122.255 scope global eth0 inet 192.168.122.130/32 scope global eth0 inet6 fe80::5054:ff:fe05:7b31/64 scope link valid_lft forever preferred_lft forever [root@hadoop1 ~]#
已成功切换过来，再看121的IP地址：
[root@hadoop2 ~]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 52:54:00:c6:65:80 brd ff:ff:ff:ff:ff:ff inet 192.168.122.121/24 brd 192.168.122.255 scope global eth0 inet 192.168.122.130/32 scope global eth0 inet6 fe80::5054:ff:fec6:6580/64 scope link valid_lft forever preferred_lft forever [root@hadoop2 ~]#
查看下浏览器：
已正常切换
备注：keepalived日志文件的分离
注意，如果有报错的话，请查看keepalived日志文件，keepalived日志文件默认的都是写入到/var/log/message下的，由于日志文件过大，现在需要分离出来日志文件，步骤：
1，修改/etc/sysconfig/keepalived 把KEEPALIVED_OPTIONS="-D" 修改为KEEPALIVED_OPTIONS="-D -d -S 0"
[root@hadoop1 sysconfig]# vi /etc/sysconfig/keepalived # Options for keepalived. See `keepalived --help' output and keepalived(8) and # keepalived.conf(5) man pages for a list of all options. Here are the most # common ones : # # --vrrp -P Only run with VRRP subsystem. # --check -C Only run with Health-checker subsystem. # --dont-release-vrrp -V Dont remove VRRP VIPs & VROUTEs on daemon stop. # --dont-release-ipvs -I Dont remove IPVS topology on daemon stop. # --dump-conf -d Dump the configuration data. # --log-detail -D Detailed log messages. # --log-facility -S 0-7 Set local syslog facility (default=LOG_DAEMON) # KEEPALIVED_OPTIONS="-D -d -S 0"
2,设置syslog，修改/etc/rsyslog.conf 最后添加：
# keepalived -S 0
local0.*
[root@hadoop1 sysconfig]# vi /etc/rsyslog.conf # rsyslog v5 configuration file # For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html # If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html #### MODULES #### $ModLoad imklog # provides kernel logging support (previously done by rklogd) #$ModLoad immark # provides --MARK-- message capability # Provides UDP syslog reception #$ModLoad imudp #$UDPServerRun 514 # Provides TCP syslog reception #$ModLoad imtcp #$InputTCPServerRun 514 #### GLOBAL DIRECTIVES #### # Use default timestamp format $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat # not useful and an extreme performance hit #$ActionFileEnableSync on # ### begin forwarding rule ### # The statement between the begin ... end define a SINGLE forwarding # rule. They belong together, do NOT split them. If you create multiple # forwarding rules, duplicate the whole block! # Remote Logging (we use TCP for reliable delivery) # # An on-disk queue is created for this action. If the remote host is # down, messages are spooled to disk and sent when it is up again. #$WorkDirectory /var/lib/rsyslog # where to place spool files #$ActionQueueFileName fwdRule1 # unique name prefix for spool files #$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) #$ActionQueueSaveOnShutdown on # save messages to disk on shutdown #$ActionQueueType LinkedList # run asynchronously #$ActionResumeRetryCount -1 # infinite retries if host is down # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional #*.* @@remote-host:514 # ### end of the forwarding rule ### #keepalived -S 0 local0.* /etc/keepalived/keepalived.log
3,重启日志文件：
local0.* /etc/keepalived/keepalived.log
[root@hadoop1 sysconfig]# /etc/init.d/rsyslog restart Shutting down system logger: [ OK ] Starting system logger: [ OK ] [root@hadoop1 sysconfig]# /etc/init.d/keepalived restart Stopping keepalived: [ OK ] Starting keepalived: [root@hadoop1 keepalived]# tail -f /etc/keepalived/keepalived.log Jan 15 17:37:27 hadoop1 Keepalived_vrrp[2008]: VRRP_Instance(VI_1) Received lower prio advert, forcing new election Jan 15 17:37:27 hadoop1 Keepalived_vrrp[2008]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.122.130
日志正常输出。