一 Heartbeat 基本理论
提供冗余系统:
HA Cluster:为提升系统调用性,组合多台主机构建成为的集群;
split brain, partitioned cluster
vote system:投票系统
HA中的各节点无法探测彼此的心跳信息时,必须无法协调工作;此种状态即为partitioned cluster;
少数服从多数的原则:quorum
with quorum > total/2
without quorum <= total/2
仲裁设备:
quorum disk = qdisk
ping node
failover: 失效转移,故障转移
failback:失效转回,故障转回
资源类型:
HA-aware:资源自身可直接调用HA集群底层的HA功能;
非HA-aware:必须借助于CRM完成在HA集群上实现HA功能;
资源的约束关系:
location:位置约束,定义资源对节点的倾向性;用数值来表示,-oo, +oo;
colocation:排列约束,定义资源彼此间“在一起”倾向性;-oo, +oo
分组:亦能实现将多个资源绑定在一起;
order:顺序约束,定义资源在同一个节点上启动时的先后顺序;
资源类型:
primitive:主资源,只能运行于集群内的某单个节点;(也称作native);
group:组资源,容器,包含一个或多个资源,这些资源可通过“组”这个资源统一进行调度;
clone:克隆资源,可以在同一个集群内的多个节点运行多份克隆;
master/slave:主从资源,在同一个集群内部于两个节点运行两份资源,其中一个主,一个为从;
资源隔离:
级别
节点:STONITH (Shooting The Other Node In The Head)
power switch
资源:fencing
FC SAN switch
解决方案:
Messaging Layer:
heartbeat
v1, v2, v3
corosync
cman (RedHat, RHCS)
keepalived (完全不同上述三种)
CRM:
heartbeat v1 haresources (配置接口:配置文件,文件名为haresources)
heartbeat v2 crm (在各节点运行一个crmd进程,配置接口:命令行客户端程序crmsh,GUI客户端:hb_gui);
heartbeat v3, pacemaker (pacemaker可以以插件或独立方式运行;配置接口,CLI接口:crmsh, pcs; GUI: hawk(webgui), LCMC, pacemaker-mgmt);
rgmanager (配置接口,CLI:clustat, cman_tool; GUI: Conga(luci+ricci))
组合方式:
heartbeat v1
heartbeat v2
heartbeat v3 + pacemaker
corosync + pacemaker
cman + rgmanager (RHCS)
cman + pacemaker
LRM: Local Resource Manager, 由CRM通过子程序提供;
RA: Resouce Agent
heartbeat legacy:heartbeat传统类型的RA,通常位于/etc/ha.d/haresources.d/目录下;
LSB:Linux Standard Base, /etc/rc.d/init.d目录下的脚本,至少接受4个参数:{start|stop|restart|status};
OCF:Open Cluster Framework
子类别:provider
STONITH:专用于实现调用STONITH设备功能的资源;通常为clone类型;
Heartbeat:心跳信息传递机制
serail cable:作用范围有限,不建议使用;
ethernet cable:
UDP Unicast
UDP Multicast
UDP Broadcast
组播地址:用于标识一个IP组播域;IANA(Internet Assigned number authority)把D类地址空间分配给IP组播使用;其范围是:224.0.0.0-239.255.255.255;
永久组播地址:224.0.0.0-224.0.0.255;
临时组播地址:224.0.1.0-238.255.255.255;
本地组播地址:239.0.0.0-239.255.255.255, 仅在特定本地范围内有效;
HA案例:ha web services
资源有三个:
ip, httpd, filesystem
fip: floating ip,172.16.100.17
daemon: httpd
约束关系:使用“组”资源,或通过排列约束让资源运行于同一节点;
顺序约束:有次序地启动资源;
程序选型:
heartbeat v2 + haresources
heartbeat v2 + crm (hb_gui)
配置HA集群的前提:
(1) 节点间时间必须同步:使用ntp协议实现;
(2) 节点间需要通过主机名互相通信,必须解析主机至IP地址;
(a) 建议名称解析功能使用hosts文件来实现;
(b) 通信中使用的名字与节点名字必须保持一致:“uname -n”命令,或“hostname”展示出的名字保持一致;
(3) 考虑仲裁设备是否会用到;
(4) 建立各节点之间的root用户能够基于密钥认证;
# ssh-keygen -t rsa -P ''
# ssh-copy-id -i /root/.ssh/id_rsa.pub root@HOSTNAME
注意:定义成为集群服务中的资源,一定不能开机自动启动;因为它们将由crm管理;
HA Cluster的工作模型:
A/P:两节点集群,active, passive;工作于主备模型;
HA Service通常只有一个:HA resources可能会有多个;
A/A:两节点集群,active/active,工作于双方模型;
N-M: N个节点,M个服务;通常N>M;
N-N:N个节点,N个服务;
资源运行的倾向性:
rgmanager:
failover domain, node priority
pacemaker:
资源黏性:运行于当前节点的倾向性;
资源约束:
位置约束:资源对运行于某节点的倾向性
inf: 正无穷
-inf: 负无穷
n:
-n:
排列约束:资源运行于一处的倾向性
inf
-inf
n
-n
顺序约束:启动的先后顺序
A --> B --> C
C --> B --> A
DC: Designated Coordinator
corosync:
AIS: Application Interface Standard,
SA Forum: OpenAIS
OpenAIS: 提供了一种集群模式,包含集群框架、集群成员管理、通信方式、集群监测,但没有集群资源管理功能;
组件包括:AMF, CLM, CPKT, EVT等;分支不同,包含的组件略有区别;
分支:picacho, whitetank, wilson,
corosync (集群管理引擎)
只是openais的一个子组件;
分裂成为两个项目:
corosync, wilson(ais的接口标准)
CentOS 5:
cman + rgmanager
CentOS 6:
cman + rgmanager
corosync + pacemaker
命令行管理工具:
crmsh: suse, CentOS 6.4-
pcs: RedHat, CentOS 6.5+
crm的常用子命令:
status
node
configure
ra
resource
configure常用的子命令:
primitive
group
clone
ms
location
colocation
order
show
property
primitive <rsc_id> class:provider:ra params param1=value1 param2=value2 op op1 param1=value op op2 parma1=value1
二 测试heartbeat+httpd
基本环境设置
10.10.10.50/24 node1.test.com node1
10.10.10.51/24 node2.test.com node2
10.10.10.5/24 vip
关闭iptables
service iptables stop chkconfig iptables off
关闭selinx
vim /etc/selinux/config SELINUX=disabled
修改主机名
vim /etc/sysconfig/network HOSTNAME=node2.test.com vim /etc/hosts 10.10.10.50 node1.test.com node1 10.10.10.51 node2.test.com node2 reboot
[root@node1 ~]# uname -n node1.test.com [root@node2 ~]# uname -n node2.test.com
安装httpd
yum install httpd [root@node1 ~]# vim /var/www/html/index.html node1.test.com 10.10.10.50 [root@node2 ~]# vim /var/www/html/index.html node2.test.com 10.10.10.51 service httpd start
测试打开 http://10.10.10.50
http://10.10.10.51
service httpd stop
准备ssh 信任(不用输账号密码进入系统)
cd ~ ssh-keygen -t rsa ssh-copy-id localhost ssh node2 cat ~/.ssh/id_rsa.pub >>.ssh/authorized_keys scp .ssh/authorized_keys node2:~/.ssh/
安装heartbeat
设置epel源 libnet在epel
wget http://mirrors.aliyun.com/epel/6/x86_64/epel-release-6-8.noarch.rpm rpm -ivh epel-release-6-8.noarch.rpm yum install net-snmp-libs libnet PyXML libtool-ltdl rpm -ivh heartbeat-2.1.4-12.el6.x86_64.rpm heartbeat-pils-2.1.4-12.el6.x86_64.rpm heartbeat-stonith-2.1.4-12.el6.x86_64.rpm
配置heartbeat
cd /etc/ha.d/ cp -r /usr/share/doc/heartbeat-2.1.4/ /etc/ha.d/ cp ./heartbeat-2.1.4/{ha.cf,authkeys,haresources} ./ chmod 600 authkeys
vim authkeys auth 2 #1 crc 2 sha1 password #3 md5 Hello!
vim ha.cf debugfile /var/log/ha-debug #logfacility local0 keepalive 2 deadtime 30 warntime 10 udpport 694 mcast eth0 225.10.0.1 694 1 0 auto_failback on node node1.test.com node node2.test.com ping 10.10.10.2 compression bz2 compression_threshold 2
vim haresources node1.test.com 10.10.10.5/24/eth0/10.10.10.254 httpd
scp -p ha.cf haresources authkeys node2:/etc/ha.d/ service heartbeat start;ssh node2 service heartbeat start
查看实时生成的日志
tail -f /var/log/ha-debug
检测运行状态,所有资源在node1上 ,node2 上只有heartbeat服务
[root@node1 ha.d]# ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:9F:AE:0D inet addr:10.10.10.50 Bcast:10.10.10.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe9f:ae0d/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1469 errors:0 dropped:0 overruns:0 frame:0 TX packets:1324 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:178040 (173.8 KiB) TX bytes:235325 (229.8 KiB) eth0:0 Link encap:Ethernet HWaddr 00:0C:29:9F:AE:0D inet addr:10.10.10.5 Bcast:10.10.10.254 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:61 errors:0 dropped:0 overruns:0 frame:0 TX packets:61 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:12332 (12.0 KiB) TX bytes:12332 (12.0 KiB) [root@node1 ha.d]# netstat -nlptu|grep 80 tcp 0 0 :::80 :::* LISTEN 3056/httpd [root@node1 ha.d]# ps aux |grep http root 3056 0.0 0.3 175276 3736 ? Ss 20:00 0:00 /usr/sbin/httpd apache 3060 0.0 0.2 175276 2460 ? S 20:00 0:00 /usr/sbin/httpd apache 3061 0.0 0.2 175276 2444 ? S 20:00 0:00 /usr/sbin/httpd apache 3062 0.0 0.2 175276 2444 ? S 20:00 0:00 /usr/sbin/httpd apache 3063 0.0 0.2 175276 2444 ? S 20:00 0:00 /usr/sbin/httpd apache 3064 0.0 0.2 175276 2444 ? S 20:00 0:00 /usr/sbin/httpd apache 3065 0.0 0.2 175276 2444 ? S 20:00 0:00 /usr/sbin/httpd apache 3066 0.0 0.2 175276 2444 ? S 20:00 0:00 /usr/sbin/httpd apache 3067 0.0 0.2 175276 2444 ? S 20:00 0:00 /usr/sbin/httpd root 3111 0.0 0.0 103304 896 pts/0 S+ 20:01 0:00 grep http [root@node1 ha.d]# ps aux |grep heartbeat root 2662 0.0 1.4 65496 14348 ? SLs 19:59 0:00 heartbeat: master control process nobody 2666 0.0 0.7 58764 7616 ? SL 19:59 0:00 heartbeat: FIFO reader nobody 2667 0.0 0.7 58892 7744 ? SL 19:59 0:00 heartbeat: write: mcast eth0 nobody 2668 0.0 0.7 58892 7744 ? SL 19:59 0:00 heartbeat: read: mcast eth0 nobody 2669 0.0 0.7 58892 7744 ? SL 19:59 0:00 heartbeat: write: ping 10.10.10.2 nobody 2670 0.0 0.7 58892 7744 ? SL 19:59 0:00 heartbeat: read: ping 10.10.10.2 root 3114 0.0 0.0 103304 896 pts/0 S+ 20:01 0:00 grep heartbeat [root@node2 ha.d]# ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:05:71:E9 inet addr:10.10.10.51 Bcast:10.10.10.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe05:71e9/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:22144 errors:0 dropped:0 overruns:0 frame:0 TX packets:16634 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:24037006 (22.9 MiB) TX bytes:7355584 (7.0 MiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:61 errors:0 dropped:0 overruns:0 frame:0 TX packets:61 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:26550 (25.9 KiB) TX bytes:26550 (25.9 KiB) [root@node2 ha.d]# netstat -nlptu|grep 80 [root@node2 ha.d]# ps aux |grep http root 2823 0.0 0.0 103308 856 pts/0 S+ 20:02 0:00 grep http [root@node2 ha.d]# ps aux |grep heartbeat root 2733 0.0 1.4 65496 14348 ? SLs 19:59 0:00 heartbeat: master control process nobody 2738 0.0 0.7 58764 7616 ? SL 19:59 0:00 heartbeat: FIFO reader nobody 2739 0.0 0.7 58892 7744 ? SL 19:59 0:00 heartbeat: write: mcast eth0 nobody 2740 0.0 0.7 58892 7744 ? SL 19:59 0:00 heartbeat: read: mcast eth0 nobody 2741 0.0 0.7 58892 7744 ? SL 19:59 0:00 heartbeat: write: ping 10.10.10.2 nobody 2742 0.0 0.7 58892 7744 ? SL 19:59 0:00 heartbeat: read: ping 10.10.10.2 root 2825 0.0 0.0 103308 856 pts/0 S+ 20:02 0:00 grep heartbeat
打开网页 http://10.10.10.5
停止第一个节点
[root@node1 ha.d]# service heartbeat stop Stopping High-Availability services: Done.
node1节点服务器全部停止 ,所有服务转移到node2
[root@node2 ha.d]# ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:05:71:E9 inet addr:10.10.10.51 Bcast:10.10.10.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe05:71e9/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:22411 errors:0 dropped:0 overruns:0 frame:0 TX packets:16943 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:24082852 (22.9 MiB) TX bytes:7416808 (7.0 MiB) eth0:0 Link encap:Ethernet HWaddr 00:0C:29:05:71:E9 inet addr:10.10.10.5 Bcast:10.10.10.254 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:61 errors:0 dropped:0 overruns:0 frame:0 TX packets:61 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:26550 (25.9 KiB) TX bytes:26550 (25.9 KiB) [root@node2 ha.d]# netstat -nlptu|grep 80 tcp 0 0 :::80 :::* LISTEN 3139/httpd [root@node2 ha.d]# ps aux |grep http root 3139 0.0 0.3 175276 3736 ? Ss 20:05 0:00 /usr/sbin/httpd apache 3141 0.0 0.2 175276 2460 ? S 20:05 0:00 /usr/sbin/httpd apache 3143 0.0 0.2 175276 2444 ? S 20:05 0:00 /usr/sbin/httpd apache 3145 0.0 0.2 175276 2444 ? S 20:05 0:00 /usr/sbin/httpd apache 3146 0.0 0.2 175276 2444 ? S 20:05 0:00 /usr/sbin/httpd apache 3149 0.0 0.2 175276 2444 ? S 20:05 0:00 /usr/sbin/httpd apache 3151 0.0 0.2 175276 2444 ? S 20:05 0:00 /usr/sbin/httpd apache 3153 0.0 0.2 175276 2444 ? S 20:05 0:00 /usr/sbin/httpd apache 3157 0.0 0.2 175276 2444 ? S 20:05 0:00 /usr/sbin/httpd root 3168 0.0 0.0 103308 856 pts/0 S+ 20:06 0:00 grep http [root@node2 ha.d]# ps aux |grep heartbeat root 2733 0.0 1.4 65496 14348 ? SLs 19:59 0:00 heartbeat: master control process nobody 2738 0.0 0.7 58764 7616 ? SL 19:59 0:00 heartbeat: FIFO reader nobody 2739 0.0 0.7 58892 7744 ? SL 19:59 0:00 heartbeat: write: mcast eth0 nobody 2740 0.0 0.7 58892 7744 ? SL 19:59 0:00 heartbeat: read: mcast eth0 nobody 2741 0.0 0.7 58892 7744 ? SL 19:59 0:00 heartbeat: write: ping 10.10.10.2 nobody 2742 0.0 0.7 58892 7744 ? SL 19:59 0:00 heartbeat: read: ping 10.10.10.2 root 3170 0.0 0.0 103308 852 pts/0 S+ 20:06 0:00 grep heartbeat
打开网页 http://10.10.10.5
重新开启node1
[root@node1 ha.d]# ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:9F:AE:0D inet addr:10.10.10.50 Bcast:10.10.10.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe9f:ae0d/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2457 errors:0 dropped:0 overruns:0 frame:0 TX packets:2066 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:299678 (292.6 KiB) TX bytes:365721 (357.1 KiB) eth0:0 Link encap:Ethernet HWaddr 00:0C:29:9F:AE:0D inet addr:10.10.10.5 Bcast:10.10.10.254 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:61 errors:0 dropped:0 overruns:0 frame:0 TX packets:61 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:12332 (12.0 KiB) TX bytes:12332 (12.0 KiB) [root@node1 ha.d]# ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:9F:AE:0D inet addr:10.10.10.50 Bcast:10.10.10.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe9f:ae0d/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2465 errors:0 dropped:0 overruns:0 frame:0 TX packets:2071 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:300586 (293.5 KiB) TX bytes:367585 (358.9 KiB) eth0:0 Link encap:Ethernet HWaddr 00:0C:29:9F:AE:0D inet addr:10.10.10.5 Bcast:10.10.10.254 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:61 errors:0 dropped:0 overruns:0 frame:0 TX packets:61 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:12332 (12.0 KiB) TX bytes:12332 (12.0 KiB) [root@node1 ha.d]# netstat -nlptu|grep 80 tcp 0 0 :::80 :::* LISTEN 3693/httpd [root@node1 ha.d]# ps aux |grep http root 3693 0.0 0.3 175276 3728 ? Ss 20:07 0:00 /usr/sbin/httpd apache 3695 0.0 0.2 175276 2460 ? S 20:07 0:00 /usr/sbin/httpd apache 3696 0.0 0.2 175276 2444 ? S 20:07 0:00 /usr/sbin/httpd apache 3698 0.0 0.2 175276 2444 ? S 20:07 0:00 /usr/sbin/httpd apache 3700 0.0 0.2 175276 2444 ? S 20:07 0:00 /usr/sbin/httpd apache 3701 0.0 0.2 175276 2444 ? S 20:07 0:00 /usr/sbin/httpd apache 3702 0.0 0.2 175276 2444 ? S 20:07 0:00 /usr/sbin/httpd apache 3703 0.0 0.2 175276 2444 ? S 20:07 0:00 /usr/sbin/httpd apache 3704 0.0 0.2 175276 2444 ? S 20:07 0:00 /usr/sbin/httpd root 3713 0.0 0.0 103304 896 pts/0 S+ 20:07 0:00 grep http [root@node1 ha.d]# ps aux |grep heartbeat root 3405 0.0 1.4 65496 14348 ? SLs 20:07 0:00 heartbeat: master control process nobody 3410 0.0 0.7 58764 7616 ? SL 20:07 0:00 heartbeat: FIFO reader nobody 3411 0.0 0.7 58892 7744 ? SL 20:07 0:00 heartbeat: write: mcast eth0 nobody 3412 0.0 0.7 58892 7744 ? SL 20:07 0:00 heartbeat: read: mcast eth0 nobody 3413 0.0 0.7 58892 7744 ? SL 20:07 0:00 heartbeat: write: ping 10.10.10.2 nobody 3414 0.0 0.7 58892 7744 ? SL 20:07 0:00 heartbeat: read: ping 10.10.10.2 root 3715 0.0 0.0 103308 896 pts/0 S+ 20:07 0:00 grep heartbeat
所有资源又全部转移到node1
三 测试heartbeat+httpd+nfs
开启新的服务器,版本为cenos 7.1 ip地址为10.10.10.60
关闭iptables
systemctl stop firewalld systemctl disable firewalld
关闭selinx
vim /etc/selinux/config SELINUX=disabled
安装nfs
m install nfs*
创建nfs共享文件目录和文件
mkdir -p /home/data/html vim /home/data/html/index.html html on nfs server 10.10.10.60
配置nfs
vim /etc/exports /home/data 10.10.10.0/24(rw,no_root_squash) exportfs -arv
启动nfs
systemctl start nfs-server systemctl status nfs-server
在node1 node2 上手动挂载测试
[root@node1 ha.d]# mount -t nfs 10.10.10.60:/home/data/html /var/www/html [root@node1 ha.d]# cat /var/www/html/index.html html on nfs server 10.10.10.60 [root@node1 ha.d]# umount /var/www/html
[root@node2 ha.d]# mount -t nfs 10.10.10.60:/home/data/html /var/www/html [root@node2 ha.d]# cat /var/www/html/index.html html on nfs server 10.10.10.60 [root@node2 /]# umount /var/www/html
修改heartbeat 设置
[root@node1 ha.d]# vim haresources node1.test.com 10.10.10.5/24 Filesystem::10.10.10.60:/home/data/html::/var/www/html::nfs httpd
复制到node2并启动服务
scp haresources node2:/etc/ha.d/ service heartbeat start;ssh node2 service heartbeat start
node1节点 检查
[root@node1 ha.d]# ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:9F:AE:0D inet addr:10.10.10.50 Bcast:10.10.10.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe9f:ae0d/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:4330 errors:0 dropped:0 overruns:0 frame:0 TX packets:3581 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:561109 (547.9 KiB) TX bytes:658645 (643.2 KiB) eth0:0 Link encap:Ethernet HWaddr 00:0C:29:9F:AE:0D inet addr:10.10.10.5 Bcast:10.10.10.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:61 errors:0 dropped:0 overruns:0 frame:0 TX packets:61 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:12332 (12.0 KiB) TX bytes:12332 (12.0 KiB) [root@node1 ha.d]# netstat -nlptu|grep 80 tcp 0 0 :::80 :::* LISTEN 4808/httpd [root@node1 ha.d]# ps aux |grep http root 4808 0.0 0.3 175276 3736 ? Ss 20:24 0:00 /usr/sbin/httpd apache 4810 0.0 0.2 175276 2460 ? S 20:24 0:00 /usr/sbin/httpd apache 4811 0.0 0.2 175276 2444 ? S 20:24 0:00 /usr/sbin/httpd apache 4814 0.0 0.2 175276 2444 ? S 20:24 0:00 /usr/sbin/httpd apache 4815 0.0 0.2 175276 2444 ? S 20:24 0:00 /usr/sbin/httpd apache 4816 0.0 0.2 175276 2444 ? S 20:24 0:00 /usr/sbin/httpd apache 4817 0.0 0.2 175276 2444 ? S 20:24 0:00 /usr/sbin/httpd apache 4818 0.0 0.2 175276 2444 ? S 20:24 0:00 /usr/sbin/httpd apache 4819 0.0 0.2 175276 2444 ? S 20:24 0:00 /usr/sbin/httpd root 4826 0.0 0.0 103304 896 pts/0 S+ 20:25 0:00 grep http [root@node1 ha.d]# ps aux |grep heartbeat root 4272 0.0 1.4 65496 14348 ? SLs 20:24 0:00 heartbeat: master control process nobody 4278 0.0 0.7 58764 7616 ? SL 20:24 0:00 heartbeat: FIFO reader nobody 4279 0.0 0.7 58892 7744 ? SL 20:24 0:00 heartbeat: write: mcast eth0 nobody 4280 0.0 0.7 58892 7744 ? SL 20:24 0:00 heartbeat: read: mcast eth0 nobody 4281 0.0 0.7 58892 7744 ? SL 20:24 0:00 heartbeat: write: ping 10.10.10.2 nobody 4282 0.0 0.7 58892 7744 ? SL 20:24 0:00 heartbeat: read: ping 10.10.10.2 root 4828 0.0 0.0 103304 896 pts/0 S+ 20:25 0:00 grep heartbeat node2节点 [root@node2 /]# ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:05:71:E9 inet addr:10.10.10.51 Bcast:10.10.10.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe05:71e9/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:23901 errors:0 dropped:0 overruns:0 frame:0 TX packets:18324 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:24337685 (23.2 MiB) TX bytes:7683518 (7.3 MiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:61 errors:0 dropped:0 overruns:0 frame:0 TX packets:61 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:26550 (25.9 KiB) TX bytes:26550 (25.9 KiB) [root@node2 /]# netstat -nlptu|grep 80 [root@node2 /]# ps aux |grep http root 3735 0.0 0.0 103308 852 pts/0 S+ 20:26 0:00 grep http [root@node2 /]# ps aux |grep heartbeat root 3684 0.0 1.4 65496 14348 ? SLs 20:24 0:00 heartbeat: master control process nobody 3689 0.0 0.7 58764 7616 ? SL 20:24 0:00 heartbeat: FIFO reader nobody 3690 0.0 0.7 58892 7744 ? SL 20:24 0:00 heartbeat: write: mcast eth0 nobody 3691 0.0 0.7 58892 7744 ? SL 20:24 0:00 heartbeat: read: mcast eth0 nobody 3692 0.0 0.7 58892 7744 ? SL 20:24 0:00 heartbeat: write: ping 10.10.10.2 nobody 3693 0.0 0.7 58892 7744 ? SL 20:24 0:00 heartbeat: read: ping 10.10.10.2 root 3739 0.0 0.0 103308 852 pts/0 S+ 20:27 0:00 grep heartbeat
打开网页 http://10.10.10.5
停止第一个节点
[root@node1 ha.d]# service heartbeat stop Stopping High-Availability services: Done.
node1节点服务器全部停止 ,所有服务转移到node2
[root@node2 /]# ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:05:71:E9 inet addr:10.10.10.51 Bcast:10.10.10.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe05:71e9/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:24306 errors:0 dropped:0 overruns:0 frame:0 TX packets:18704 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:24389056 (23.2 MiB) TX bytes:7747299 (7.3 MiB) eth0:0 Link encap:Ethernet HWaddr 00:0C:29:05:71:E9 inet addr:10.10.10.5 Bcast:10.10.10.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:61 errors:0 dropped:0 overruns:0 frame:0 TX packets:61 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:26550 (25.9 KiB) TX bytes:26550 (25.9 KiB) [root@node2 /]# netstat -nlptu|grep 80 tcp 0 0 :::80 :::* LISTEN 4197/httpd [root@node2 /]# ps aux |grep http root 4197 0.0 0.3 175276 3728 ? Ss 20:27 0:00 /usr/sbin/httpd apache 4199 0.0 0.2 175276 2460 ? S 20:27 0:00 /usr/sbin/httpd apache 4202 0.0 0.2 175276 2444 ? S 20:27 0:00 /usr/sbin/httpd apache 4203 0.0 0.2 175276 2444 ? S 20:27 0:00 /usr/sbin/httpd apache 4205 0.0 0.2 175276 2444 ? S 20:27 0:00 /usr/sbin/httpd apache 4207 0.0 0.2 175276 2444 ? S 20:27 0:00 /usr/sbin/httpd apache 4210 0.0 0.2 175276 2444 ? S 20:27 0:00 /usr/sbin/httpd apache 4214 0.0 0.2 175276 2444 ? S 20:27 0:00 /usr/sbin/httpd apache 4217 0.0 0.2 175276 2444 ? S 20:27 0:00 /usr/sbin/httpd root 4224 0.0 0.0 103308 856 pts/0 S+ 20:28 0:00 grep http [root@node2 /]# ps aux |grep heartbeat root 3684 0.0 1.4 65496 14348 ? SLs 20:24 0:00 heartbeat: master control process nobody 3689 0.0 0.7 58764 7616 ? SL 20:24 0:00 heartbeat: FIFO reader nobody 3690 0.0 0.7 58892 7744 ? SL 20:24 0:00 heartbeat: write: mcast eth0 nobody 3691 0.0 0.7 58892 7744 ? SL 20:24 0:00 heartbeat: read: mcast eth0 nobody 3692 0.0 0.7 58892 7744 ? SL 20:24 0:00 heartbeat: write: ping 10.10.10.2 nobody 3693 0.0 0.7 58892 7744 ? SL 20:24 0:00 heartbeat: read: ping 10.10.10.2 root 4226 0.0 0.0 103308 856 pts/0 S+ 20:28 0:00 grep heartbeat
刷新网页 http://10.10.10.5
重新开启node1
[root@node1 ha.d]# ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:9F:AE:0D inet addr:10.10.10.50 Bcast:10.10.10.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe9f:ae0d/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:5051 errors:0 dropped:0 overruns:0 frame:0 TX packets:4184 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:656248 (640.8 KiB) TX bytes:765685 (747.7 KiB) eth0:0 Link encap:Ethernet HWaddr 00:0C:29:9F:AE:0D inet addr:10.10.10.5 Bcast:10.10.10.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:61 errors:0 dropped:0 overruns:0 frame:0 TX packets:61 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:12332 (12.0 KiB) TX bytes:12332 (12.0 KiB) [root@node1 ha.d]# netstat -nlptu|grep 80 tcp 0 0 :::80 :::* LISTEN 5614/httpd [root@node1 ha.d]# ps aux |grep http root 5614 0.0 0.3 175276 3724 ? Ss 20:29 0:00 /usr/sbin/httpd apache 5616 0.0 0.2 175276 2456 ? S 20:29 0:00 /usr/sbin/httpd apache 5617 0.0 0.2 175276 2440 ? S 20:29 0:00 /usr/sbin/httpd apache 5619 0.0 0.2 175276 2440 ? S 20:29 0:00 /usr/sbin/httpd apache 5621 0.0 0.2 175276 2440 ? S 20:29 0:00 /usr/sbin/httpd apache 5622 0.0 0.2 175276 2440 ? S 20:29 0:00 /usr/sbin/httpd apache 5623 0.0 0.2 175276 2440 ? S 20:29 0:00 /usr/sbin/httpd apache 5624 0.0 0.2 175276 2440 ? S 20:29 0:00 /usr/sbin/httpd apache 5625 0.0 0.2 175276 2440 ? S 20:29 0:00 /usr/sbin/httpd root 5632 0.0 0.0 103304 896 pts/0 S+ 20:29 0:00 grep http [root@node1 ha.d]# ps aux |grep heartbeat root 5185 0.0 1.4 65496 14348 ? SLs 20:29 0:00 heartbeat: master control process nobody 5188 0.0 0.7 58764 7616 ? SL 20:29 0:00 heartbeat: FIFO reader nobody 5189 0.0 0.7 58892 7744 ? SL 20:29 0:00 heartbeat: write: mcast eth0 nobody 5190 0.0 0.7 58892 7744 ? SL 20:29 0:00 heartbeat: read: mcast eth0 nobody 5191 0.0 0.7 58892 7744 ? SL 20:29 0:00 heartbeat: write: ping 10.10.10.2 nobody 5192 0.0 0.7 58892 7744 ? SL 20:29 0:00 heartbeat: read: ping 10.10.10.2 root 5634 0.0 0.0 103304 896 pts/0 S+ 20:29 0:00 grep heartbeat
所有资源又全部转移到node1
四 测试heartbeat CRM LAMP Wordpress
安装mysql 和php
m install mysql-server php php-mysql
测试php
[root@node1 ha.d]# vim /var/www/html/info.php <?php phpinfo(); ?> [root@node1 ha.d]# service httpd start
打开http://10.10.10.50/info.php 测试PHP 信息
[root@node1 ha.d]# service httpd stop
测试mysql
[root@node1 ha.d]# service mysqld start [root@node1 ha.d]# mysql mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | test | +--------------------+ 3 rows in set (0.00 sec) mysql> exit Bye [root@node1 ha.d]# service mysqld stop
设置mysql data on nfs
在nfs 服务器
mkdir /home/data/mysql
在节点1
mkdir /data vim /etc/my.cnf [mysqld] datadir=/data/ mount -t nfs 10.10.10.60:/home/data/mysql /data /usr/bin/mysql_install_db --user=mysql --datadir=/data/ [root@node1 ha.d]# service mysqld start [root@node1 ha.d]# mysql mysql> create database wordpress; Query OK, 1 row affected (0.00 sec) mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | test | | wordpress | +--------------------+ 4 rows in set (0.01 sec) mysql> exit Bye [root@node1 ha.d]# service mysqld stop [root@node1 ha.d]# umount /data
在节点2
[root@node2 /]# mkdir /data [root@node2 /]# vim /etc/my.cnf [mysqld] datadir=/data/ [root@node2 /]# mount -t nfs 10.10.10.60:/home/data/mysql /data [root@node2 /]# /usr/bin/mysql_install_db --user=mysql --datadir=/data/ [root@node2 /]# service mysqld start [root@node2 /]# mysql mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | test | | wordpress | +--------------------+ 4 rows in set (0.00 sec) mysql> exit Bye [root@node2 /]# service mysqld stop [root@node2 /]# umount /data
安装 heartbeat-gui
yum install pygtk2-libglade xorg-x11-xauth rpm -ivh heartbeat-gui-2.1.4-12.el6.x86_64.rpm
echo "password" | passwd --stdin hacluster
yum install xorg-x11-xauth #如果未安装图型系统,要安装此组件,要不xshell报错 vim /etc/ha.d/ha.cf crm on service heartbeat start;ssh node2 service heartbeat start
在xshell 执行
hb_gui&
连接127.0.0.1 安装gui默认生成hacluster 用户, 默认无密码
请使用echo "password" | passwd --stdin hacluster 创建密码
连接成功后如图显示
开始添加资源 ,些次添加资源组 名为wordpress
添加第一个资源名为 ha_ip
类型为IPaddr 参数为 ip 10.10.10.5
添加第二个资源名为ha_nfs_mysql ,和我们之前测试时使用的参数一至,为mysql的共享存储
类型为Filesystem
参数为 device 10.10.10.60:/home/data/mysql
directory /data
fstype nfs
添加第三个资源名为ha_nfs_httpd ,和我们之前测试时使用的参数一至,为httpd的共享存储
类型为Filesystem
参数为 device 10.10.10.60:/home/data/html
directory /var/www/html
fstype nfs
添加第四个资源名为ha_mysql
类型为我们安装的mysqld ,不需要参数
添加第五个资源名为ha_httpd
类型为我们安装的httpd ,不需要参数
添加主机colocations ,为资源起动所有的位置
第一个为,ha_ip ha_nfs_mysql 必须在同一个node上
第一个为,ha_ip ha_nfs_httpd 必须在同一个node上
第一个为,ha_mysql ha_nfs_mysql 必须在同一个node上
第一个为,ha_httpd ha_nfs_httpd 必须在同一个node上
以上设置保证所有资源启动在同一个node上,根据需要修改。
添加资源order,意为资源的启动顺序
第一个为, ha_ip 比ha_nfs_mysql先启动
如果不能启动ha_ip ,则不启动ha_nfs_mysql
停止时,先停止ha_nfs_mysql ,后停止ha_ip
如果不能停止ha_nfs_mysql,则不停止ha_ip
第二个为, ha_ip 比ha_nfs_httpd先启动
如果不能启动ha_ip ,则不启动ha_nfs_httpd
停止时,先停止ha_nfs_httpd ,后停止ha_ip
如果不能停止ha_nfs_httpd,则不停止ha_ip
第三个为, ha_nfs_mysql 比ha_mysql先启动
如果不能启动ha_nfs_mysql ,则不启动ha_mysql
停止时,先停止ha_mysql ,后停止ha_nfs_mysql
如果不能停止ha_mysql,则不停止ha_nfs_mysql
第四个为, ha_nfs_httpd 比ha_httpd先启动
如果不能启动ha_nfs_httpd ,则不启动ha_httpd
停止时,先停止ha_httpd,后停止ha_nfs_httpd
如果不能停止ha_httpd,则不停止ha_nfs_httpd
右键ha_wordpress ,start 启动ha_wordpress资源
以上配置启动顺序为 ha_ip --> ha_nfs_mysql --> ha_mysql
ha_ip --> ha_nfs_httpd--> ha_httpd
右键节点node2 standby
资源全部转移动node1
安装wordpress
wget https://cn.wordpress.org/wordpress-4.3.1-zh_CN.zip
unzip wordpress-4.3.1-zh_CN.zip
cp -rf wordpress/ /var/www/html/
http://10.10.10.5/wordpress/wp-admin/setup-config.php
如果没有设置权限 ,
cd /var/www/html/
vim wp-config.php
复制上图中生成的代码,然后点击进行安装。
发表一个博客
设置node2 为 standby
结果刷新后发表的内容也可以显示