corosync+pacemaker+san实现web服务高可用

一：实验环境

节点	OS	IP	SAN_IP	VIP
node1	rhel 6.5	192.168.10.11	172.16.1.1	192.168.10.100
node2	rhel 6.5	192.168.10.12	172.16.1.2
san	rhel 6.5		172.16.1.3

注：

1.corosync和pacemaker的概念这里就不说了，网上有很多资料

2.其中两节点IP地址已按上图设置好

3.已连接好san (映射本地盘符为/dev/sdb)

4.两节点已配置相互ssh信任，并已做了时间同步

 

二：安装相关软件（节点1和2都安装）

1.安装corosync、pacemaker

[root@node1 ~]# for i in 1 2; do ssh node$i yum -y install corosync* pacemaker* ; done

注：rhel 5 系列corosync、pacemaker下载地址为：

http://clusterlabs.org/    根据自己的发行版本，下载对应的软件包

2.安装crmsh

到下面地址下载crmsh、pssh、python-pssh

http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/x86_64/

本次下载的各版本为：

crmsh-2.1-1.6.x86_64.rpm

pssh-2.3.1-4.1.x86_64.rpm

python-pssh-2.3.1-4.1.x86_64.rpm

 

安装：

[root@node1 ~]# for i in 1 2; do ssh node$i yum -y --nogpgcheck localinstall /root/*.rpm; done

 

3.安装apache

[root@node1 ~]# for i in 1 2; do ssh node$i yum -y install httpd; done

[root@node1 ~]# for i in 1 2; do ssh node$i chkconfig httpd off; done

 

三：配置corosync

1.[root@node1 ~]# cd /etc/corosync/

2.[root@node1 corosync]# cp corosync.conf.example corosync.conf

3. 完成后的配置文件如下所示：

[root@node1 corosync]# cat corosync.conf

# Please read the corosync.conf.5 manualpage

compatibility: whitetank

 

totem {

       version: 2

       secauth: off

       threads: 0

       interface {

                ringnumber: 0

                bindnetaddr: 192.168.10.0     //在哪个网段上进行多播，根据实际情况修改

                mcastaddr: 226.94.1.1

                mcastport: 5405

                ttl: 1

       }

}

 

logging {

       fileline: off

       to_stderr: no

       to_logfile: yes

       to_syslog: no

       logfile: /var/log/cluster/corosync.log   //日志所在位置

       debug: off

       timestamp: on

       logger_subsys {

                subsys: AMF

                debug: off

       }

}

 

amf {

       mode: disabled

}

 

#

# 以下为添加部分

service {

       ver: 0

       name: pacemaker          //启动corosync时，同时启动pacemaker

}

 

aisexec {

       user: root

       group: root

}

4.复制配置文件到node2上

[root@node1 corosync]# scp corosync.conf node2:/etc/corosync/

5.启动corosync服务

[root@node1 ~]# /etc/init.d/corosync start

Starting Corosync Cluster Engine(corosync):               [  OK  ]

[root@node1 ~]# ssh node2 "/etc/init.d/corosync start"

Starting Corosync Cluster Engine(corosync): [  OK  ]

6.设置corosync随机启动

[root@node1 ~]# for i in 1 2; do ssh node$i chkconfig corosync on; done

 

四：集群服务配置

1.查看目前的集群状态

[root@node1 ~]# crm status

Last updated: Tue Jun 23 15:28:58 2015

Last change: Tue Jun 23 15:23:58 2015 via crmd on node1

Stack: classic openais (with plugin)

Current DC: node1 - partition with quorum

Version: 1.1.10-14.el6-368c726

2 Nodes configured, 2 expected votes

0 Resources configured

 

Online: [ node1 node2 ]

 

由以上可知，节点1和2都在线，还未配置任何资源

 

2.设置集群属性

[root@node1 ~]# crm configure

crm(live)configure# property stonith-enabled=false   //禁用stonith设备

crm(live)configure# property no-quorum-policy=ignore //达不到法定票数的策略为忽略

crm(live)configure# verify

crm(live)configure# commit

crm(live)configure# show

node node1

node node2

property cib-bootstrap-options: \

       dc-version=1.1.10-14.el6-368c726 \

       cluster-infrastructure="classic openais (with plugin)" \

       expected-quorum-votes=2 \

       stonith-enabled=false \

       no-quorum-policy=ignore

 

3.添加文件系统（Filesystem）资源

crm(live)configure# primitive webstore ocf:heartbeat:Filesystem params \

  > device=/dev/sdb1 directory=/var/www/html fstype=xfs \

  > op start timeout=60 \

  > op stop timeout=60

crm(live)configure# verify

先不要提交，接着设置资源wetstore最优先运行在node1节点上

crm(live)configure# location webstore_perfer_node1 webstore 50: node1

crm(live)configure# verify

现在提交

crm(live)configure# commit

返回到上一级，查看目前集群状态

crm(live)configure# cd

crm(live)# status

Last updated: Tue Jun 23 15:55:03 2015

Last change: Tue Jun 23 15:54:14 2015 via cibadmin on node1

Stack: classic openais (with plugin)

Current DC: node1 - partition with quorum

Version: 1.1.10-14.el6-368c726

2 Nodes configured, 2 expected votes

1 Resources configured

 

Online: [ node1 node2 ]

 

webstore       (ocf::heartbeat:Filesystem):    Started node1

 

由以上可知，webstore目前运行在node1上

 

4.添加httpd服务资源并设置httpd服务必须和webstore在一起，webstore必须先启动后，httpd服务才能启动

crm(live)configure# primitive httpd lsb:httpd

crm(live)configure# colocation httpd_with_httpd inf: httpd webstore

crm(live)configure# order webstore_before_httpd Mandatory: webstore:start httpd

crm(live)configure# verify

crm(live)configure# commit

crm(live)configure# cd

crm(live)# status

Last updated: Tue Jun 23 15:58:53 2015

Last change: Tue Jun 23 15:58:46 2015 via cibadmin on node1

Stack: classic openais (with plugin)

Current DC: node1 - partition with quorum

Version: 1.1.10-14.el6-368c726

2 Nodes configured, 2 expected votes

2 Resources configured

 

Online: [ node1 node2 ]

 

 webstore      (ocf::heartbeat:Filesystem):   Started node1

 httpd (lsb:httpd):    Started node1

 

5.添加虚拟IP资源，并设置虚拟IP必须和httpd服务在一起，httpd服务启动后，才能启动虚拟IP

crm(live)configure# primitive webip ocf:heartbeat:IPaddr params \

  > ip=192.168.10.100 nic=eth0

crm(live)configure# colocation webip_with_httpd inf: webip httpd

crm(live)configure# order httpd_before_webip Mandatory: httpd webip

crm(live)configure# verify

crm(live)configure# commit

crm(live)configure# cd

crm(live)# status

Last updated: Tue Jun 23 16:02:03 2015

Last change: Tue Jun 23 16:01:54 2015 via cibadmin on node1

Stack: classic openais (with plugin)

Current DC: node1 - partition with quorum

Version: 1.1.10-14.el6-368c726

2 Nodes configured, 2 expected votes

3 Resources configured

 

Online: [ node1 node2 ]

 

 webstore      (ocf::heartbeat:Filesystem):   Started node1

 httpd (lsb:httpd):    Started node1

 webip (ocf::heartbeat:IPaddr):       Started node1

 

 

五：高可用测试

1.使node1离线后，查看集群状态

[root@node1 ~]# crm node standby

[root@node1 ~]# crm status

Last updated: Tue Jun 23 16:05:40 2015

Last change: Tue Jun 23 16:05:37 2015 viacrm_attribute on node1

Stack: classic openais (with plugin)

Current DC: node1 - partition with quorum

Version: 1.1.10-14.el6-368c726

2 Nodes configured, 2 expected votes

3 Resources configured

 

Node node1: standby

Online: [ node2 ]

 

 webstore      (ocf::heartbeat:Filesystem):   Started node2

 httpd (lsb:httpd):    Started node2

 webip (ocf::heartbeat:IPaddr):       Started node2

 

由以上可知，资源切换到了node2上

 

2.使node1重新上线

[root@node1 ~]# crm node online

[root@node1 ~]# crm status

Last updated: Tue Jun 23 16:06:43 2015

Last change: Tue Jun 23 16:06:40 2015 viacrm_attribute on node1

Stack: classic openais (with plugin)

Current DC: node1 - partition with quorum

Version: 1.1.10-14.el6-368c726

2 Nodes configured, 2 expected votes

3 Resources configured

 

Online: [ node1 node2 ]

 

 webstore      (ocf::heartbeat:Filesystem):   Started node1

 httpd (lsb:httpd):    Started node1

 webip (ocf::heartbeat:IPaddr):       Started node1

 

由以上可知，资源又回到了node1，这和我们设置的优先运行在node1上相符

 

至此一个简单的web高可用配置完成

本文出自 “永不止步” 博客，谢绝转载！