使用资源管理工具crmsh配置web高可用服务器

使用资源管理工具crmsh配置web高可用服务器

框架：crmsh(Corosync+pacemaker)+httpd

集群节点1:192.168.88.132 cen7.field.com

集群节点2:192.168.88.133 node2.field.com

vip: 192.168.88.88

web服务：192.168.88.132 cen7.field.com

 

1、crmsh安装及使用简要说明

1）.rpm包安装crmsh

可以从以下网址获取crmsh安装包                       

http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-7/noarch/

http://rpm.pbone.net/index.php3?stat=3&limit=2&srodzaj=3&dl=40&search=python-parallax

[root@cen7 ~]# ll

总用量 956

-rw-------. 1 root root   1420 6月  13 20:50 anaconda-ks.cfg

-rw-r--r--  1 root root 763644 8月   1 20:27 crmsh-3.0.0-6.2.noarch.rpm

-rw-r--r--  1 root root  95480 8月   1 20:34 crmsh-scripts-3.0.0-6.2.noarch.rpm

drwx------. 3 root root     22 6月  13 23:09 ~None

-rw-r--r--  1 root root  27316 8月   1 20:27 pssh-2.3.1-7.3.noarch.rpm

-rw-r--r--  1 root root  34628 8月   1 20:36 python-parallax-1.0.0a1-7.1.noarch.rpm

-rw-r--r--  1 root root  43268 8月   1 20:27 python-pssh-2.3.1-7.3.noarch.rpm

[root@cen7 ~]# yum install *.rpm

已安装:

  crmsh.noarch 0:3.0.0-6.2         crmsh-scripts.noarch 0:3.0.0-6.2         pssh.noarch 0:2.3.1-7.3         python-parallax.noarch 0:1.0.0a1-7.1         python-pssh.noarch 0:2.3.1-7.3       

作为依赖被安装:

  python-dateutil.noarch 0:1.5-7.el7                                                                 rsync.x86_64 0:3.1.2-4.el7                                                               

完毕！

2）.crm用法简要说明

[root@cen7 ~]# crm help

[root@cen7 ~]# crm status

Stack: corosync

Current DC: cen7.field.com (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition with quorum

Last updated: Thu Aug  2 05:05:12 2018

Last change: Thu Aug  2 04:07:34 2018 by root via cibadmin on cen7.field.com

2 nodes configured

0 resources configured

Online: [ cen7.field.com node2.field.com ]

No resources

[root@node2 ~]# crm status

Stack: corosync

Current DC: cen7.field.com (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition with quorum

Last updated: Thu Aug  2 05:06:02 2018

Last change: Thu Aug  2 04:07:34 2018 by root via cibadmin on cen7.field.com

2 nodes configured

0 resources configured

Online: [ cen7.field.com node2.field.com ]

No resources

crm交互式模式：

[root@cen7 ~]# crm

#使用help查看帮助

crm(live)# help

crm(live)# resource

#资源模式下可以查看各定义的资源及操作方法

crm(live)resource# help

crm(live)resource# cd ..

crm(live)# configure

#configure 用于设置具体参数

#configure模式下使用show命令查看配置

crm(live)configure# show

node 1: cen7.field.com

node 2: node2.field.com

property cib-bootstrap-options: \

         have-watchdog=false \

         dc-version=1.1.18-11.el7_5.3-2b07d5c5a9 \

         cluster-infrastructure=corosync \

         cluster-name=mycluster \

         stonith-enabled=false

crm(live)configure# edit

#使用edit直接编辑，完成后:wq保存退出

#edit ，id为该项的名称

crm(live)configure# help

crm(live)configure# help primitive

#primitive：主资源，原始资源；在集群中只能运行一个实例；

crm(live)configure#

其它参数的使用方法均可使用help [option]查看

2、安装httpd，配置web服务器

[root@cen7 ~]# yum install httpd -y

[root@cen7 ~]# echo "

cen7.field.com

" >/var/www/html/index.html

[root@cen7 ~]# systemctl start httpd.service

[root@cen7 ~]# curl 192.168.88.133

node2.field.com

[root@node2 ~]# yum -y install httpd

建立测试页

[root@node2 ~]# echo "

node2.field.com

" >/var/www/html/index.html

[root@node2 ~]# systemctl start httpd.service

[root@node2 ~]# curl 192.168.88.132

cen7.field.com

3、配置Corosync+pacemaker+httpd集群

使用crmsh交互模式配置

crm(live)# cd ra

crm(live)ra# help

Resource Agents (RA) lists and documentation

This level contains commands which show various information about

the installed resource agents. It is available both at the top

level and at the configure level.

Commands:

         classes        List classes and providers

         info           Show meta data for a RA

         list           List RA for a class (and provider)

         providers      Show providers for a RA and a class

         validate       Validate parameters for RA

 

         cd             Navigate the level structure

         help           Show help (help topics for list of topics)

         ls             List levels and commands

         quit           Exit the interactive shell

         up             Go back to previous level

crm(live)# ra

#resource agents, 查看哪些可供使用的资源代理

crm(live)ra# classes

#资源代理类型：lsb, ocf, systemd, service，资源代理的提供程序：heartbeat , pacemaker

lsb

ocf / .isolation heartbeat openstack pacemaker

service

systemd

crm(live)ra# help list

Example:

list ocf pacemaker

crm(live)ra# list lsb

netconsole  network    

crm(live)ra# list systemd

NetworkManager                                  NetworkManager-dispatcher

....

httpd-->可以在systemd资源代理中找到httpd     initrd-cleanup

.....

wpa_supplicant  

crm(live)ra# list ocf

#list命令

crm(live)ra# info ocf:IPaddr

#info命令：查看资源代理相关信息

crm(live)ra# info ocf:IPaddr2

crm(live)ra# cd

crm(live)# configure

crm(live)configure# help primitive

1）、定义VIP资源

crm(live)configure# primitive webip ocf:heartbeat:IPaddr params ip=192.168.88.88

#primitive定义资源

crm(live)configure# show

node 1: cen7.field.com

node 2: node2.field.com

primitive webip IPaddr \

         params ip=192.168.88.88

property cib-bootstrap-options: \

         have-watchdog=false \

         dc-version=1.1.18-11.el7_5.3-2b07d5c5a9 \

         cluster-infrastructure=corosync \

         cluster-name=mycluster \

         stonith-enabled=false

crm(live)configure# verify

#检查当前配置是否正确, 相当于crm_verify -L

crm(live)configure# commit

#提交配置使配置生效，修改后没提交系统不会保存更改的信息

crm(live)configure# cd

crm(live)# status

#status显示集群状态信息

Stack: corosync

Current DC: cen7.field.com (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition with quorum

Last updated: Thu Aug  2 05:35:01 2018

Last change: Thu Aug  2 05:34:49 2018 by root via cibadmin on node2.field.com

2 nodes configured

1 resource configured

Online: [ cen7.field.com node2.field.com ]

Full list of resources:

 webip     (ocf::heartbeat:IPaddr):  Started cen7.field.com

[root@cen7 ~]# ip addr list |grep ens32

2: ens32: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000

    inet 192.168.88.132/24 brd 192.168.88.255 scope global noprefixroute ens32

    inet 192.168.88.88/24 brd 192.168.88.255 scope global secondary ens32

#可以看到，VIP资源已经配置成功并且启动在cen7接点上   

[root@cen7 ~]# crm

crm(live)# cd node

#node：对节点进行各类操作

crm(live)node# help

crm(live)node# stanby

#standby可以将当前节点转为备用状态

crm(live)node# cd

crm(live)# status

Stack: corosync

Current DC: cen7.field.com (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition with quorum

Last updated: Thu Aug  2 05:47:15 2018

Last change: Thu Aug  2 05:47:09 2018 by root via crm_attribute on cen7.field.com

2 nodes configured

1 resource configured

Node cen7.field.com: standby

Online: [ node2.field.com ]

Full list of resources:

 webip     (ocf::heartbeat:IPaddr):  Started node2.field.com

crm(live)#

[root@node2 ~]# ip addr list | grep ens33

2: ens33: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000

    inet 192.168.88.133/24 brd 192.168.88.255 scope global noprefixroute dynamic ens33

    inet 192.168.88.88/24 brd 192.168.88.255 scope global secondary ens33

#可以看到，当前节点转为备用状态后vip流转到node2节点

[root@cen7 ~]# crm

crm(live)# cd node

crm(live)node# online

#online命令可以使备用节点重新上线

crm(live)node# cd

crm(live)# cd ra

crm(live)ra# info systemd:httpd

#info命令查看httpd资源代理相关信息

systemd unit file for emd:httpd (systemd:httpd)

emd:httpd.service

Operations' defaults (advisory minimum):

    start         timeout=100

    stop          timeout=100

    status        timeout=100

    monitor       timeout=100 interval=60

crm(live)ra# cd

crm(live)# configure

2）、定义httpd资源

crm(live)configure# primitive webserver systemd:httpd

crm(live)configure# verify

crm(live)configure# commit

crm(live)configure# cd

crm(live)# status

Stack: corosync

Current DC: cen7.field.com (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition with quorum

Last updated: Thu Aug  2 05:51:50 2018

Last change: Thu Aug  2 05:51:44 2018 by root via crm_attribute on cen7.field.com

2 nodes configured

2 resources configured

Online: [ cen7.field.com node2.field.com ]

Full list of resources:

 webip     (ocf::heartbeat:IPaddr):  Started node2.field.com

 webserver      (systemd:httpd):      Started cen7.field.com

#未配置组资源或约束时，各个资源顺序启动在各节点上。

将VIP资源和httpd资源设置为组资源，使其启动在同一节点上

3）、将vip资源和httpd资源定义为同个组

crm(live)configure# group webservice webip webserver

#group创建组资源，将几项资源编成一组

crm(live)configure# verify

crm(live)configure# commit

crm(live)configure# cd

crm(live)# status

Stack: corosync

Current DC: cen7.field.com (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition with quorum

Last updated: Thu Aug  2 05:55:27 2018

Last change: Thu Aug  2 05:55:23 2018 by root via cibadmin on cen7.field.com

2 nodes configured

2 resources configured

Online: [ cen7.field.com node2.field.com ]

Full list of resources:

 Resource Group: webservice

     webip     (ocf::heartbeat:IPaddr):  Started node2.field.com

     webserver      (systemd:httpd):      Starting node2.field.com

#配置完成，可以看到，两个资源均启动在node2节点上

4、集群测试

测试1、访问：192.168.88.88

[root@cen7 ~]# curl 192.168.88.88

node2.field.com

[root@node2 ~]# curl 192.168.88.88

node2.field.com

测试2：将node2节点转为备用状态，测试资源流转是否正常

[root@node2 ~]# crm node standby

[root@node2 ~]# crm status

Stack: corosync

Current DC: cen7.field.com (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition with quorum

Last updated: Thu Aug  2 05:58:00 2018

Last change: Thu Aug  2 05:57:49 2018 by root via crm_attribute on node2.field.com

2 nodes configured

2 resources configured

Node node2.field.com: standby

Online: [ cen7.field.com ]

Full list of resources:

 Resource Group: webservice

     webip     (ocf::heartbeat:IPaddr):  Started cen7.field.com

     webserver      (systemd:httpd):      Started cen7.field.com

#可以看到，node2转为备用状态后，资源流转到cen7节点    

[root@node2 ~]# curl 192.168.88.88

cen7.field.com

测试3：重新让node2上线，未定义约束时，不会抢回资源

[root@node2 ~]# crm node online

[root@node2 ~]# crm status

Stack: corosync

Current DC: cen7.field.com (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition with quorum

Last updated: Thu Aug  2 05:59:11 2018

Last change: Thu Aug  2 05:59:07 2018 by root via crm_attribute on node2.field.com

2 nodes configured

2 resources configured

Online: [ cen7.field.com node2.field.com ]

Full list of resources:

 Resource Group: webservice

     webip     (ocf::heartbeat:IPaddr):  Started cen7.field.com

     webserver      (systemd:httpd):      Started cen7.field.com

[root@node2 ~]#