使用资源管理工具pcs配置Corosync+pacemaker

使用资源管理工具pcs配置Corosync+pacemaker

一、Corosync+pacemaker架构

1、高可用集群结构层：(从下往上)

1).Messaging Layer:消息层，传递心跳以及集群事务信息

  corosync v1, v2(votequorum)

2).CRM:Cluster Resource Manager:集群资源管理器

作用：使本身不具备的高可用能力的服务也能够运行在高可用集群中，并以高可用模式提供服务

传递方式：CRM-->LRM:Local Resource Manager -->Pengine：Policy Engine(只运行在主节点上，决策)

    CIB:Cluster Information Base:集群信息库

集群的全生命周期管理工具：

pcs: agent(pcsd)

crmsh: agentless (pssh)

独立服务：

pacemaker:CRM-->LRM-->Pengine-->CIB

  重要的基本组件：cib/crmd/lrmd/pengine

  配置接口：crmsh (agentless), pssh 并行ssh

  pcs (agent、a/s), pcsd：由pcsd守护进程负责接收pcs发出的命令并执行

  实现方式：group(资源组、资源类型), constraint(约束)

3).RA: Resource Agent:资源代理

作用：运行在各节点中的crm中的lrm负责管理本地资源，包括资源的启动、停止、重启、监控等（主要是一些控制脚本）

脚本实现方式：

LSB: /etc/rc.d/init.d/目录下的脚本； centos5/6/7

systemd：/usr/lib/systemd/system 真正读取：/etc/systemd/system/multi-user.wants 处于enable状态的服务；

OCF:Open Cluster Framework开放集群框架，公司或组织自行研发的脚本： [provider]

  heartbeat、pacemaker、linbit

service：通用服务类型: /etc/ha.d/haresources.d/目录下的脚本；

stonith：STONITH：shoot the other node on the head  节点级别隔离：专用于stonith设备的类别

  Fence: 资源级别的隔离

2、集群资源定义

1).资源类型：

primitive：主资源，原始资源；在集群中只能运行一个实例；

clone：克隆资源，在集群中可运行多个实例；

  匿名克隆、全局惟一克隆、状态克隆(主动、被动)

  multi-state(master/slave)：克隆资源的特殊实现；多状态资源；

group: 组资源；

  启动或停止；资源监视；相关性：

2).资源属性：

priority: 优先级；

target-role：started, stopped, master;

is-managed: 是否允许集群管理此资源；

resource-stickiness: 资源粘性；

allow-migrate: 是否允许迁移；

3).约束：constraint

#“inf:”表示无穷大，以下配置表示mysqlip和(mynfs mysqld)间的黏性值无穷大，即三个资源始终在同一节点上运行。

#此外“-inf:”表示黏性值负无穷，定义-inf:的两个资源始终不在同一节点运行。

#mandatory默认值，表示强制约束，

location位置约束：资源对节点的倾向性,指定资源首选在哪些节点上运行

  (-oo, +oo):

  任何值+无穷大=无穷大

  任何值+负无穷大=负无穷大

  无穷大+负无穷大=负无穷大

colocation排列约束:资源彼此间是否能运行于同一节点的倾向性；指定哪些资源捆绑一起，在同一节点上运行

  (-oo, +oo)    

order顺序约束:多个资源启动顺序依赖关系;指定排列约束中的资源启动顺序，该顺序和colocation顺序相反      

  (-oo, +oo),Mandatory

3、配置属性

全局属性：property, stonith-enable等等；

高可用服务：资源，通过RA

webip, webstore, webserver

  节点粘性：设置默认值m,共n个节点，则当前节点粘性为n*m

  如：定义node1粘性100，其它为0，则各节点黏性值

  node1: 100 + 0 + 0

  node2: 0 + 0 + 0

  node3: 0 + 0 + 0

4、pcs：集群的全生命周期管理工具：agent(pcsd)

属性定义参数：

cluster：集群

  auth

  setup

resource：资源

  describe 描述

  list 列出

  create 创建

  delete 删除

constraint：约束

  colocation 倾向性

  order 顺序

  location 位置

property  全局属性

  list 列出

  set 设置

  status：状态查看

  config：配置

5、配置基础

CentOS 7： corosync v2 + pacemaker

         corosync v2: vote system

         pacemaker: 独立服务

配置集群的前提

(1).时间同步；

(2).基于当前正在使用的主机名互相访问；

(3).是否会用到仲裁设备；

 

二、配置pcs安装Corosync和pacemaker

1、配置集群的前提

1).设定各节点可以基于密钥进行ssh通信

[root@cen7 ~]# ssh-keygen -t rsa

[root@cen7 ~]# cd ~/.ssh

[root@cen7 .ssh]# ll

总用量 16

-rw-r--r--. 1 root root  401 6月  13 22:29 authorized_keys

-rw-------. 1 root root 1675 6月  13 22:28 id_rsa

-rw-r--r--. 1 root root  401 6月  13 22:28 id_rsa.pub

-rw-r--r--. 1 root root 1759 6月  13 23:08 known_hosts

[root@cen7 .ssh]# chmod 600 authorized_keys

[root@cen7 .ssh]# scp authorized_keys 192.168.88.133:/root/.ssh

2).确保时间点同步

[root@cen7 ~]# date;ssh node2 'date'

2018年 08月 02日 星期四 01:36:35 CST

2018年 08月 02日 星期四 01:36:36 CST

3).确保网络顺畅

[root@cen7 ~]# ping cen7.field.com

PING localhost (127.0.0.1) 56(84) bytes of data.

64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.127 ms

64 bytes from localhost (127.0.0.1): icmp_seq=2 ttl=64 time=0.069 ms

^C

--- localhost ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 1000ms

rtt min/avg/max/mdev = 0.069/0.098/0.127/0.029 ms

[root@cen7 ~]# ping node2.field.com

PING node2.field.com (192.168.88.133) 56(84) bytes of data.

64 bytes from node2.field.com (192.168.88.133): icmp_seq=1 ttl=64 time=1.46 ms

64 bytes from node2.field.com (192.168.88.133): icmp_seq=2 ttl=64 time=0.643 ms

^C

--- node2.field.com ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 1002ms

rtt min/avg/max/m

4).所有节点的主机名称和对应的IP地址解析服务可以正常工作，且每个节点的主机名称需要跟"uname -n"命令的结果保持一致；

[root@cen7 ~]# uname -n ;ssh node2 'uname -n'

cen7.field.com

node2.field.com

因此，需要保证两个节点上的/etc/hosts文件均为下面的内容：

192.168.88.132   cen7.field.com cen7

192.168.88.133   node2.field.com node2

为了使得重新启动系统后仍能保持如上的主机名称，还分别需要在各节点执行类似如下的命令：

cen7:

[root@cen7 ~]#  sed -i 's@\(HOSTNAME=\).*@\1cen7.field.com@g'  /etc/sysconfig/network

[root@cen7 ~]#  hostname cen7.field.com

Node2：

[root@node2 ~]#  sed -i 's@\(HOSTNAME=\).*@\1node2.field.com@g' /etc/sysconfig/network

[root@node2 ~]#  hostname node2.field.com

2、yum安装pcs

[root@cen7 ~]# cat /etc/resolv.conf                         

nameserver 8.8.8.8

nameserver 172.28.1.26

nameserver 172.28.1.28         

[root@cen7 ~]# yum install pcs -y                    

                   已安装:

  pcs.x86_64 0:0.9.162-5.el7.centos.1                                                                                                                                                         

作为依赖被安装:

  avahi-libs.x86_64 0:0.6.31-19.el7           bc.x86_64 0:1.06.95-13.el7                       cifs-utils.x86_64 0:6.2-10.el7                     clufter-bin.x86_64 0:0.77.0-2.el7          

  clufter-common.noarch 0:0.77.0-2.el7        corosync.x86_64 0:2.4.3-2.el7_5.1                corosynclib.x86_64 0:2.4.3-2.el7_5.1               cups-libs.x86_64 1:1.6.3-35.el7            

  device-mapper-event.x86_64 7:1.02.146-4.el7 device-mapper-event-libs.x86_64 7:1.02.146-4.el7 device-mapper-persistent-data.x86_64 0:0.7.3-3.el7 gnutls.x86_64 0:3.3.26-9.el7               

  gssproxy.x86_64 0:0.7.0-17.el7              keyutils.x86_64 0:1.5.8-3.el7                    libaio.x86_64 0:0.3.109-13.el7                     libbasicobjects.x86_64 0:0.1.1-29.el7      

  libcgroup.x86_64 0:0.41-15.el7              libcollection.x86_64 0:0.7.0-29.el7              liberation-fonts-common.noarch 1:1.07.2-16.el7     liberation-sans-fonts.noarch 1:1.07.2-16.el7

  libevent.x86_64 0:2.0.21-4.el7              libini_config.x86_64 0:1.3.1-29.el7              libldb.x86_64 0:1.2.2-1.el7                        libnfsidmap.x86_64 0:0.25-19.el7           

  libpath_utils.x86_64 0:0.2.1-29.el7         libqb.x86_64 0:1.0.1-6.el7                       libref_array.x86_64 0:0.1.5-29.el7                 libtalloc.x86_64 0:2.1.10-1.el7            

  libtdb.x86_64 0:1.3.15-1.el7                libtevent.x86_64 0:0.9.33-2.el7                  libtirpc.x86_64 0:0.2.4-0.10.el7                   libverto-libevent.x86_64 0:0.2.5-4.el7     

  libwbclient.x86_64 0:4.7.1-6.el7            lvm2.x86_64 7:2.02.177-4.el7                     lvm2-libs.x86_64 7:2.02.177-4.el7                  net-snmp-libs.x86_64 1:5.7.2-33.el7_5.2    

  net-tools.x86_64 0:2.0-0.22.20131004git.el7 nettle.x86_64 0:2.7.1-8.el7                      nfs-utils.x86_64 1:1.3.0-0.54.el7                  overpass-fonts.noarch 0:2.1-1.el7          

  pacemaker.x86_64 0:1.1.18-11.el7_5.3        pacemaker-cli.x86_64 0:1.1.18-11.el7_5.3         pacemaker-cluster-libs.x86_64 0:1.1.18-11.el7_5.3  pacemaker-libs.x86_64 0:1.1.18-11.el7_5.3  

  perl-TimeDate.noarch 1:2.30-2.el7           psmisc.x86_64 0:22.20-15.el7                     python-clufter.noarch 0:0.77.0-2.el7               python-lxml.x86_64 0:3.2.1-4.el7           

  quota.x86_64 1:4.01-17.el7                  quota-nls.noarch 1:4.01-17.el7                   resource-agents.x86_64 0:3.9.5-124.el7             rpcbind.x86_64 0:0.2.0-44.el7              

  ruby.x86_64 0:2.0.0.648-33.el7_4            ruby-irb.noarch 0:2.0.0.648-33.el7_4             ruby-libs.x86_64 0:2.0.0.648-33.el7_4              rubygem-bigdecimal.x86_64 0:1.2.0-33.el7_4 

  rubygem-io-console.x86_64 0:0.4.2-33.el7_4  rubygem-json.x86_64 0:1.7.7-33.el7_4             rubygem-psych.x86_64 0:2.0.0-33.el7_4              rubygem-rdoc.noarch 0:4.0.0-33.el7_4       

  rubygems.noarch 0:2.0.14.1-33.el7_4         samba-client-libs.x86_64 0:4.7.1-6.el7           samba-common.noarch 0:4.7.1-6.el7                  samba-common-libs.x86_64 0:4.7.1-6.el7     

  tcp_wrappers.x86_64 0:7.6-77.el7            trousers.x86_64 0:0.3.14-2.el7                 

完毕！

可以发现，yum安装pcs后默认安装了众多依赖包，包括Corosync和pacemaker

3、使用pcs配置Corosync和pacemaker，配置集群

方便配置，在ansible中定义ha                

[root@cen7 ~]# vim /etc/ansible/hosts

[ha]

192.168.88.132

192.168.88.133

1）.启动集群中的pscd服务

[root@cen7 ~]# ansible ha -m service -a 'name=pcsd state=started enabled=yes'

192.168.88.132 | SUCCESS => {

    "changed": true,

    "enabled": true,

    "name": "pcsd",

    "state": "started",

    "status": {

.....

    }

}

192.168.88.133 | SUCCESS => {

    "changed": true,

    "enabled": true,

    "name": "pcsd",

    "state": "started",

    "status": {

......

    }

}

2）.确认pcsd进程运行

[root@cen7 ~]# systemctl status pcsd

● pcsd.service - PCS GUI and remote configuration interface

   Loaded: loaded (/usr/lib/systemd/system/pcsd.service; enabled; vendor preset: disabled)

   Active: active (running) since 四 2018-08-02 01:53:52 CST; 49s ago

     Docs: man:pcsd(8)

           man:pcs(8)

 Main PID: 2122 (pcsd)

   CGroup: /system.slice/pcsd.service

           └─2122 /usr/bin/ruby /usr/lib/pcsd/pcsd > /dev/null &

8月 02 01:53:49 cen7.field.com systemd[1]: Starting PCS GUI and remote configuration interface...

8月 02 01:53:52 cen7.field.com systemd[1]: Started PCS GUI and remote configuration interface...

3）.修改pcs安装时默认用户hacluster密码，两个节点密码设置一致。

[root@cen7 home]# ansible ha -m shell -a 'echo hacluster |passwd --stdin hacluster'

192.168.88.132 | SUCCESS | rc=0 >>

更改用户 hacluster 的密码 。

passwd：所有的身份验证令牌已经成功更新。

192.168.88.133 | SUCCESS | rc=0 >>

更改用户 hacluster 的密码 。

passwd：所有的身份验证令牌已经成功更新。

4）.集群节点间认证

注意认证时用户必须是pcs安装时的默认用户hacluster，否则无法认证通过

[root@cen7 home]# pcs cluster auth cen7.field.com node2.field.com -u hacluster

Password:

cen7.field.com: Authorized

Error: Unable to communicate with node2.field.com

此处认证不通过，逐一排查原因，可能是：iptables、setLinux、Firewall，centos7默认开启Firewall

[root@node2 home]# uname -n

node2.field.com

[root@node2 home]# service iptables stop

Redirecting to /bin/systemctl stop iptables.service

Failed to stop iptables.service: Unit iptables.service not loaded.

[root@node2 home]# getenforce

Disabled

[root@node2 home]# systemctl enable pcsd.service

[root@node2 home]# firewall-cmd  --state

Running

#centos7默认开启Firewall

[root@node2 home]# systemctl  stop  firewalld

[root@node2 home]# firewall-cmd  --state

not running

[root@node2 home]# chkconfig firewalld off

注意：正在将请求转发到“systemctl disable firewalld.service”。

Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.

Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

关闭firewalld后认证通过

[root@cen7 home]# pcs cluster auth cen7.field.com node2.field.com -u hacluster

Password:

cen7.field.com: Authorized

node2.field.com: Authorized

5）.创建并启动集群

[root@cen7 home]# pcs cluster setup --name mycluster cen7.field.com node2.field.com

Destroying cluster on nodes: cen7.field.com, node2.field.com...

cen7.field.com: Stopping Cluster (pacemaker)...

node2.field.com: Stopping Cluster (pacemaker)...

node2.field.com: Successfully destroyed cluster

cen7.field.com: Successfully destroyed cluster

Sending 'pacemaker_remote authkey' to 'cen7.field.com', 'node2.field.com'

cen7.field.com: successful distribution of the file 'pacemaker_remote authkey'

node2.field.com: successful distribution of the file 'pacemaker_remote authkey'

Sending cluster config files to the nodes...

cen7.field.com: Succeeded

node2.field.com: Succeeded

Synchronizing pcsd certificates on nodes cen7.field.com, node2.field.com...

cen7.field.com: Success

node2.field.com: Success

Restarting pcsd on the nodes in order to reload the certificates...

cen7.field.com: Success

node2.field.com: Success

由以上信息，可以看到已经启动成功

[root@cen7 home]# cd /etc/corosync/

[root@cen7 corosync]# ls

corosync.conf  corosync.conf.example  corosync.conf.example.udpu  corosync.xml.example  uidgid.d

6）.编辑corosync.conf，配置如下：（单播配置）

说明：此处只是简单配置，使集群启动。

[root@cen7 corosync]# cat corosync.conf

totem {

    version: 2

    cluster_name: mycluster

    secauth: off

    transport: udpu

#传播协议：单播传输参数udpu

}

#在 nodelist 之下可以为某一节点设置只与该节点相关的信息，这些设置项只能包含在 node 之中，即只能对属于集群的节点服务器进行设置，

#而且只应包括那些与默认设置不同的参数。每台服务器都必须配置 ring0_addr。

nodelist {

    node {

        ring0_addr: cen7.field.com

        nodeid: 1

    }

    node {

        ring0_addr: node2.field.com

        nodeid: 2

    }

}

quorum {

    provider: corosync_votequorum

    two_node: 1

}

logging {

    to_logfile: yes

    logfile: /var/log/cluster/corosync.log

    to_syslog: yes

}

7）.生成节点间通信时用到的认证密钥文件

[root@cen7 corosync]# corosync-keygen

[root@cen7 corosync]# scp corosync.conf authkey node2:/etc/corosync

8）.启动集群所有节点

[root@cen7 corosync]# pcs cluster start --all

cen7.field.com: Starting Cluster...

node2.field.com: Starting Cluster...

9）.确认pcsd进程运行

[root@cen7 corosync]# ps -aux |tail

root      5857  6.7 22.1 196292 95992 ?        SLsl 03:34   0:06 corosync

root      5864  0.3  1.6 104252  7196 ?        Ss   03:34   0:00 /usr/sbin/pacemakerd -f

haclust+  5865  1.8  3.2 106016 13996 ?        Ss   03:34   0:01 /usr/libexec/pacemaker/cib

root      5866  0.3  1.3 106764  6004 ?        Ss   03:34   0:00 /usr/libexec/pacemaker/stonithd

root      5867  0.1  1.0  98776  4364 ?        Ss   03:34   0:00 /usr/libexec/pacemaker/lrmd

haclust+  5868  0.4  1.2 127960  5620 ?        Ss   03:34   0:00 /usr/libexec/pacemaker/attrd

haclust+  5869  1.9  4.5 116520 19824 ?        Ss   03:34   0:01 /usr/libexec/pacemaker/pengine

haclust+  5870  1.7  1.9 181324  8604 ?        Ss   03:34   0:01 /usr/libexec/pacemaker/crmd

root      5892  1.0  0.4 155324  1880 pts/0    R+   03:35   0:00 ps -aux

root      5893  0.0  0.1 108200   664 pts/0    S+   03:35   0:00 tail

查看corosync-cfgtool工具使用帮助

[root@cen7 corosync]# corosync-cfgtool --help

corosync-cfgtool: invalid option -- '-'

corosync-cfgtool [-i ] [-s] [-r] [-R] [-k nodeid] [-a nodeid] [-h] [-H]

A tool for displaying and configuring active parameters within corosync.

options:

         -i       Finds only information about the specified interface IP address.

         -s      Displays the status of the current rings on this node.

         -r      Reset redundant ring state cluster wide after a fault to

                   re-enable redundant ring operation.

         -R     Tell all instances of corosync in this cluster to reload corosync.conf.

         -k      Kill a node identified by node id.

         -a      Display the IP address(es) of a node

         -h      Print basic usage.

         -H     Shutdown corosync cleanly on this node.

corosync-cfgtool: invalid option -- 'e'

corosync-cfgtool: invalid option -- 'l'

corosync-cfgtool: invalid option -- 'p'

[root@cen7 corosync]# corosync-cfgtool -s

Printing ring status.

Local node ID 1

RING ID 0

         id      = 127.0.0.1 

         status       = ring 0 active with no faults

[root@cen7 corosync]#

注意：id = 127.0.0.1  表示集群失败，使用“pcs status”可以发现Node node2.field.com: UNCLEAN (offline)

原因是/etc/hosts中配置了127.0.0.1

编辑两台机的/etc/hosts,注销如下行并重启

[root@cen7 ~]# vim /etc/hosts

#127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 cen7.field.com

#::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.88.132   cen7.field.com cen7

192.168.88.133    node2.field.com node2

10）.查看集群节点的启动状态

[root@cen7 corosync]# pcs status

Cluster name: mycluster

WARNING: no stonith devices and stonith-enabled is not false

Stack: corosync

Current DC: cen7.field.com (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition WITHOUT quorum

Last updated: Thu Aug  2 03:47:09 2018

Last change: Thu Aug  2 03:34:33 2018 by hacluster via crmd on cen7.field.com

2 nodes configured

0 resources configured

Node node2.field.com: UNCLEAN (offline)

Online: [ cen7.field.com ]

No resources

Daemon Status:

  corosync: active/disabled

  pacemaker: active/disabled

从上面的信息可以看出只启动一个节点，处于不正常工作状态。    

[root@cen7 corosync]# reboot

可以看到，重启已经正常启动节点

[root@cen7 ~]# pcs cluster start --all

cen7.field.com: Starting Cluster...

node2.field.com: Starting Cluster...

[root@cen7 ~]# corosync-cfgtool -s

Printing ring status.

Local node ID 1

RING ID 0

         id      = 192.168.88.132

         status       = ring 0 active with no faults

[root@cen7 ~]# corosync-cmapctl |grep members

runtime.totem.pg.mrp.srp.members.1.config_version (u64) = 0

runtime.totem.pg.mrp.srp.members.1.ip (str) = r(0) ip(192.168.88.132)

runtime.totem.pg.mrp.srp.members.1.join_count (u32) = 1

runtime.totem.pg.mrp.srp.members.1.status (str) = joined

runtime.totem.pg.mrp.srp.members.2.config_version (u64) = 0

runtime.totem.pg.mrp.srp.members.2.ip (str) = r(0) ip(192.168.88.133)

runtime.totem.pg.mrp.srp.members.2.join_count (u32) = 1

runtime.totem.pg.mrp.srp.members.2.status (str) = joined

[root@cen7 ~]# pcs status

Cluster name: mycluster

WARNING: no stonith devices and stonith-enabled is not false

Stack: corosync

Current DC: cen7.field.com (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition with quorum

Last updated: Thu Aug  2 03:59:40 2018

Last change: Thu Aug  2 03:34:33 2018 by hacluster via crmd on cen7.field.com

2 nodes configured

0 resources configured

Online: [ cen7.field.com node2.field.com ]

No resources

Daemon Status:

  corosync: active/disabled

  pacemaker: active/disabled

  pcsd: active/enabled

从上面的信息可以看出两个节点都已经正常启动，并且集群已经处于正常工作状态。   

[root@node2 ~]# corosync-cfgtool -s

Printing ring status.

Local node ID 2

RING ID 0

         id      = 192.168.88.133

         status       = ring 0 active with no faults

[root@node2 ~]# corosync-cmapctl |grep members

runtime.totem.pg.mrp.srp.members.1.config_version (u64) = 0

runtime.totem.pg.mrp.srp.members.1.ip (str) = r(0) ip(192.168.88.132)

runtime.totem.pg.mrp.srp.members.1.join_count (u32) = 1

runtime.totem.pg.mrp.srp.members.1.status (str) = joined

runtime.totem.pg.mrp.srp.members.2.config_version (u64) = 0

runtime.totem.pg.mrp.srp.members.2.ip (str) = r(0) ip(192.168.88.133)

runtime.totem.pg.mrp.srp.members.2.join_count (u32) = 1

runtime.totem.pg.mrp.srp.members.2.status (str) = joined

查看properties帮助

[root@cen7 ~]# pcs properties -h

Usage: pcs [-f file] [-h] [commands]...

Control and configure pacemaker and corosync.

Options:

    -h, --help         Display usage and exit.

    -f file            Perform actions on file instead of active CIB.

    --debug            Print all network traffic and external commands run.

    --version          Print pcs version information. List pcs capabilities if

                       --full is specified.

    --request-timeout  Timeout for each outgoing request to another node in

                       seconds. Default is 60s.

    --force            Override checks and errors, the exact behavior depends on

                       the command. WARNING: Using the --force option is

                       strongly discouraged unless you know what you are doing.

Commands:

    cluster     Configure cluster options and nodes.

    resource    Manage cluster resources.

    stonith     Manage fence devices.

    constraint  Manage resource constraints.

    property    Manage pacemaker properties.

    acl         Manage pacemaker access control lists.

    qdevice     Manage quorum device provider on the local host.

    quorum      Manage cluster quorum settings.

    booth       Manage booth (cluster ticket manager).

    status      View cluster status.

    config      View and manage cluster configuration.

    pcsd        Manage pcs daemon.

    node        Manage cluster nodes.

    alert       Manage pacemaker alerts.

查看所有全局设定

[root@cen7 ~]# pcs property list --all

Cluster Properties:

 batch-limit: 0

 cluster-delay: 60s

 cluster-infrastructure: corosync

 cluster-ipc-limit: 500

 cluster-name: mycluster

 cluster-recheck-interval: 15min

 concurrent-fencing: false

 crmd-finalization-timeout: 30min

 crmd-integration-timeout: 3min

 crmd-transition-delay: 0s

 dc-deadtime: 20s

 dc-version: 1.1.18-11.el7_5.3-2b07d5c5a9

 default-action-timeout: (null)

 default-resource-stickiness: (null)

 election-timeout: 2min

 enable-acl: false

 enable-startup-probes: true

 have-watchdog: false

 is-managed-default: (null)

 load-threshold: 80%

 maintenance-mode: false

 migration-limit: -1

 no-quorum-policy: stop

 node-action-limit: 0

 node-health-base: 0

 node-health-green: 0

 node-health-red: -INFINITY

 node-health-strategy: none

 node-health-yellow: 0

 notification-agent: /dev/null

 notification-recipient:

 pe-error-series-max: -1

 pe-input-series-max: 4000

 pe-warn-series-max: 5000

 placement-strategy: default

 remove-after-stop: false

 shutdown-escalation: 20min

 start-failure-is-fatal: true

 startup-fencing: true

 stonith-action: reboot

 stonith-enabled: true

 stonith-max-attempts: 10

 stonith-timeout: 60s

 stonith-watchdog-timeout: (null)

 stop-all-resources: false

 stop-orphan-actions: true

 stop-orphan-resources: true

 symmetric-cluster: true

11）.检查全局配置是否有问题

[root@cen7 ~]# crm_verify -L -V

   error: unpack_resources: Resource start-up disabled since no STONITH resources have been defined

   error: unpack_resources: Either configure some or disable STONITH with the stonith-enabled option

   error: unpack_resources: NOTE: Clusters with shared data need STONITH to ensure data integrity

Errors found during check: config not valid

以上错误是因为默认开启stonish,配置集群的工作属性，禁用stonith，可消除该错误。

[root@cen7 ~]# pcs property set stonith-enabled=false

pcs property list ：只显示修改项

[root@cen7 ~]# pcs property list

Cluster Properties:

 cluster-infrastructure: corosync

 cluster-name: mycluster

 dc-version: 1.1.18-11.el7_5.3-2b07d5c5a9

 have-watchdog: false

 stonith-enabled: false

[root@cen7 ~]# crm_verify -L -V

[root@cen7 ~]#