Heartbeat2+DRBD

Prerequisites

- Setup Minimal CentOS 5
- be sure that both nodes can resolve correctly names (either through dns or /etc/hosts)
- yum update (as usual … )
- yum install heartbeat drbd kmod-drbd (available in the extras repository)

Current situation
* node1 192.168.0.11/24 , source disc /dev/sdb that will be replicated
* node2 192.168.0.12/24 , target disc /dev/sdb

DRBD Configuration

vi /etc/drbd.conf
global { usage-count no; }
resource repdata {
protocol C;
startup { wfc-timeout 0; degr-wfc-timeout 120; }
disk { on-io-error detach; } # or panic, …
net { cram-hmac-alg “sha1″; shared-secret “Cent0Sru!3z”; } # don’t forget to choose a secret for auth !
syncer { rate 10M; }
on node1 {
device /dev/drbd0;
disk /dev/sdb;
address 192.168.0.11:7788;
meta-disk internal;
}
on node2 {
device /dev/drbd0;
disk /dev/sdb;
address 192.168.0.12:7788;
meta-disk internal;
}
}

scp /etc/drbd.conf root@node2:/etc/

- Initialize the meta-data area on disk before starting drbd (! on both nodes!)
[root@node1 etc]# drbdadm create-md repdata

- start drbd on both nodes (service drbd start)
[root@node1 etc]# service drbd start
[root@node2 etc]# service drbd start

[root@node1 etc]# drbdadm — –overwrite-data-of-peer primary repdata
[root@node1 etc]# watch -n 1 cat /proc/drbd

- we can now format /dev/drbd0 and mount it on node1 : mkfs.ext3 /dev/drbd0 ; mkdir /repdata ; mount /dev/drbd0 /repdata
- create some fake data on node 1 :
[root@node1 etc]# for i in {1..5};do dd if=/dev/zero of=/repdata/file$i bs=1M count=100;done

- now switch manually to the second node :
[root@node1 /]# umount /repdata ; drbdadm secondary repdata
[root@node2 /]# mkdir /repdata ; drbdadm primary repdata ; mount /dev/drbd0 /repdata
[root@node2 /]# ls /repdata/ file1 file2 file3 file4 file5 lost+found
Great, data was replicated …. now let’s delete/add some file :
[root@node2 /]# rm /repdata/file2 ; dd if=/dev/zero of=/repdata/file6 bs=100M count=2

- Now switch back to the first node :
[root@node2 /]# umount /repdata/ ; drbdadm secondary repdata
[root@node1 /]# drbdadm primary repdata ; mount /dev/drbd0 /repdata
[root@node1 /]# ls /repdata/ file1 file3 file4 file5 file6 lost+found

OK … Drbd is working … let’s be sure that it will always be started : chkconfig drbd on

Heartbeat V2 Configuration

vi /etc/ha.d/ha.cf
keepalive 1
deadtime 30
warntime 10
initdead 120
bcast eth0
node node1
node node2
crm yes

vi /etc/ha.d/authkeys (with permissions 600 !!!) :
auth 1
1 sha1 MySecret

Start the heartbeat service on node1 :

[root@node1 ha.d]# service heartbeat start
Starting High-Availability services: [OK]

Check the cluster status :
[root@node1 ha.d]# crm_mon

Replicate now the ha.cf and authkeys to node2 and start heartbeat
[root@node1 ha.d]# scp /etc/ha.d/ha.cf /etc/ha.d/authkeys root@node2:/etc/ha.d/
[root@node2 ha.d]# service heartbeat start

Verify cluster with crm_mon :
=====
Last updated: Wed Sep 12 16:20:39 2007
Current DC: node1.centos.org (6cb712e4-4e4f-49bf-8200-4f15d6bd7385)
2 Nodes configured.
0 Resources configured.
=====
Node: node1 (6cb712e4-4e4f-49bf-8200-4f15d6bd7385): online
Node: node2 (f6112aae-8e2b-403f-ae93-e5fd4ac4d27e): online

vi /var/lib/heartbeat/crm/cib.xml

 <cib
 generated
="false"
 admin_epoch
="0"
 epoch
="25"
 num_updates
="1"
 have_quorum
="true"
 ignore_dtd
="false"
 num_peers
="0"
 cib-last-written
="Sun Sep 16 19:47:18 2007"
 cib_feature_revision
="1.3"
 ccm_transition
="1"
>


   <configuration> 


     <crm_config
/>


     <nodes> 


       <node
 id
="6cb712e4-4e4f-49bf-8200-4f15d6bd7385"
 uname
="node1"
 type
="normal"
/>


       <node
 id
="f6112aae-8e2b-403f-ae93-e5fd4ac4d27e"
 uname
="node2"
 type
="normal"
/>


     </nodes> 


     <resources> 


       <group
 id
="My-DRBD-group"
 ordered
="true"
 collocated
="true"
>


         <primitive
 id
="IP-Addr"
 class
="ocf"
 type
="IPaddr2"
 provider
="heartbeat"
>


           <instance_attributes
 id
="IP-Addr_instance_attrs"
>


             <attributes> 


               <nvpair
 id
="IP-Addr_target_role"
 name
="target_role"
 value
="started"
/>


               <nvpair
 id
="2e967596-73fe-444e-82ea-18f61f3848d7"
 name
="ip"
 value
="192.168.0.110"
/>


             </attributes> 


           </instance_attributes> 


         </primitive> 


         <instance_attributes
 id
="My-DRBD-group_instance_attrs"
>


           <attributes> 


             <nvpair
 id
="My-DRBD-group_target_role"
 name
="target_role"
 value
="started"
/>


           </attributes> 


         </instance_attributes> 


         <primitive
 id
="DRBD_data"
 class
="heartbeat"
 type
="drbddisk"
 provider
="heartbeat"
>


           <instance_attributes
 id
="DRBD_data_instance_attrs"
>


             <attributes> 


               <nvpair
 id
="DRBD_data_target_role"
 name
="target_role"
 value
="started"
/>


               <nvpair
 id
="93d753a8-e69a-4ea5-a73d-ab0d0367f001"
 name
="1"
 value
="repdata"
/>


             </attributes> 


           </instance_attributes> 


         </primitive> 


         <primitive
 id
="FS_repdata"
 class
="ocf"
 type
="Filesystem"
 provider
="heartbeat"
>


           <instance_attributes
 id
="FS_repdata_instance_attrs"
>


             <attributes> 


               <nvpair
 id
="FS_repdata_target_role"
 name
="target_role"
 value
="started"
/>


               <nvpair
 id
="96d659dd-0881-46df-86af-d2ec3854a73f"
 name
="fstype"
 value
="ext3"
/>


               <nvpair
 id
="8a150609-e5cb-4a75-99af-059ddbfbc635"
 name
="device"
 value
="/dev/drbd0"
/>


               <nvpair
 id
="de9706e8-7dfb-4505-b623-5f316b1920a3"
 name
="directory"
 value
="/repdata"
/>


             </attributes> 


           </instance_attributes> 


         </primitive> 


       </group> 


     </resources> 


     <constraints> 


       <rsc_location
 id
="runs_on_pref_node"
 rsc
="My-DRBD-group"
>


         <rule
 id
="prefered_runs_on_pref_node"
 score
="100"
>


           <expression
 attribute
="#uname"
 id
="786ef2b1-4289-4570-8923-4c926025e8fd"
 operation
="eq"
 value
="node1"
/>


         </rule> 


       </rsc_location> 


     </constraints> 


   </configuration> 


 </cib> 

Firewall considerations
You will need to make sure that the nodes can talk on ports:
DRBD: 7788
HEARTBEAT: 694

Other

从haresources生成cib.xml
/usr/local/lib/heartbeat/haresources2cib.py –stout -c /etc/ha.d/ha.cf /etc/ha.d/haresources

1)查看所有资源
crm_resource -L

2)查看资源跑在哪个节点上
crm_resource -W -r DRBD_data

4)启动/停止资源
crm_resource -r DRBD_data -p target_role -v started
crm_resource -r DRBD_data -p target_role -v stopped

5)查看资源在cib.xml中的定义
crm_resource -x -r DRBD_data

6)将资源从当前节点移动向另个节点
crm_resource -M -r DRBD_data

7)将资源移向指定节点
crm_resource -M -r DRBD_data -H node2

8)允许资源回到正常的节点
crm_resource -U -r DRBD_data

9)将资源从CRM中删除
crm_resource -D -r DRBD_data -t primitive

10)将资源组从CRM中删除
crm_resource -D -r My-DRBD-group -t group

11)将资源从CRM中禁用
crm_resource -p is_managed -r DRBD_data -t primitive -v off

12)将资源重新从CRM中启用
crm_resource -p is_managed -r DRBD_data -t primitive -v on

13)重启资源
crm_resource -C -H node2 -r DRBD_data

14)检查所有节点上未在CRM中的资源
crm_resource -P

15)检查指定节点上未在CRM中的资源
crm_resource -P -H node2