HeartBeat+DRBD+NFS 高可用

heartbeat、drbd、nfs全部使用yum安装。

NFS客户端使用的是centos 7 可以实现自动挂载。

此文是测试成功后的总结

主节点：

Last login: Thu Jul 30 09:12:52 2015 from 10.0.0.1
[root@heartbeat1 ~]#
[root@heartbeat1 ~]# cat /etc/redhat-release
CentOS release 6.6 (Final)
[root@heartbeat1 ~]# uname -ra
Linux heartbeat1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15 04:27:16 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
[root@heartbeat1 ~]# tail -3 /etc/hosts
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.0.0.7    heartbeat1
10.0.0.8    heartbeat2
[root@heartbeat1 ~]# tail -3 /etc/rc.local
/sbin/route add -host 10.0.0.26 dev eth1
modprobe drbd
drbdadm up data
[root@heartbeat1 ~]# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0C:29:51:A3:1E 
          inet addr:10.0.0.7  Bcast:10.0.0.255  Mask:255.255.255.0     #备节点ip：10.0.0.8
eth1      Link encap:Ethernet  HWaddr 00:0C:29:51:A3:28 
          inet addr:10.0.10.7  Bcast:10.0.10.255  Mask:255.255.255.0    #备节点心跳ip10.0.10.8

heartbeat 配置文件：（主节点备节点配置相同）
[root@heartbeat1 ha.d]# grep -v ^# /etc/ha.d/authkeys
auth 3
3 md5 Hello!
[root@heartbeat1 ha.d]# grep -v ^# /etc/ha.d/ha.cf
debugfile /var/log/ha-debug
logfile    /var/log/ha-log
logfacility    local0
keepalive 2
deadtime 5
warntime 10
initdead 60
udpport 694
mcast eth1 225.0.0.1 694 1 0
auto_failback on
node heartbeat1
node heartbeat2
[root@heartbeat1 ha.d]# grep -v ^# /etc/ha.d/haresources
heartbeat1      IPaddr::10.0.0.50/24/eth0 drbddisk::data Filesystem::/dev/drbd0::/md1::ext4
[root@heartbeat1 ha.d]#

共享磁盘分区：（主备节点相同）
parted /dev/sdc mklabel gpt
parted /dev/sdc mkpart primary 0 1024
parted /dev/sdc mkpart primary 1025 2146
parted /dev/sdc p    查看分区表

共享磁盘分区信息：（主备节点配置相同）
[root@heartbeat1 ~]# parted /dev/sdc p
Model: VMware, VMware Virtual S (scsi)
Disk /dev/sdc: 2147MB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number  Start   End     Size    File system  Name     Flags
1      17.4kB  1024MB  1024MB  ext4         primary
2      1025MB  2146MB  1121MB               primary

DRBD配置文件：（主备节点配置相同）
[root@heartbeat1 ~]# cat /etc/drbd.d/global_common.conf
global    {
    usage-count no;
}
common    {
    syncer    {
        rate 1M;
        verify-alg crc32c;
        al-extents 517;
    }
}
#primary for drbd1
resource data {
    protocol C;
    disk    {
        on-io-error detach;
        }
    on heartbeat1 {
        device    /dev/drbd0;
        disk    /dev/sdc1;
        address    10.0.0.7:7788;
        meta-disk    /dev/sdc2 [0];
    }
    on heartbeat2    {
        device /dev/drbd0;
        disk    /dev/sdc1;
        address    10.0.0.8:7788;
        meta-disk    /dev/sdc2 [0];
    }
}   
[root@heartbeat1 ~]#

DRBD启动：
第一步初始化：
[root@heartbeat1 ~]# drbdadm create-md data    初始化data磁盘
[root@heartbeat1 ~]# drbdadm up data
[root@heartbeat1 ~]# cat /proc/drbd    如果是Secondary 说明启动正常
启动后进行格式化磁盘：（只是主节点上配置，备节点会自动同步配置）
[root@heartbeat1 drbd.d]# mkfs.ext4 -b 4096 /dev/drbd0 (对应的是/dev/sdc1  ,drbd 主上配置)
[root@heartbeat1 drbd.d]# tune2fs -c -1 /dev/drbd0
tune2fs 1.41.12 (17-May-2010)
Setting maximal mount count to -1

测试磁盘：
mkdir /md1    创建挂载目录
mount /dev/drbd0 /md1    挂载drbd0设备测试
[root@heartbeat1 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
                       35G  4.0G   30G  13% /
tmpfs                 491M     0  491M   0% /dev/shm
/dev/sda1             477M   34M  418M   8% /boot
/dev/drbd0            749M  3.5M  707M   1% /md1    自动挂载点

NFS配置文件：（主备节点相同）
[root@heartbeat1 ~]# cat /etc/exports
/md1    10.0.0.*(rw,sync)          目录 允许ip段 权限

NFS启动：主从同时配置： 先启动rpcbind  服务
[root@heartbeat2 ~]# /etc/init.d/rpcbind restart
Stopping rpcbind:                                          [  OK  ]
Starting rpcbind:                                          [  OK  ]
[root@heartbeat2 ~]# /etc/init.d/nfs restart
Shutting down NFS daemon:                                  [FAILED]
Shutting down NFS mountd:                                  [FAILED]
Shutting down NFS quotas:                                  [FAILED]
Starting NFS services:                                     [  OK  ]
Starting NFS quotas:                                       [  OK  ]
Starting NFS mountd:                                       [  OK  ]
Starting NFS daemon:                                       [  OK  ]
Starting RPC idmapd:                                       [  OK  ]
[root@heartbeat2 ~]# chkconfig rpcbind on
[root@heartbeat2 ~]# chkconfig nfs on
[root@heartbeat1 ~]# rpcinfo -p localhost      检查启动状态

[root@heartbeat1 ~]# showmount -e 10.0.0.50
Export list for 10.0.0.50:
/md1 10.0.0.*

客户端挂载：
[root@localhost ~]# mount -t nfs 10.0.0.50:/md1/ /md1/
[root@localhost ~]#
[root@localhost ~]# ls /md1/
lafjd       oldboy14  oldboy21
[root@localhost ~]# echo "mount -t nfs 10.0.0.50:/md1/ /md1/" >> /etc/rc.local    添加开机启动
[root@localhost ~]# tail -1 /etc/rc.local
mount -t nfs 10.0.0.50:/md1/ /md1/

也可以使用exportfs 配置NFS（详情 man exportfs）
[root@heartbeat1 ha.d]#exportfs  -o rw,sync,all_squash,anonuid=65534,anongid=65534,mp,fsid=2

10.0.0.0/24:/md1
[root@heartbeat1 ha.d]# /etc/init.d/nfs restart
[root@heartbeat1 ha.d]# showmount -e 10.0.0.7
Export list for 10.0.0.7:
/md1 10.0.0.*

----------------------------------------------------------------------------
查看drbd运行状态：
[root@heartbeat1 ~]# cat /proc/drbd
version: 8.4.4 (api:1/proto:86-101)
GIT-hash: 74402fecf24da8e5438171ee8c19e28627e1c98a build by root@heartbeat1, 2015-07-21 14:36:54
0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
    ns:8 nr:0 dw:4 dr:776 al:1 bm:1 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
[root@heartbeat1 ~]#

[root@heartbeat1 ~]# lsmod |grep drbd
drbd                  327515  4
libcrc32c               1246  1 drbd

drbdadm  命令说明：
[root@heartbeat1 ~]# drbdadm -h

USAGE: drbdadm COMMAND [OPTION...] {all|RESOURCE...}

GENERAL OPTIONS:
  --stacked, -S
  --dry-run, -d
  --verbose, -v
  --config-file=..., -c ...
  --config-to-test=..., -t ...
  --drbdsetup=..., -s ...
  --drbdmeta=..., -m ...
  --drbd-proxy-ctl=..., -p ...
  --sh-varname=..., -n ...
  --peer=..., -P ...
  --version, -V
  --setup-option=..., -W ...
  --help, -h

COMMANDS:
attach                             disk-options                      
detach                             connect                           
net-options                        disconnect                        
up                                 resource-options                  
down                               primary                           
secondary                          invalidate                        
invalidate-remote                  outdate                           
resize                             verify                            
pause-sync                         resume-sync                       
adjust                             adjust-with-progress              
wait-connect                       wait-con-int                      
role                               cstate                            
dstate                             dump                              
dump-xml                           create-md                         
show-gi                            get-gi                            
dump-md                            wipe-md                           
apply-al                           hidden-commands                   

Version: 8.4.4 (api:1)
GIT-hash: 74402fecf24da8e5438171ee8c19e28627e1c98a build by root@heartbeat1, 2015-07-21 14:36:42

[root@heartbeat1 ~]# drbdadm create-md data    初始化data磁盘
[root@heartbeat1 ~]# drbdadm disconnect data    断开data磁盘连接
[root@heartbeat1 ~]# drbdadm connect data    连接data磁盘

注意事项：
启动heartbeat：(重启先stop在start)
/etc/init.d/heartbeat start
关掉开机启动（heartbeat开机启动容易发生裂脑问题）
chkconfig heartbeat off
chkconfig drbd    off

解决drbd Unknown 问题：
cs:StandAlone ro:Secondary/Unknown ds:UpToDate/DUnknown   r-----
方法一：
在选定的非主节点上执行
drbdadm secondary data
drbdadm -- --discard-my-data connect mysql
主节点上执行
drbdadm connect data

如果以上方法不行：
重新初始化
drbdadm detach data
###dd if=/dev/zero bs=1M count=100 of=/dev/sdb1
drbdadm down data
drbdadm create-md data
在主节点上执行
drbdadm connect data