DRBD块级别数据同步高可用

测试环境

OS: RHEL 7.1

软件:

drbd84-utils-8.9.6-1.el7.elrepo.x86_64.rpm

elrepo-release-7.0-2.el7.elrepo.noarch.rpm

kmod-drbd84-8.4.6-1.el7.elrepo.x86_64.rpm

0.配置每个节点的硬件和系统

网络

[root@ha1 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.56.101 ha1
192.168.56.102 ha2

磁盘

[root@ha2 tmp]# fdisk -l

Disk /dev/sda: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000e0b61

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     1026047      512000   83  Linux
/dev/sda2         1026048    41861119    20417536   8e  Linux LVM

Disk /dev/sdb: 104 MB, 104857600 bytes, 204800 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/rhel-swap: 2147 MB, 2147483648 bytes, 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/rhel-root: 18.8 GB, 18756927488 bytes, 36634624 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

[root@ha2 tmp]# fdisk /dev/sdb
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0xe828098c.

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1): 
First sector (2048-204799, default 2048): 
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-204799, default 204799): 
Using default value 204799
Partition 1 of type Linux and of size 99 MiB is set

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

注意不要格式化为文件系统！！！

1.下载和安装软件包

下载地址：http://elrepo.reloumirrors.net/elrepo/el7/

需要按照顺序安装

[root@ha2 tmp]# rpm -ivh elrepo-release-7.0-2.el7.elrepo.noarch.rpm
warning: elrepo-release-7.0-2.el7.elrepo.noarch.rpm: Header V4 DSA/SHA1 Signature, key ID baadae52: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:elrepo-release-7.0-2.el7.elrepo  ################################# [100%]
[root@ha2 tmp]# rpm -ivh drbd84-utils-8.9.6-1.el7.elrepo.x86_64.rpm
warning: drbd84-utils-8.9.6-1.el7.elrepo.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID baadae52: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:drbd84-utils-8.9.6-1.el7.elrepo  ################################# [100%]
[root@ha2 tmp]# rpm -ivh kmod-drbd84-8.4.6-1.el7.elrepo.x86_64.rpm
warning: kmod-drbd84-8.4.6-1.el7.elrepo.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID baadae52: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:kmod-drbd84-8.4.6-1.el7.elrepo   ################################# [100%]
Working. This may take some time ...
Done.

注意：如果在任何节点上未安装公钥时出现任何错误，请导入目录/ etc / pki / rpm-gpg /中的所有公钥，如下所示，然后重新安装。

rpm --import /etc/pki/rpm-gpg/*

2.配置 每个节点

默认配置文件为

 /etc/drbd.conf

[root@ha2 tmp]# cat /etc/drbd.conf
# You can find an example in  /usr/share/doc/drbd.../drbd.conf.example

include "drbd.d/global_common.conf";
include "drbd.d/*.res";

按照惯例，/etc/drbd.d/global_common.conf包含DRBD配置的全局和公共部分，而“.res”文件每个包含一个资源部分。

让我们创建一个DRBD资源文件“testdata1.res”。 我们建议始终创建唯一的名称来标识资源。 应以“.res”文件扩展名结尾。

vi /etc/drbd.d/testdata1.res
resource testdata1 {
protocol C;           
on ha1 {
                device /dev/drbd0;
                disk /dev/sdb1;
                address 192.168.56.101:7788;
                meta-disk internal;
        }
        on ha2 {
                device /dev/drbd0;
                disk /dev/sdb1;
                address 192.168.56.102:7788;
                meta-disk internal;
        }
} 

3.在每个节点上安装元数据

[root@ha2 ~]# drbdadm create-md testdata1
initializing activity log
NOT initializing bitmap
Writing meta data...
New drbd meta data block successfully created.

4. 在每个节点上同时启动服务

[root@ha1 ~]# systemctl start drbd
[root@ha1 ~]# systemctl enable drbd
ln -s '/usr/lib/systemd/system/drbd.service' '/etc/systemd/system/multi-user.target.wants/drbd.service'

5.设置主备

在主节点上（备端不需要操作）

[root@ha1 ~]# drbdadm primary testdata1
0: State change failed: (-2) Need access to UpToDate data
Command 'drbdsetup-84 primary 0' terminated with exit code 17
失败的话可以强制启动
[root@ha1 ~]# drbdadm primary testdata1 --force

检查状态

[root@ha1 ~]# cat /proc/drbd
version: 8.4.6 (api:1/proto:86-101)
GIT-hash: 833d830e0152d1e457fa7856e71e11248ccf3f70 build by root@ha1, 2017-03-29 15:33:34
 0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r-----
    ns:45724 nr:0 dw:0 dr:46452 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:55612
        [========>...........] sync'ed: 48.0% (55612/101336)K
        finish: 0:00:12 speed: 4,572 (4,572) K/sec
[root@ha1 ~]#  drbd-overview
 0:testdata1/0  Connected Primary/Secondary UpToDate/UpToDate 

6.创建和挂载文件系统

在主节点

[root@ha1 ~]# mkfs.ext3 /dev/drbd0
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
Stride=0 blocks, Stripe width=0 blocks
25376 inodes, 101336 blocks
5066 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67371008
13 block groups
8192 blocks per group, 8192 fragments per group
1952 inodes per group
Superblock backups stored on blocks: 
        8193, 24577, 40961, 57345, 73729

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done 

[root@ha1 ~]# mount /dev/drbd0 /mnt/disk

7.测试

主节点创建文件

[root@ha1 ~]# touch /mnt/disk/testfile
[root@ha1 ~]# ll /mnt/disk
total 12
drwx------ 2 root root 12288 Mar 29 17:04 lost+found
-rw-r--r-- 1 root root     0 Mar 29 17:05 testfile

卸载磁盘

[root@ha1 ~]# umount /mnt/disk

把主节点设置成备节点

[root@ha1 ~]# drbdadm secondary testdata1

备节点设置成主节点

[root@ha2 ~]# drbdadm primary testdata1





  --==  Thank you for participating in the global usage survey  ==--
The server's response is:

挂载磁盘并检查文件

[root@ha2 ~]# mount /dev/drbd0 /mnt/disk
[root@ha2 ~]# ll /mnt/disk
total 12
drwx------ 2 root root 12288 Mar 29 17:04 lost+found
-rw-r--r-- 1 root root     0 Mar 29 17:05 testfile

可见备节点磁盘自动格式化为ext3并且创建了文件