DRBD简单配置

DRBD 是由内核模块和相关脚本而构成，是一个用软件实现的、无共享的、服务器之间镜像块设备内容的存储复制解决方案，用以构建高可用性的集群。其实现方式是通过网络来镜像整个设备。您可以把它看作是一种网络RAID。DRBD负责接收数据，把数据写到本地磁盘，然后发送给另一个主机。另一个主机再将数据存到自己的磁盘中。

源码安装比较简单

./configure –prefix=/usr –with-km,参数with-km是Enable kernel module，2.6.33版本内核已经默认支持，之前的版本需要加这个参数编译

Make

Make install

 

在资源文件就可以正常使用了。这个文件是不存在的，需要添加。在安装后的目录下查看drbd.conf文件

# You can find an example in  /usr/share/doc/drbd.../drbd.conf.example

 

include "drbd.d/global_common.conf";

include "drbd.d/*.res";

在drbd.d目录下添加一个后缀为res的文件

 

我添加的是：

[root@redhat62-2 drbd.d]# cat dbdata.res

resource dbdata{

on redhat62{

device /dev/drbd0;

disk /dev/sdb1;

address 192.168.163.150:7789;

meta-disk internal;

}

on redhat62-2{

device /dev/drbd0;

disk /dev/sdb1;

address 192.168.163.151:7789;

meta-disk internal;

}

}

把这个文件复制到另外一台机器的相同目录下，

启动drbd，

然后执行drbdadm create-md all

可以通过ls –l /dev/drbd0看到drbd设备，

使用cat /proc/drbd能够看到drbd的一些信息

执行对drdb设备的文件系统创建

Mkfs.ext4 /dev/drbd0

然后就可以挂载文件系统了。

 

在另外一个节点也执行相同的操作，但是在执行之前需要先把这个节点的设备unmount，然后停止drbd服务。

 

在主节点写入的信息可以通过下面的方式看到同步了多少

[root@redhat62 drbd.d]# cat /proc/drbd

version: 8.4.2 (api:1/proto:86-101)

GIT-hash: 7ad5f850d711223713d6dcadc3dd48860321070c build by root@redhat62, 2012-10-28 22:00:56

 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----

ns:238164 nr:1044204 dw:1282368 dr:1397 al:64 bm:64 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0

如果想看到另外个节点磁盘上的内容，需要把住节点的drbd服务停掉，或者umount设备，然后drbdadm secondary all，在另外一个节点执行drbdadm primary all，mount文件系统即可。

 

drbd的一些状态信息官方说明：

 Status information in /proc/drbd

/proc/drbd is a virtual file displaying real-time status information about all DRBD resources currently configured. You may interrogate this file’s contents using this command:

$ cat /proc/drbd
version: 8.4.0 (api:1/proto:86-100)
GIT-hash: 09b6d528b3b3de50462cd7831c0a3791abc665c3 build by [email protected], 2011-10-12 09:07:35
 0: cs:Connected ro:Secondary/Secondary ds:UpToDate/UpToDate C r-----
    ns:0 nr:0 dw:0 dr:656 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
 1: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r---
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
 2: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r---
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0

The first line, prefixed with version:, shows the DRBD version used on your system. The second line contains information about this specific build.

The other four lines in this example form a block that is repeated for every DRBD device configured, prefixed by the device minor number. In this case, this is 0, corresponding to the device /dev/drbd0.

The resource-specific output from /proc/drbd contains various pieces of information about the resource:

cs (connection state). Status of the network connection. ro (roles). Roles of the nodes. The role of the local node is displayed first, followed by the role of the partner node shown after the slash. ds (disk states). State of the hard disks. Prior to the slash the state of the local node is displayed, after the slash the state of the hard disk of the partner node is shown.

 Replication protocol. Replication protocol used by the resource. Either A, B or C.

I/O Flags. Six state flags reflecting the I/O status of this resource.

Performance indicators. A number of counters and gauges reflecting the resource’s utilization and performance.

Connection states

A resource’s connection state can be observed either by monitoring /proc/drbd, or by issuing the drbdadm cstate command:

# drbdadm cstate 
Connected

A resource may have one of the following connection states:

StandAlone. No network configuration available. The resource has not yet been connected, or has been administratively disconnected (using drbdadm disconnect), or has dropped its connection due to failed authentication or split brain.

Disconnecting. Temporary state during disconnection. The next state is StandAlone.

Unconnected. Temporary state, prior to a connection attempt. Possible next states: WFConnection and WFReportParams.

Timeout. Temporary state following a timeout in the communication with the peer. Next state: Unconnected.

BrokenPipe. Temporary state after the connection to the peer was lost. Next state: Unconnected.

NetworkFailure. Temporary state after the connection to the partner was lost. Next state: Unconnected.

ProtocolError. Temporary state after the connection to the partner was lost. Next state: Unconnected.

TearDown. Temporary state. The peer is closing the connection. Next state: Unconnected.

WFConnection. This node is waiting until the peer node becomes visible on the network.

WFReportParams. TCP connection has been established, this node waits for the first network packet from the peer.

Connected. A DRBD connection has been established, data mirroring is now active. This is the normal state.

StartingSyncS. Full synchronization, initiated by the administrator, is just starting. The next possible states are: SyncSource or PausedSyncS.

StartingSyncT. Full synchronization, initiated by the administrator, is just starting. Next state: WFSyncUUID.

WFBitMapS. Partial synchronization is just starting. Next possible states: SyncSource or PausedSyncS.

WFBitMapT. Partial synchronization is just starting. Next possible state: WFSyncUUID.

WFSyncUUID. Synchronization is about to begin. Next possible states: SyncTarget or PausedSyncT.

SyncSource. Synchronization is currently running, with the local node being the source of synchronization.

SyncTarget. Synchronization is currently running, with the local node being the target of synchronization.

PausedSyncS. The local node is the source of an ongoing synchronization, but synchronization is currently paused. This may be due to a dependency on the completion of another synchronization process, or due to synchronization having been manually interrupted by drbdadm pause-sync.

PausedSyncT. The local node is the target of an ongoing synchronization, but synchronization is currently paused. This may be due to a dependency on the completion of another synchronization process, or due to synchronization having been manually interrupted by drbdadm pause-sync.

VerifyS. On-line device verification is currently running, with the local node being the source of verification.

VerifyT. On-line device verification is currently running, with the local node being the target of verification.

 Resource roles

A resource’s role can be observed either by monitoring /proc/drbd, or by issuing the drbdadm role command:

# drbdadm role 
Primary/Secondary

The local resource role is always displayed first, the remote resource role last.

You may see one of the following resource roles:

Primary. The resource is currently in the primary role, and may be read from and written to. This role only occurs on one of the two nodes, unless dual-primary mode is enabled.

Secondary. The resource is currently in the secondary role. It normally receives updates from its peer (unless running in disconnected mode), but may neither be read from nor written to. This role may occur on one or both nodes.

Unknown. The resource’s role is currently unknown. The local resource role never has this status. It is only displayed for the peer’s resource role, and only in disconnected mode.

Disk states

A resource’s disk state can be observed either by monitoring /proc/drbd, or by issuing the drbdadm dstate command:

# drbdadm dstate 
UpToDate/UpToDate

The local disk state is always displayed first, the remote disk state last.

Both the local and the remote disk state may be one of the following:

Diskless. No local block device has been assigned to the DRBD driver. This may mean that the resource has never attached to its backing device, that it has been manually detached using drbdadm detach, or that it automatically detached after a lower-level I/O error.

Attaching. Transient state while reading meta data.

Failed. Transient state following an I/O failure report by the local block device. Next state: Diskless.

Negotiating. Transient state when an Attach is carried out on an already-Connected DRBD device.

Inconsistent. The data is inconsistent. This status occurs immediately upon creation of a new resource, on both nodes (before the initial full sync). Also, this status is found in one node (the synchronization target) during synchronization.

Outdated. Resource data is consistent, but outdated.

DUnknown. This state is used for the peer disk if no network connection is available.

Consistent. Consistent data of a node without connection. When the connection is established, it is decided whether the data is UpToDate or Outdated.

UpToDate. Consistent, up-to-date state of the data. This is the normal state.

 I/O state flags

The I/O state flag field in /proc/drbd contains information about the current state of I/O operations associated with the resource. There are six such flags in total, with the following possible values:

  1. I/O suspension. Either r for running or s for suspended I/O. Normally r.
  2. Serial resynchronization. When a resource is awaiting resynchronization, but has deferred this because of a resync-after dependency, this flag becomes a. Normally -.
  3. Peer-initiated sync suspension. When resource is awaiting resynchronization, but the peer node has suspended it for any reason, this flag becomes p. Normally -.
  4. Locally initiated sync suspension. When resource is awaiting resynchronization, but a user on the local node has suspended it, this flag becomes u. Normally -.

5.  Locally blocked I/O. Normally -. May be one of the following flags:

    • d: I/O blocked for a reason internal to DRBD, such as a transient disk state.
    • b: Backing device I/O is blocking.
    • n: Congestion on the network socket.
    • a: Simultaneous combination of blocking device I/O and network congestion.
  1. Activity Log update suspension. When updates to the Activity Log are suspended, this flag becomes s. Normally -.

 Performance indicators

The second line of /proc/drbd information for each resource contains the following counters and gauges:

ns (network send). Volume of net data sent to the partner via the network connection; in Kibyte.

nr (network receive). Volume of net data received by the partner via the network connection; in Kibyte.

dw (disk write). Net data written on local hard disk; in Kibyte.

dr (disk read). Net data read from local hard disk; in Kibyte.

al (activity log). Number of updates of the activity log area of the meta data.

bm (bit map). Number of updates of the bitmap area of the meta data.

lo (local count). Number of open requests to the local I/O sub-system issued by DRBD.

pe (pending). Number of requests sent to the partner, but that have not yet been answered by the latter.

ua (unacknowledged). Number of requests received by the partner via the network connection, but that have not yet been answered.

ap (application pending). Number of block I/O requests forwarded to DRBD, but not yet answered by DRBD.

ep (epochs). Number of epoch objects. Usually 1. Might increase under I/O load when using either the barrier or the none write ordering method.

wo (write order). Currently used write ordering method: b(barrier), f(flush), d(drain) or n(none).

oos (out of sync). Amount of storage currently out of sync; in Kibibytes.