在 RHEL 5.5 下解决 MSA2312fc 的多路径问题

在 RHEL 5.5 下解决 MSA2312fc 的多路径问题

ByRobinOn2011年10月10日 ・ Leave a Comment ・ InHardware, OS, 信息世界 | IT

.

.

.

.

最近我在利用红帽的 RHCS 搭建 Oracle HA 环境的时候，又遇到了共享存储的多路径问题，上次遇到的产品是 IBM DS4700，而这次的产品是 HP MSA2312fc。IBM 产品的多路径解决方案使用的是重新编译内核，而 HP 的存储产品的解决方案则更标准化一些，主要利用 Red Hat 的 DM-Multipath 的特性来配置解决。咱们直入主题，具体的解决方案步骤如下：

1. 确认 RHEL 系统中是否安装了 device-mapper-multipath 软件。如果没有安装，请挂载 RHEL 5.5 的光盘镜像，自行安装。我在实践中 device-mapper-multipath 已经安装成功，具体如下：
   

   [root@testrac1 .ssh]# rpm -qa | grep device-mapper-multipath*
   device-mapper-multipath-0.4.7-34.el5

   
   

2. 下载 HPDMmultipath 软件。请点击 这里下载。

3. 解压安装 HPDMmultipath 软件。具体操作如下：
   

   [root@testrac1 media]# tar -zxvf HPDMmultipath-4.4.1.tar.gz
   [root@testrac1 media]# cd HPDMmultipath-4.4.1
   [root@testrac1 HPDMmultipath-4.4.1]# ls
   bin conf COPYING docs HPDMMP_Install.log INSTALL README.txt SRPMS
   [root@testrac1 HPDMmultipath-4.4.1]# ./INSTALL

   ** HPDMmultipath-4.4.1 kit Installation. Date : Fri Sep 30 08:49:35 CST 2011 **

   Checking for the Operating System version. Please wait…

   HP Device Mapper Multipath v4.4.1 kit �C Installation Menu

   1. Install HPDM Multipath Utilities
   2. Uninstall HPDM Multipath Utilities
   3. Exit

   Enter choice [1/2/3] : 1

   Note: This will install HPDMmultipath-4.4.1 tools for HP StorageWorks disk
   arrays.

   Warning: If you are retaining the existing /etc/multipath.conf file, you will
   have to manually edit the file with HP recommended parameters.
   Please refer user documentation for more details.

   Would you like to overwrite the existing /etc/multipath.conf file with the
   new multipath configuration file ? (y/n) : y

   Saving /etc/multipath.conf file to /etc/multipath.conf.savefile
   Copying new multipath configuration file multipath.conf to /etc directory

   Configuring multipath services to start at boot time….OK

   Installation completed successfully!

   [root@testrac1 HPDMmultipath-4.4.1]#

   
   

4. 针对具体的 HP 存储型号进行 DM-multipath 相关配置，具体操作如下：
   

   [root@testrac1 HPDMmultipath-4.4.1]# vi /etc/multipath.conf

   # For MSA23xxfc and MSA23xxi arrays

   device {
   vendor “HP”
   product “MSA2312fc|MSA2324fc|MSA2312i|MSA2324i”
   path_grouping_policy group_by_prio
   getuid_callout “/sbin/scsi_id -g -u -s /block/%n”
   path_checker tur
   path_selector “round-robin 0″
   prio_callout “/sbin/mpath_prio_alua /dev/%n”
   rr_weight uniform
   failback immediate
   hardware_handler “0″
   no_path_retry 18
   rr_min_io 100
   }

   
   

5. 配置 multipathd 服务，重启系统。
   

   [root@testrac1 HPDMmultipath-4.4.1]# chkconfig multipathd �Clist
   multipathd 0:off 1:off 2:off 3:on 4:off 5:on 6:off

   [root@testrac1 HPDMmultipath-4.4.1]# chkconfig multipathd on
   [root@testrac1 HPDMmultipath-4.4.1]# chkconfig multipathd �Clist
   multipathd 0:off 1:off 2:on 3:on 4:on 5:on 6:off

   [root@testrac1 HPDMmultipath-4.4.1]# reboot
   Broadcast message from root (pts/1) (Fri Sep 30 08:58:08 2011):
   The system is going down for reboot NOW!

   
   

6. 建立链路，查看链路状况。
   

   [root@testrac1 ~]# multipath -v0
   [root@testrac1 ~]# multipath -ll
   mpath0 (3600c0ff00010383eec84394e01000000) dm-2 HP,MSA2312fc
   [size=140G][features=1 queue_if_no_path][hwhandler=0][rw]
   \_ round-robin 0 [prio=50][active]
   \_ 1:0:1:1 sdb 8:16 [active][ready]
   \_ round-robin 0 [prio=10][enabled]
   \_ 0:0:2:1 sda 8:0 [active][ready]
   [root@testrac1 ~]#

   
   

7. 再次编辑 /etc/multipath.conf 配置文件，把 multipath 这个段前的 ＃ 号去掉，然后将 wwid 后面的 id 填写为前面 multipath -ll 命令输出的 id ，并将 alias 改为 mpath0。具体配置信息如下：
   

   multipath {
   wwid 3600c0ff00010383eec84394e01000000
   alias mpath0
   path_grouping_policy group_by_prio
   path_selector “round-robin 0″
   failback immediate
   rr_weight uniform
   no_path_retry 10
   rr_min_io 100
   }

   
   

8. 重启 multipathd 服务查看是否配置成功。如果 /dev/mapper 目录下有 mpath0 设备文件则表示上述配置成功。
   

   [root@testrac1 ~]# service multipathd restart
   Stopping multipathd daemon: [ OK ]
   Starting multipathd daemon: [ OK ]
   [root@testrac1 ~]# multipath -ll
   mpath0 (3600c0ff00010383eec84394e01000000) dm-2 HP,MSA2312fc
   [size=140G][features=1 queue_if_no_path][hwhandler=0][rw]
   \_ round-robin 0 [prio=50][active]
   \_ 1:0:1:1 sdb 8:16 [active][ready]
   \_ round-robin 0 [prio=10][enabled]
   \_ 0:0:2:1 sda 8:0 [active][ready]
   [root@testrac1 .ssh]# ls /dev/mapper/
   control mpath0 VolGroup00-LogVol00 VolGroup00-LogVol01
   [root@testrac1 ~]#

   
   

如果大家对 DM-Multipath 的详细配置方法感兴趣请参考 这里。再多说几句，我在 Oracle HA 的实践中，开始犯了个错误，以为应该对 /dev 目录下的 dm-2 设备进行分区格式化，结果造成格式化后无法挂载。后来才发现应该对 /dev/mapper/mpath0 进行操作。另外，在使用 fdisk 命令进行分区的过程中我遇到了 “error 22: Invalid argument ” 错误，后来通过运用 kpartx 命令得到解决。具体的分区格式化操作如下：

[root@testrac1 ~]# fdisk /dev/mapper/mpath0

The number of cylinders for this disk is set to 18236.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): m
Command action
a toggle a bootable flag
b edit bsd disklabel
c toggle the dos compatibility flag
d delete a partition
l list known partition types
m print this menu
n add a new partition
o create a new empty DOS partition table
p print the partition table
q quit without saving changes
s create a new empty Sun disklabel
t change a partition’s system id
u change display/entry units
v verify the partition table
w write table to disk and exit
x extra functionality (experts only)

Command (m for help):
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-18236, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-18236, default 18236):
Using default value 18236

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 22: Invalid argument.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.

[root@testrac1 ~]# fdisk -l
Disk /dev/cciss/c0d0: 146.7 GB, 146778685440 bytes
255 heads, 32 sectors/track, 35132 cylinders
Units = cylinders of 8160 * 512 = 4177920 bytes

Device Boot Start End Blocks Id System
/dev/cciss/c0d0p1 * 1 25 101984 83 Linux
/dev/cciss/c0d0p2 26 35132 143236560 8e Linux LVM

Disk /dev/sda: 149.9 GB, 149999992832 bytes
255 heads, 63 sectors/track, 18236 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

Disk /dev/sdb: 149.9 GB, 149999992832 bytes
255 heads, 63 sectors/track, 18236 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

Disk /dev/dm-2: 149.9 GB, 149999992832 bytes
255 heads, 63 sectors/track, 18236 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/dm-2p1 1 18236 146480638+ 83 Linux

[root@testrac1 ~]# kpartx -a /dev/mapper/mpath0
[root@testrac1 ~]# kpartx -l /dev/mapper/mpath0
mpath0p1 : 0 292961277 /dev/mapper/mpath0 63
[root@testrac1 ~]#
[root@testrac1 ~]# ls /dev/mapper/mpath0
mpath0 mpath0p1

[root@testrac1 ~]# mkfs.ext3 /dev/mapper/mpath0p1
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
18317312 inodes, 36620159 blocks
1831007 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
1118 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 35 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
[root@testrac1 ~]#

相关日志：

• 2011年08月3日 -- RHEL 5.5 下解决 IBM DS4700 的多路径问题 (0)

• 2011年07月12日 -- 配置编译安装的 Apache 为系统服务 (0)

• 2012年05月31日 -- Red Hat 改变内核源码打包方式 (0)

• 2011年11月24日 -- 在 RHEL 5.5 下安装 Cognos Server (0)

• 2011年10月25日 -- yum 命令出现“段错误”的解决方案 (0)

• 2011年11月2日 -- 在 RHEL 5.5 下安装 Python 和 IPython (0)

摘自：http://www.chenjunlu.com/2011/10/how-to-solve-the-multipath-problem-for-hp-msa2312fc-under-rhel-5-5/