用Openfiler 给Oracle RAC集群配置共享存储

用Openfiler 给Oracle RAC集群配置共享存储.

实验环境

RAC OS: Oracle linux 6.3

openfiler 版本: 2.3或2.99

1.配置Openfiler  

 

安装：

需要两块网卡：

eth0: 192.168.115.109

eth1:172.168.146.109

 

关于网卡：

 

openfiler版本是2.3的话,1块或2块无所谓，最后绑定的时候使用1块，我在绑定到主机时使用的是eth1

如果是2.99的话，最好使用1块网卡，如果是2块的话，且都和主机能通，绑定到主机时会磁盘会多出来一份。

 

Openfiler操作系统安装省略，给Openfiler加载两块盘，1块盘用来安装openfiler操作系统，另1块盘用于为其他机器做存储。

 

我在实验中操作系统用的盘分了20G，为其他机器用的盘分了50G。

安装完成后的界面

 

 

登陆到openfiler Web 管理GUI

Openfiler的默认用户名为openfiler，密码为password

 

 

需要添加访问该存储的机器

 

配置存储

1.    增加physicalvolumes

 

在volumnes /blockdevicees页面添加，本次测试增加/dev/sdb1

2.    增加volume groups

在 “volumnes/volume Groups”页面

 

 

 

3.在volume groups创建 logical volumes

在 “volumnes/add volume”页面

 

4.配置lun

在 “volumnes/iscsi targets”页面操作

如果是上面灰色状态，说明iscsi-target服务未启动

[root@openfiler ~]# service iscsi-target status

iSCSI target stopped

[root@openfiler ~]# service iscsi-target start

Starting iSCSI target service: [  OK  ]

[root@openfiler ~]# chkconfig iscsi-target on

 

启1个自己好认的名字

 

再增加1块IQN

再增加1块IQN

 

DB机器上配置

yum  install iscsi-initiator-utils

 

[root@racnode1media]# service iscsid start

[root@racnode1media]# chkconfig iscsid on

[root@racnode1media]# chkconfig iscsi on

 

[root@racnode1media]# vi /etc/hosts

172.168.146.109openfiler1-priv

~

 

 

检验配置是否正常工作

[root@racnode1~]#  iscsiadm -mdiscovery -t sendtargets -p openfiler1-priv

172.168.146.109:3260,1iqn.2006-01.com.openfiler:rac.fra

172.168.146.109:3260,1iqn.2006-01.com.openfiler:rac.data

172.168.146.109:3260,1iqn.2006-01.com.openfiler:rac.crs

手动登录到 iSCSI目标

# iscsiadm -m node -T iqn.2006-01.com.openfiler:rac.crs  -p 172.168.146.109 -l

Logging in to[iface: default, target: iqn.2006-01.com.openfiler:rac.crs, portal:172.168.146.109,3260] (multiple)

Login to [iface:default, target: iqn.2006-01.com.openfiler:rac.crs, portal:172.168.146.109,3260] successful.

# iscsiadm -m node -T iqn.2006-01.com.openfiler:rac.data  -p 172.168.146.109 -l

Logging in to[iface: default, target: iqn.2006-01.com.openfiler:rac.data, portal:172.168.146.109,3260] (multiple)

Login to [iface:default, target: iqn.2006-01.com.openfiler:rac.data, portal:172.168.146.109,3260] successful.

# iscsiadm -m node -T iqn.2006-01.com.openfiler:rac.fra  -p 172.168.146.109 -l

Logging in to[iface: default, target: iqn.2006-01.com.openfiler:rac.fra, portal: 172.168.146.109,3260](multiple)

Login to [iface:default, target: iqn.2006-01.com.openfiler:rac.fra, portal:172.168.146.109,3260] successful.

# iscsiadm -m node-T iqn.2006-01.com.openfiler:racdb.data1 -p 172.168.146.109 --op update -nnode.startup -v automatic

 

配置自动登录

[root@racnode1 ~]# iscsiadm -m node-T iqn.2006-01.com.openfiler:rac.crs  -p172.168.146.109 --op update -n node.startup -v automatic

[root@racnode1~]#  iscsiadm -mnode -T iqn.2006-01.com.openfiler:rac.data -p 172.168.146.109 --op update -n node.startup -v automatic

[root@racnode1~]#  iscsiadm -mnode -T iqn.2006-01.com.openfiler:rac.fra -p 172.168.146.109 --op update -n node.startup -v automatic

 

创建永久映射关系：

 

(以下脚本来自于网络，经过验证，脚本可行，无须修改 )

 

[root@racnode1by-path]# vi /etc/udev/rules.d/55-openiscsi.rules
 
#/etc/udev/rules.d/55-openiscsi.rules
KERNEL=="sd*",BUS=="scsi", PROGRAM="/etc/udev/scripts/iscsidev.sh %b",
SYMLINK+="iscsi/%c/part%n"
~
 
[root@racnode1 ~]#vi /etc/udev/scripts/iscsidev.sh
 
#!/bin/sh
# FILE:/etc/udev/scripts/iscsidev.sh
 
BUS=${1}
HOST=${BUS%%:*}
[ -e/sys/class/iscsi_host ] || exit 1
file="/sys/class/iscsi_host/host${HOST}/device/session*/iscsi_session*/
targetname"
target_name=$(cat${file})
# This is not anopen-scsi drive
if [ -z"${target_name}" ]; then
exit 1
fi
 
# Check if QNAPdrive
check_qnap_target_name=${target_name%%:*}
if [ $check_qnap_target_name= "iqn.2004-04.com.qnap" ]; then
target_name=`echo"${target_name%.*}"`
fi
 
echo"${target_name##*.}"
"/etc/udev/scripts/iscsidev.sh"22L, 497C
#!/bin/sh
# FILE:/etc/udev/scripts/iscsidev.sh
 
BUS=${1}
HOST=${BUS%%:*}
[ -e /sys/class/iscsi_host] || exit 1
file="/sys/class/iscsi_host/host${HOST}/device/session*/iscsi_session*/
targetname"
target_name=$(cat${file})
# This is not anopen-scsi drive
if [ -z"${target_name}" ]; then
exit 1
fi
 
# Check if QNAPdrive
check_qnap_target_name=${target_name%%:*}
if [$check_qnap_target_name = "iqn.2004-04.com.qnap" ]; then
target_name=`echo"${target_name%.*}"`
fi
 
echo"${target_name##*.}"
 
 

做完后通过命令 fdisk 或直接 到/dev/disk/by-path下面可以看到映射到的新盘。  比如 /dev/sdb  /dev/sdc /dev/sde

可能的问题：DB机器上发现不了

---openfiler2.99，openfile2.3都适用

 

[root@racnode1~]#  iscsiadm -m discovery -t sendtargets-p 172.168.146.110

iscsiadm: Noportals found

 

 

 

[root@localhost network-scripts]# cat/etc/initiators.allow

 

# PLEASE DO NOT MODIFY THIS CONFIGURATIONFILE!

#      This configuration file was autogenerated

#      by Openfiler. Any manual changes will be overwritten

#      Generated at: Tue Apr 1 9:48:28 CST 2014

 

iqn.2006-01.com.openfiler:rac.crs  172.168.146.51/24, 172.168.146.52/24

iqn.2006-01.com.openfiler:rac.data  172.168.146.51/24, 172.168.146.52/24

iqn.2006-01.com.openfiler:rac.fra  172.168.146.51/24, 172.168.146.52/24

 

# End of Openfiler configuration

 

[root@localhost network-scripts]# cat /etc/initiators.deny

 

# PLEASE DO NOT MODIFY THIS CONFIGURATIONFILE!

#      This configuration file was autogenerated

#      by Openfiler. Any manual changes will be overwritten

#      Generated at: Tue Apr 1 9:48:28 CST 2014

 

iqn.2006-01.com.openfiler:rac.crs ALL

 

iqn.2006-01.com.openfiler:rac.data ALL

 

iqn.2006-01.com.openfiler:rac.fra ALL

 

 

# End of Openfiler configuration

 

[root@localhost network-scripts]#

 

注释掉了/etc/initiators.deny 文件中的内容，访问正常。

 

#iqn.2006-01.com.openfiler:rac.crs ALL

 

#iqn.2006-01.com.openfiler:rac.data ALL

 

#iqn.2006-01.com.openfiler:rac.fra ALL

 

 

 

[root@racnode1 ~]#  iscsiadm -m discovery -t sendtargets -p172.168.146.110

172.168.146.110:3260,1iqn.2006-01.com.openfiler:rac.fra

172.168.146.110:3260,1iqn.2006-01.com.openfiler:rac.data

172.168.146.110:3260,1iqn.2006-01.com.openfiler:rac.crs