创建ASM磁盘的两种方式：asmlib、udev（RHEL 7.6）
文章目录
- 1 ASMlib
- 2 UDEV
- 2.1 Linux 5
- 2.2 Linux 6
- 2.3 Linux 7
环境介绍：
OS版本：RedHat Enterprise Linux Server release 7.6 (Maipo)
DB版本：Oracle 11g
搭建rac时的两种创建ASM共享磁盘的方式：ASMlib和UDEV
1 ASMlib
- 安装asmlib依赖包
安装asmlib需要3个包，其中kmod-oracleasm-2.0.8-22.1.0.1.el7_6.x86_64可以通过yum进行安装
其它两个需要在官网下载（ http://www.oracle.com/technetwork/topics/linux/asmlib/index-101839.html）
[root@rac1 asmlib]# yum install kmod-oracleasm
Loaded plugins: langpacks, ulninfo
Resolving Dependencies
--> Running transaction check
---> Package kmod-oracleasm.x86_64 0:2.0.8-22.1.0.1.el7_6 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
=====================================================================================
Package Arch Version Repository Size
=====================================================================================
Installing:
kmod-oracleasm x86_64 2.0.8-22.1.0.1.el7_6 ol7_latest 295 k
Transaction Summary
=====================================================================================
Install 1 Package
Total download size: 295 k
Installed size: 1.5 M
Is this ok [y/d/N]: y
Downloading packages:
kmod-oracleasm-2.0.8-22.1.0.1.el7_6.x86_64.rpm | 295 kB 00:00:01
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : kmod-oracleasm-2.0.8-22.1.0.1.el7_6.x86_64 1/1
Verifying : kmod-oracleasm-2.0.8-22.1.0.1.el7_6.x86_64 1/1
Installed:
kmod-oracleasm.x86_64 0:2.0.8-22.1.0.1.el7_6
Complete!
[root@rac1 asmlib]# ls
oracleasmlib-2.0.12-1.el7.x86_64.rpm oracleasm-support-2.1.11-2.el7.x86_64.rpm
[root@rac1 asmlib]# rpm -ivh *.rpm
Preparing... ################################# [100%]
Updating / installing...
1:oracleasm-support-2.1.11-2.el7 ################################# [ 50%]
Note: Forwarding request to 'systemctl enable oracleasm.service'.
Created symlink from /etc/systemd/system/multi-user.target.wants/oracleasm.service to /usr/lib/systemd/system/oracleasm.service.
2:oracleasmlib-2.0.12-1.el7 ################################# [100%]
- 配置
[root@rac1 eql]# oracleasm configure -i
Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface []: grid
Default group to own the driver interface []: asmadmin
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
[root@rac1 eql]#
[root@rac1 eql]#
[root@rac1 eql]# oracleasm configure -d
Writing Oracle ASM library driver configuration: done
[root@rac1 eql]# oracleasm configure -e
Writing Oracle ASM library driver configuration: done
[root@rac1 eql]# oracleasm createdisk ocr /dev/eql/t10ocr
oracleasm module not loaded or /dev/oracleasm not mounted.
- 查看oracleasm状态
[root@rac1 eql]# oracleasm status
Checking if ASM is loaded: no
Checking if /dev/oracleasm is mounted: no
[root@rac1 eql]# oracleasm exit
- 加载oracleasm内核模块
[root@rac1 eql]# oracleasm init
Creating /dev/oracleasm mount point: /dev/oracleasm
Loading module "oracleasm": oracleasm
Configuring "oracleasm" to use device physical block size
Mounting ASMlib driver filesystem: /dev/oracleasm
[root@rac1 eql]# oracleasm status
Checking if ASM is loaded: yes
Checking if /dev/oracleasm is mounted: yes
- 创建磁盘
[root@rac1 eql]# oracleasm createdisk ocr /dev/eql/t10ocr
Device "/dev/eql/t10ocr" is already labeled for ASM disk ""
[root@rac1 eql]# oracleasm createdisk data /dev/eql/t10data01
Writing disk header: done
Instantiating disk: done
[root@rac1 eql]# oracleasm createdisk data /dev/eql/t10data02
Disk "DATA" already exists
# 此处报错原因是重名
- 查看所有的asm磁盘列表
[root@rac1 eql]# oracleasm listdisks
DATA
- 删除已创建的asm磁盘
[root@rac1 eql]# oracleasm deletedisk data
Clearing disk header: done
Dropping disk: done
[root@rac1 eql]# oracleasm listdisks
[root@rac1 eql]# oracleasm createdisk data02 /dev/eql/t10data02
Writing disk header: done
Instantiating disk: done
[root@rac1 eql]# oracleasm createdisk data01 /dev/eql/t10data01
Writing disk header: done
Instantiating disk: done
[root@rac1 eql]# oracleasm createdisk data03 /dev/eql/t10data03
Writing disk header: done
Instantiating disk: done
[root@rac1 eql]# oracleasm createdisk fra /dev/eql/t10fra
Writing disk header: done
Instantiating disk: done
[root@rac1 eql]# oracleasm listdisks
DATA01
DATA02
DATA03
FRA
[root@rac1 eql]# ll /dev/oracleasm/disks/
total 0
brw-rw---- 1 grid asmadmin 252, 10 May 20 16:21 DATA01
brw-rw---- 1 grid asmadmin 252, 13 May 20 16:21 DATA02
brw-rw---- 1 grid asmadmin 252, 16 May 20 16:21 DATA03
brw-rw---- 1 grid asmadmin 252, 7 May 20 16:22 FRA
- 不存在OCR盘，但是不能创建，报错如下
[root@rac1 eql]# oracleasm createdisk ocr /dev/eql/t10ocr
Device "/dev/eql/t10ocr" is already labeled for ASM disk ""
解决办法
1.清理磁盘头
[root@rac1 eql]# dd if=/dev/zero of=/dev/eql/t10ocr bs=1024 count=100
100+0 records in
100+0 records out
102400 bytes (102 kB) copied, 0.0192977 s, 5.3 MB/s
[root@rac1 eql]# oracleasm createdisk ocr /dev/eql/t10ocr
Writing disk header: done
Instantiating disk: done
[root@rac1 eql]# oracleasm listdisks
DATA01
DATA02
DATA03
FRA
OCR
2.重命名
参考：https://blog.csdn.net/wuliusir/article/details/22273491
- 节点 2 上不用依次创建，可以用 scandisk 命令扫出来
1.直接使用scandisks命令出现报错
[root@rac2 asmlib]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "OCR"
Unable to instantiate disk "OCR"
Instantiating disk "OCR"
Unable to instantiate disk "OCR"
Instantiating disk "OCR"
Unable to instantiate disk "OCR"
Instantiating disk "OCR"
Unable to instantiate disk "OCR"
Instantiating disk "OCR"
2.查看是否有oracleasm进程
# ps -ef|grep oracleasm
查看/dev下是否有oracleasm文件夹
# ll /dev/oracleasm
发现并没有相关进程，在/dev下也没有相关的文件夹，这样说的话scandisks肯定扫描不出来
原因是rac的oracleasm并未初始化
[root@rac2 asmlib]# oracleasm status
Checking if ASM is loaded: no
Checking if /dev/oracleasm is mounted: no
[root@rac2 asmlib]# oracleasm exit
初始化oracleasm
[root@rac2 asmlib]# oracleasm init
Creating /dev/oracleasm mount point: /dev/oracleasm
Loading module "oracleasm": oracleasm
Configuring "oracleasm" to use device physical block size
Mounting ASMlib driver filesystem: /dev/oracleasm
[root@rac2 asmlib]# oracleasm status
Checking if ASM is loaded: yes
Checking if /dev/oracleasm is mounted: yes
[root@rac2 asmlib]# ps -ef|grep oracleasm
root 13174 42668 0 16:46 pts/1 00:00:00 grep --color=auto oracleasm
[root@rac2 asmlib]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "OCR"
Instantiating disk "FRA"
Instantiating disk "DATA01"
Instantiating disk "DATA02"
Instantiating disk "DATA03"
[root@rac2 asmlib]# ll /dev/oracleasm
total 0
drwxr-xr-x 1 root root 0 May 20 16:45 disks
drwxrwx--- 1 root root 0 May 20 16:45 iid
[root@rac2 asmlib]# oracleasm listdisks
DATA01
DATA02
DATA03
FRA
OCR
- PRVG-10122
由于此时并没有对rac2的ASMlib进行配置，所以在安装GRID的时候出现报错
PRVG-10122 : ASMLib configuration value set to configuration parameter “ORACLEASM_UID” on the node “rac2” does not match with cluster nodes - Cause: The ASMLib configuration check found inconsistent settings across cluster nodes. - Action: Ensure that the ASMLib is correctly installed and configured on all the nodes with same configuration settings and that the user has the necessary access privileges for the configuration file
解决办法：
1.查看两个节点的oracleasm配置信息
-rac 1
[root@rac1 ~]# oracleasm configure
ORACLEASM_ENABLED=true
ORACLEASM_UID=grid
ORACLEASM_GID=asmadmin
ORACLEASM_SCANBOOT=true
ORACLEASM_SCANORDER=""
ORACLEASM_SCANEXCLUDE=""
ORACLEASM_SCAN_DIRECTORIES=""
ORACLEASM_USE_LOGICAL_BLOCK_SIZE="false"
-rac2
[root@rac2 ~]# oracleasm configure
ORACLEASM_ENABLED=false
ORACLEASM_UID=
ORACLEASM_GID=
ORACLEASM_SCANBOOT=true
ORACLEASM_SCANORDER=""
ORACLEASM_SCANEXCLUDE=""
ORACLEASM_SCAN_DIRECTORIES=""
ORACLEASM_USE_LOGICAL_BLOCK_SIZE="false"
由上述信息可知，ASMlib未在节点rac2上配置
因此需要节点rac2进行oracleasm配置
[root@rac2 ~]# oracleasm configure -i
Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface []: grid
Default group to own the driver interface []: asmadmin
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
再次查看
[root@rac2 ~]# oracleasm configure
ORACLEASM_ENABLED=true
ORACLEASM_UID=grid
ORACLEASM_GID=asmadmin
ORACLEASM_SCANBOOT=true
ORACLEASM_SCANORDER=""
ORACLEASM_SCANEXCLUDE=""
ORACLEASM_SCAN_DIRECTORIES=""
ORACLEASM_USE_LOGICAL_BLOCK_SIZE="false"
问题解决。
小结：
在使用ASMlib创建ASM时，需要对所有节点进行配置和加载。创建磁盘的工作在一个节点完成就可以，其余的节点通过scandisks可以扫出来。
在重启rac的时候，除了创建磁盘的节点，其余节点都要重新scan。
2 UDEV
udev在linux5、6、7的命令有变化
2.1 Linux 5
for i in b c d e f g h i ;
do
echo "KERNEL==\"sd*\", BUS==\"scsi\", PROGRAM==\"/sbin/scsi_id -g -u -s %p\", RESULT==\"`scsi_id -g -u -s /block/sd$i`\", NAME=\"asm-disk$i\", OWNER=\"grid\", GROUP=\"asmadmin\", MODE=\"0660\""
done
# 启动udev
/sbin/start_udev
2.2 Linux 6
for i in b c d e f g ;
do
echo "KERNEL==\"sd*\", BUS==\"scsi\", PROGRAM==\"/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/\$name\",
RESULT==\"`/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/sd$i`\", NAME=\"asm-disk$i\", OWNER=\"grid\",
GROUP=\"asmadmin\", MODE=\"0660\"" >> /etc/udev/rules.d/99-oracle-asmdevices.rules
done
# 启动udev
/sbin/start_udev
2.3 Linux 7
for i in b c d e ;
do
echo "KERNEL==\"sd*\", ENV{DEVTYPE}==\"disk\", SUBSYSTEM==\"block\", PROGRAM==\"/lib/udev/scsi_id -g -u -d \$devnode\",
RESULT==\"`/lib/udev/scsi_id -g -u -d /dev/sd$i`\", SYMLINK+=\"asm-disk$i\", OWNER=\"grid\", GROUP=\"asmadmin\", MODE=
\"0660\"" >> /etc/udev/rules.d/99-oracle-asmdevices.rules
done
# 加载rules文件,重新加载udev rule
/sbin/udevadm control --reload
# 检查新的设备名称
/sbin/udevadm trigger --type=devices --action=change
# 诊断udev rule
/sbin/udevadm test /sys/block/*
找不到udev命令时，可以通过whereis查找命令
[root@rac1 ~]# whereis udev
udev: /usr/lib/udev /etc/udev /usr/share/man/man7/udev.7.gz
for i in b c d e f;
do
echo "KERNEL==\"sd*\", SUBSYSTEM==\"block\", PROGRAM==\"/usr/lib/udev/scsi_id -g -u -d \$devnode\",RESULT==\"`/usr/lib/udev/scsi_id -g -u -d /dev/sd$i`\",SYMLINK+=\"asm-disk$i\", NAME=\"asm-disk$i\",OWNER=\"grid\", GROUP=\"asmadmin\", MODE=\"0660\"" >> /etc/udev/rules.d/99-oracle-asmdevices.rules
done
[root@rac1 rules.d]# cat 99-oracle-asmdevices.rules
KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="36090a08810326a4fb5cdf5a87dc893e3",SYMLINK+="asm-diskb", NAME="asm-diskb",OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="36090a0881032ba51b5cd25a97dc8636d",SYMLINK+="asm-diskc", NAME="asm-diskc",OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="36090a0881032fa53b5cd45a97dc8438d",SYMLINK+="asm-diskd", NAME="asm-diskd",OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="36090a08810321a56b5cd65a97dc8731a",SYMLINK+="asm-diske", NAME="asm-diske",OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="36090a08810329a57b5cd85a97dc893e8",SYMLINK+="asm-diskf", NAME="asm-diskf",OWNER="grid", GROUP="asmadmin", MODE="0660"
[root@rac1 rules.d]# /sbin/udevadm control --reload-rules
[root@rac1 rules.d]# ll -l /dev/asm*
ls: cannot access /dev/asm*: No such file or directory
[root@rac1 rules.d]# /sbin/udevadm trigger --type=devices --action=change
[root@rac1 rules.d]# ll -l /dev/asm*
lrwxrwxrwx 1 root root 3 May 17 10:05 /dev/asm-diskb -> sdb
lrwxrwxrwx 1 root root 3 May 17 10:05 /dev/asm-diskc -> sdc
lrwxrwxrwx 1 root root 3 May 17 10:05 /dev/asm-diskd -> sdd
lrwxrwxrwx 1 root root 3 May 17 10:05 /dev/asm-diske -> sds
lrwxrwxrwx 1 root root 3 May 17 10:05 /dev/asm-diskf -> sdr
[root@rac1 ~]# for i in b c d e f;
> do
or; chown grid:asmadmin /dev/asm-disk$i; chmod 0660 /dev/asm-disk$i'\""
done> echo "KERNEL==\"sd*\", ENV{DEVTYPE}==\"disk\",SUBSYSTEM==\"block\", PROGRAM==\r/lib/udev/scsi_id -g -u -d \$devnode\",RESULT==\"`/usr/lib/udev/scsi_id -g -u /dev/sd$i`\",RUN+=\"/bin/sh -c 'mknod /dev/asm-disk$i b \$major \$minor; chown grid:asmadmin /dev/asm-disk$i; chmod 0660 /dev/asm-disk$i'\""
> done
KERNEL=="sd*", ENV{DEVTYPE}=="disk",SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="36090a08810326a4fb5cdf5a87dc893e3",RUN+="/bin/sh -c 'mknod /dev/asm-diskb b $major $minor; chown grid:asmadmin /dev/asm-diskb; chmod 0660 /dev/asm-diskb'"
KERNEL=="sd*", ENV{DEVTYPE}=="disk",SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="36090a0881032ba51b5cd25a97dc8636d",RUN+="/bin/sh -c 'mknod /dev/asm-diskc b $major $minor; chown grid:asmadmin /dev/asm-diskc; chmod 0660 /dev/asm-diskc'"
KERNEL=="sd*", ENV{DEVTYPE}=="disk",SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="36090a0881032fa53b5cd45a97dc8438d",RUN+="/bin/sh -c 'mknod /dev/asm-diskd b $major $minor; chown grid:asmadmin /dev/asm-diskd; chmod 0660 /dev/asm-diskd'"
KERNEL=="sd*", ENV{DEVTYPE}=="disk",SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="36090a08810321a56b5cd65a97dc8731a",RUN+="/bin/sh -c 'mknod /dev/asm-diske b $major $minor; chown grid:asmadmin /dev/asm-diske; chmod 0660 /dev/asm-diske'"
KERNEL=="sd*", ENV{DEVTYPE}=="disk",SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="36090a08810329a57b5cd85a97dc893e8",RUN+="/bin/sh -c 'mknod /dev/asm-diskf b $major $minor; chown grid:asmadmin /dev/asm-diskf; chmod 0660 /dev/asm-diskf'"
[root@rac1 ~]# vi 99-oracle-asmdevices.rules
[root@rac1 ~]# /sbin/udevadm trigger --type=devices --action=change
[root@rac1 ~]# ll /dev/asm*
brw-r--r-- 1 grid asmadmin 8, 16 May 17 13:52 /dev/asm-diskb
brw-r--r-- 1 grid asmadmin 8, 32 May 17 13:52 /dev/asm-diskc
brw-r--r-- 1 grid asmadmin 8, 48 May 17 13:52 /dev/asm-diskd
brw-r--r-- 1 grid asmadmin 8, 64 May 17 13:52 /dev/asm-diske
brw-r--r-- 1 grid asmadmin 8, 80 May 17 13:52 /dev/asm-diskf
[root@rac1 eql]# ls
t10data01 t10data02 t10data03 t10fra t10ocr
[root@rac1 eql]# pwd
/dev/eql
[root@rac1 eql]# /usr/lib/udev/scsi_id -g -u -d /dev/eql/t10ocr
36090a08810326a4fb5cdf5a87dc893e3
[root@rac1 eql]# /usr/lib/udev/scsi_id -g -u -d /dev/eql/t10fra
36090a0881032ba51b5cd25a97dc8636d
[root@rac1 eql]# /usr/lib/udev/scsi_id -g -u -d /dev/eql/t10data01
36090a0881032fa53b5cd45a97dc8438d
[root@rac1 eql]# /usr/lib/udev/scsi_id -g -u -d /dev/eql/t10data02
36090a08810321a56b5cd65a97dc8731a
[root@rac1 eql]# /usr/lib/udev/scsi_id -g -u -d /dev/eql/t10data03
36090a08810329a57b5cd85a97dc893e8
KERNEL=="sd*", ENV{DEVTYPE}=="disk",SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="36090a08810326a4fb5cdf5a87dc893e3",RUN+="/bin/sh -c 'mknod /dev/asm-ocr b $major $minor; chown grid:asmadmin /dev/asm-ocr; chmod 0660 /dev/asm-ocr'"
KERNEL=="sd*", ENV{DEVTYPE}=="disk",SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="36090a0881032ba51b5cd25a97dc8636d",RUN+="/bin/sh -c 'mknod /dev/asm-fra b $major $minor; chown grid:asmadmin /dev/asm-fra; chmod 0660 /dev/asm-fra'"
KERNEL=="sd*", ENV{DEVTYPE}=="disk",SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="36090a0881032fa53b5cd45a97dc8438d",RUN+="/bin/sh -c 'mknod /dev/asm-data01 b $major $minor; chown grid:asmadmin /dev/asm-data01; chmod 0660 /dev/asm-data01'"
KERNEL=="sd*", ENV{DEVTYPE}=="disk",SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="36090a08810321a56b5cd65a97dc8731a",RUN+="/bin/sh -c 'mknod /dev/asm-data02 b $major $minor; chown grid:asmadmin /dev/asm-data02; chmod 0660 /dev/asm-data02'"
KERNEL=="sd*", ENV{DEVTYPE}=="disk",SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="36090a08810329a57b5cd85a97dc893e8",RUN+="/bin/sh -c 'mknod /dev/asm-data03 b $major $minor; chown grid:asmadmin /dev/asm-data03; chmod 0660 /dev/asm-data03'"
[root@rac1 eql]# /sbin/udevadm control --reload-rules
[root@rac1 eql]# /sbin/udevadm trigger --type=devices --action=change
[root@rac1 eql]# ll -l /dev/asm*
brw-rw---- 1 grid asmadmin 8, 128 May 17 14:21 /dev/asm-data01
brw-rw---- 1 grid asmadmin 8, 112 May 17 14:21 /dev/asm-data02
brw-rw---- 1 grid asmadmin 8, 80 May 17 14:21 /dev/asm-data03
brw-rw---- 1 grid asmadmin 8, 176 May 17 14:21 /dev/asm-fra
brw-rw---- 1 grid asmadmin 8, 240 May 17 14:21 /dev/asm-ocr