在11gR2 RAC中修改ASM DISK Path磁盘路径
原文地址：http://www.askmaclean.com/archives/howto-change-11gr2-asm-disk-path.html
aix 6.1，grid 11.2.0.3+asm11.2.0.3+rac
建数据库的时候使用的是aix自带的多路径软件mpio，建了diskgroup
现在改造成veritas dmp多路径，已经修改了asm的disk_strings＝/dev/vx/rdmp/*，crs/asm启动的时候已经可以识别到磁盘/dev/vx/rdmp/开头的磁盘，但是读取不回原来的diskgroup信息。
crs启动的报错日志：
2012-07-13 15:07:29.748: [ GPNP][1286]clsgpnp_profileCallUrlInt: [at clsgpnp.c:2108 clsgpnp_profileCallUrlInt] get-profile call to url “ipc://GPNPD_ggtest1” disco “” [f=0 claimed- host: cname: seq: auth:]
2012-07-13 15:07:29.762: [ GPNP][1286]clsgpnp_profileCallUrlInt: [at clsgpnp.c:2236 clsgpnp_profileCallUrlInt] Result: (0) CLSGPNP_OK. Successful get-profile CALL to remote “ipc://GPNPD_ggtest1” disco “”
2012-07-13 15:07:29.762: [ CSSD][1286]clssnmReadDiscoveryProfile: voting file discovery string(/dev/vx/rdmp/*)
2012-07-13 15:07:29.762: [ CSSD][1286]clssnmvDDiscThread: using discovery string /dev/vx/rdmp/* for initial discovery
2012-07-13 15:07:29.762: [ SKGFD][1286]Discovery with str:/dev/vx/rdmp/*:2012-07-13 15:07:29.762: [ SKGFD][1286]UFS discovery with :/dev/vx/rdmp/*:
2012-07-13 15:07:29.769: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_919:
2012-07-13 15:07:29.770: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_212:
2012-07-13 15:07:29.770: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_211:
2012-07-13 15:07:29.770: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_210:
2012-07-13 15:07:29.770: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_209:
2012-07-13 15:07:29.771: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_181:
2012-07-13 15:07:29.771: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_180:
2012-07-13 15:07:29.771: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/disk_3:
2012-07-13 15:07:29.771: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/disk_2:
2012-07-13 15:07:29.771: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/disk_1:
2012-07-13 15:07:29.771: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/disk_0:
2012-07-13 15:07:29.771: [ SKGFD][1286]OSS discovery with :/dev/vx/rdmp/*:
2012-07-13 15:07:29.771: [ SKGFD][1286]Handle 1115e7510 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_916:
2012-07-13 15:07:29.772: [ SKGFD][1286]Handle 1118758b0 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_912:
2012-07-13 15:07:29.773: [ SKGFD][1286]Handle 1118d9cf0 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_908:
2012-07-13 15:07:29.773: [ SKGFD][1286]Handle 1118da450 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_904:
2012-07-13 15:07:29.773: [ SKGFD][1286]Handle 1118dad70 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_903:
2012-07-13 15:07:29.802: [ CLSF][1286]checksum failed for disk:/dev/vx/rdmp/v_df8000_916:
2012-07-13 15:07:29.803: [ SKGFD][1286]Lib :UFS:: closing handle 1115e7510 for disk :/dev/vx/rdmp/v_df8000_916:2012-07-13 15:07:29.803: [ SKGFD][1286]Lib :UFS:: closing handle 1118758b0 for disk :/dev/vx/rdmp/v_df8000_912:
2012-07-13 15:07:29.804: [ SKGFD][1286]Handle 1115e6710 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_202:
2012-07-13 15:07:29.808: [ SKGFD][1286]Handle 1115e7030 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_201:
2012-07-13 15:07:29.809: [ SKGFD][1286]Handle 1115e7ad0 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_200:
2012-07-13 15:07:29.809: [ SKGFD][1286]Handle 1118733f0 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_199:
2012-07-13 15:07:29.816: [ CLSF][1286]checksum failed for disk:/dev/vx/rdmp/v_df8000_186:
2012-07-13 15:07:29.816: [ SKGFD][1286]Lib :UFS:: closing handle 1118de5d0 for disk :/dev/vx/rdmp/v_df8000_186:2012-07-13 15:07:29.816: [ CSSD][1286]clssnmvDiskVerify: Successful discovery of 0 disks
2012-07-13 15:07:29.816: [ CSSD][1286]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery
2012-07-13 15:07:29.816: [ CSSD][1286]clssnmvFindInitialConfigs: No voting files found
2012-07-13 15:07:29.816: [ CSSD][1286](:CSSNM00070:)clssnmCompleteInitVFDiscovery: Voting file not found. Retrying discovery in 15 seconds
2012-07-13 15:07:30.169: [ CSSD][1029]clssgmExecuteClientRequest(): type(37) size(80) only connect and exit messages are allowed before lease acquisition proc(1115e4870) client(0)
其需求为修改ASM DISK PATH的磁盘设备路径，但是由于11gR2 RAC+ASM的特殊性，导致CRS无法正常启动，虽然使用crsctl start crs -excl -nocrs的方式可以启动CSS服务和ASM实例，
但是最后还是报(clssnmCompleteInitVFDiscovery: Voting file not found)，这是由于Voteing file投票磁盘的位置没有合理更新导致的。
这里我们来学习一下，如何正确的修改11gR2 RAC+ASM下的ASM DISK路径：
1.我们来营造一个修改ASM DISK路径的环境，这里我们使用UDEV的设备名绑定服务，并利用UDEV将原本的ASM DISK从/dev/asm-disk* 修改为 /dev/rasm-disk*的形式， 这只需要修改udev rule文件即可实现：
[grid@maclean1 ~]$ export ORACLE_HOME=/g01/grid/app/11.2.0/grid [grid@maclean1 ~]$ /g01/grid/app/11.2.0/grid/bin/sqlplus / as sysasm SQL*Plus: Release 11.2.0.3.0 Production on Sun Jul 15 04:09:28 2012 Copyright (c) 1982, 2011, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production With the Real Application Clusters and Automatic Storage Management options SQL> show parameter diskstri NAME TYPE VALUE ------------------------------------ ----------- ------------------------------ asm_diskstring string /dev/asm*
可以看到当前ASM实例使用的asm_diskstring 为/dev/asm*， 切换到root用户修改UDEV RULE文件 ：
[root@maclean1 rules.d]# cp 99-oracle-asmdevices.rules 99-oracle-asmdevices.rules.bak [root@maclean1 rules.d]# vi 99-oracle-asmdevices.rules [root@maclean1 rules.d]# cat 99-oracle-asmdevices.rules KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB09cadb31-cfbea255_", NAME="rasm-diskb", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB5f097069-59efb82f_", NAME="rasm-diskc", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB4e1a81c0-20478bc4_", NAME="rasm-diskd", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VBdcce9285-b13c5a27_", NAME="rasm-diske", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB82effe1a-dbca7dff_", NAME="rasm-diskf", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB950d279f-c581cb51_", NAME="rasm-diskg", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB14400d81-651672d7_", NAME="rasm-diskh", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB31b1237b-78aa22bb_", NAME="rasm-diski", OWNER="grid", GROUP="asmadmin", MODE="0660"
以上修改了原始99-oracle-asmdevices.rules的UDEV RULE规则文件，生成的设备名被修改为/dev/rasm-disk*的形式，不同于之前的ASM DISK设备名， 这要求我们后续的一系列操作来保证RAC CRS可以正常启动。
当前运行时的votedisk和ocr 存放位置:
[root@maclean1 rules.d]# /g01/grid/app/11.2.0/grid/bin/crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 6896bfc3d1464f9fbf0ea9df87e023ad (/dev/asm-diskb) [SYSTEMDG]
2. ONLINE 58eb81b656084ff2bfd315d9badd08b7 (/dev/asm-diskc) [SYSTEMDG]
3. ONLINE 6bf7324625c54f3abf2c942b1e7f70d9 (/dev/asm-diskd) [SYSTEMDG]
4. ONLINE 43ad8ae20c354f5ebf7083bc30bf94cc (/dev/asm-diske) [SYSTEMDG]
5. ONLINE 4c225359d51b4f93bfba01080664b3d7 (/dev/asm-diskf) [SYSTEMDG]
Located 5 voting disk(s).
[root@maclean1 rules.d]# /g01/grid/app/11.2.0/grid/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 2844
Available space (kbytes) : 259276
ID : 879001605
Device/File Name : +SYSTEMDG
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
因为votedisk file的位置具体指向一个ASM DISK，所以后续我们会需要crsctl replace votedisk， 我们现在重启LINUX OS:
[root@maclean1 rules.d]# init 6 rebooting ............ [root@maclean1 dev]# ls -l *asm* brw-rw---- 1 grid asmadmin 8, 16 Jul 15 04:15 rasm-diskb brw-rw---- 1 grid asmadmin 8, 32 Jul 15 04:15 rasm-diskc brw-rw---- 1 grid asmadmin 8, 48 Jul 15 04:15 rasm-diskd brw-rw---- 1 grid asmadmin 8, 64 Jul 15 04:15 rasm-diske brw-rw---- 1 grid asmadmin 8, 80 Jul 15 04:15 rasm-diskf brw-rw---- 1 grid asmadmin 8, 96 Jul 15 04:15 rasm-diskg brw-rw---- 1 grid asmadmin 8, 112 Jul 15 04:15 rasm-diskh brw-rw---- 1 grid asmadmin 8, 128 Jul 15 04:15 rasm-diski
重启后自动获得了形如/dev/rasm-disk*的ASM DISK,查阅ASM日志可以发现css服务仍搜索/dev/asm*路径来获得ASM DISK，但是这将导致找不到任何有效的ASM DISK:
more /g01/grid/app/11.2.0/grid/log/maclean1/cssd/ocssd.log 2012-07-15 04:17:45.208: [ SKGFD][1099548992]Discovery with str:/dev/asm*: 2012-07-15 04:17:45.208: [ SKGFD][1099548992]UFS discovery with :/dev/asm*: 2012-07-15 04:17:45.208: [ SKGFD][1099548992]OSS discovery with :/dev/asm*: 2012-07-15 04:17:45.208: [ CSSD][1099548992]clssnmvDiskVerify: Successful discovery of 0 disks 2012-07-15 04:17:45.208: [ CSSD][1099548992]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery 2012-07-15 04:17:45.208: [ CSSD][1099548992]clssnmvFindInitialConfigs: No voting files found 2012-07-15 04:17:45.208: [ CSSD][1099548992](:CSSNM00070:)clssnmCompleteInitVFDiscovery: Voting file not found. Retrying discovery in 15 seconds 2012-07-15 04:17:45.251: [ CSSD][1096661312]clssgmExecuteClientRequest(): type(37) size(80) only connect and exit messages are allowed before lease acquisition proc(0x26a8ba0) client((nil)) 2012-07-15 04:17:45.251: [ CSSD][1096661312]clssgmDeadProc: proc 0x26a8ba0 2012-07-15 04:17:45.251: [ CSSD][1096661312]clssgmDestroyProc: cleaning up proc(0x26a8ba0) con(0xfe6) skgpid ospid 3751 with 0 clients, refcount 0 2012-07-15 04:17:45.252: [ CSSD][1096661312]clssgmDiscEndpcl: gipcDestroy 0xfe6 2012-07-15 04:17:45.829: [ CSSD][1096661312]clssscSelect: cookie accept request 0x2318ea0 2012-07-15 04:17:45.829: [ CSSD][1096661312]clssgmAllocProc: (0x2659480) allocated 2012-07-15 04:17:45.830: [ CSSD][1096661312]clssgmClientConnectMsg: properties of cmProc 0x2659480 - 1,2,3,4,5 2012-07-15 04:17:45.830: [ CSSD][1096661312]clssgmClientConnectMsg: Connect from con(0x114e) proc(0x2659480) pid(3751) version 11:2:1:4, properties: 1,2,3,4,5 2012-07-15 04:17:45.830: [ CSSD][1096661312]clssgmClientConnectMsg: msg flags 0x0000 2012-07-15 04:17:45.939: [ CSSD][1096661312]clssscSelect: cookie accept request 0x253ddd0 2012-07-15 04:17:45.939: [ CSSD][1096661312]clssscevtypSHRCON: getting client with cmproc 0x253ddd0 2012-07-15 04:17:45.939: [ CSSD][1096661312]clssgmRegisterClient: proc(3/0x253ddd0), client(61/0x26877b0) 2012-07-15 04:17:45.939: [ CSSD][1096661312]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(0x253ddd0) client(0x26877b0) 2012-07-15 04:17:45.939: [ CSSD][1096661312]clssgmDiscEndpcl: gipcDestroy 0x1174 2012-07-15 04:17:46.070: [ CSSD][1096661312]clssscSelect: cookie accept request 0x26368a0 2012-07-15 04:17:46.070: [ CSSD][1096661312]clssscevtypSHRCON: getting client with cmproc 0x26368a0 2012-07-15 04:17:46.070: [ CSSD][1096661312]clssgmRegisterClient: proc(5/0x26368a0), client(50/0x26877b0)
由于11gR2中CRS服务依赖于ASM，因为ocr存放在ASM中，所以ASM若无法有效启动，这导致CRS服务也无法正常工作:
[root@maclean1 ~]# crsctl check has CRS-4638: Oracle High Availability Services is online [root@maclean1 ~]# crsctl check crs CRS-4638: Oracle High Availability Services is online CRS-4535: Cannot communicate with Cluster Ready Services CRS-4530: Communications failure contacting Cluster Synchronization Services daemon CRS-4534: Cannot communicate with Event Manager
2. 以上是修改ASM DISK PATH磁盘路径的现状，我们需要通过以下操作来恢复CRS:
首先彻底关闭OHASD服务：
[root@maclean1 ~]# crsctl stop has -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'maclean1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'maclean1'
CRS-2673: Attempting to stop 'ora.crf' on 'maclean1'
CRS-2677: Stop of 'ora.mdnsd' on 'maclean1' succeeded
CRS-2677: Stop of 'ora.crf' on 'maclean1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'maclean1'
CRS-2677: Stop of 'ora.gipcd' on 'maclean1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'maclean1'
CRS-2677: Stop of 'ora.gpnpd' on 'maclean1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'maclean1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
3. 以-excl -nocrs方式启动CRS，这将仅启动ASM 实例而不会启动CRS服务：
[root@maclean1 ~]# crsctl start crs -excl -nocrs
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.mdnsd' on 'maclean1'
CRS-2676: Start of 'ora.mdnsd' on 'maclean1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'maclean1'
CRS-2676: Start of 'ora.gpnpd' on 'maclean1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'maclean1'
CRS-2672: Attempting to start 'ora.gipcd' on 'maclean1'
CRS-2676: Start of 'ora.cssdmonitor' on 'maclean1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'maclean1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'maclean1'
CRS-2672: Attempting to start 'ora.diskmon' on 'maclean1'
CRS-2676: Start of 'ora.diskmon' on 'maclean1' succeeded
CRS-2676: Start of 'ora.cssd' on 'maclean1' succeeded
CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip' on 'maclean1'
CRS-2672: Attempting to start 'ora.ctssd' on 'maclean1'
CRS-2681: Clean of 'ora.cluster_interconnect.haip' on 'maclean1' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'maclean1'
CRS-2676: Start of 'ora.ctssd' on 'maclean1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'maclean1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'maclean1'
CRS-2676: Start of 'ora.asm' on 'maclean1' succeeded
#建议同时修改CRS_HOME所在的ORACLE_BASE为777权限，避免可能的问题
[root@maclean1 ~]# chmod 777 /g01
4.修改ASM实例的disk_strings为当前的ASM DISK PATH信息:
[root@maclean1 ~]# su - grid
[grid@maclean1 ~]$ sqlplus / as sysasm
SQL*Plus: Release 11.2.0.3.0 Production on Sun Jul 15 04:40:40 2012
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
SQL> alter system set asm_diskstring='/dev/rasm*';
System altered.
SQL> alter diskgroup systemdg mount;
Diskgroup altered.
SQL> create spfile from memory;
File created.
SQL> startup force mount;
ORA-32004: obsolete or deprecated parameter(s) specified for ASM instance
ASM instance started
Total System Global Area 283930624 bytes
Fixed Size 2227664 bytes
Variable Size 256537136 bytes
ASM Cache 25165824 bytes
ASM diskgroups mounted
SQL> show parameter spfile
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
spfile string /g01/grid/app/11.2.0/grid/dbs/
spfile+ASM1.ora
SQL> show parameter disk
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
asm_diskgroups string SYSTEMDG
asm_diskstring string /dev/rasm*
SQL> create pfile from spfile;
File created.
SQL> create spfile='+SYSTEMDG' from pfile;
File created.
SQL> startup force;
ORA-32004: obsolete or deprecated parameter(s) specified for ASM instance
ASM instance started
Total System Global Area 283930624 bytes
Fixed Size 2227664 bytes
Variable Size 256537136 bytes
ASM Cache 25165824 bytes
ASM diskgroups mounted
SQL> show parameter spfile
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
spfile string +SYSTEMDG/maclean-cluster/asmp
arameterfile/registry.253.7886
82933
以上成功修改了asm_diskstring ，且更新了ASM DISKGROUP上的SPFILE ， 由于ASM使用共享的SPFILE所以其他节点上一般无需在做其他操作。
5. crsctl replace votedisk 命令将votedisk重置位置：
[root@maclean1 ~]# crsctl replace votedisk +systemdg
Successful addition of voting disk 864a00efcfbe4f42bfd0f4f6b60472a0.
Successful addition of voting disk ab14d6e727614f29bf53b9870052a5c8.
Successful addition of voting disk 754c03c168854f46bf2daee7287bf260.
Successful addition of voting disk 9ed58f37f3e84f28bfcd9b101f2af9f3.
Successful addition of voting disk 4ce7b7c682364f12bf4df5ce1fb7814e.
Successfully replaced voting disk group with +systemdg.
CRS-4266: Voting file(s) successfully replaced
[root@maclean1 ~]# crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 864a00efcfbe4f42bfd0f4f6b60472a0 (/dev/rasm-diskb) [SYSTEMDG]
2. ONLINE ab14d6e727614f29bf53b9870052a5c8 (/dev/rasm-diskc) [SYSTEMDG]
3. ONLINE 754c03c168854f46bf2daee7287bf260 (/dev/rasm-diskd) [SYSTEMDG]
4. ONLINE 9ed58f37f3e84f28bfcd9b101f2af9f3 (/dev/rasm-diske) [SYSTEMDG]
5. ONLINE 4ce7b7c682364f12bf4df5ce1fb7814e (/dev/rasm-diskf) [SYSTEMDG]
Located 5 voting disk(s).
[root@maclean1 ~]# ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 2844
Available space (kbytes) : 259276
ID : 879001605
Device/File Name : +SYSTEMDG
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
以上replace了votedisk到新的 ASM DISK上，并确认votedisk和OCR均为可用状态。
6.重启CRS服务：
[root@maclean1 ~]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'maclean1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'maclean1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'maclean1'
CRS-2673: Attempting to stop 'ora.asm' on 'maclean1'
CRS-2677: Stop of 'ora.mdnsd' on 'maclean1' succeeded
CRS-2677: Stop of 'ora.asm' on 'maclean1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'maclean1'
CRS-2677: Stop of 'ora.ctssd' on 'maclean1' succeeded
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'maclean1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'maclean1'
CRS-2677: Stop of 'ora.cssd' on 'maclean1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'maclean1'
CRS-2677: Stop of 'ora.gipcd' on 'maclean1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'maclean1'
CRS-2677: Stop of 'ora.gpnpd' on 'maclean1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'maclean1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@maclean1 ~]# crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.BACKUPDG.dg
ONLINE ONLINE maclean1
ora.DATA.dg
ONLINE ONLINE maclean1
ora.LISTENER.lsnr
ONLINE ONLINE maclean1
ora.SYSTEMDG.dg
ONLINE ONLINE maclean1
ora.asm
ONLINE ONLINE maclean1 Started
ora.gsd
OFFLINE OFFLINE maclean1
ora.net1.network
ONLINE ONLINE maclean1
ora.ons
ONLINE ONLINE maclean1
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE maclean1
ora.cvu
1 ONLINE ONLINE maclean1
ora.maclean1.vip
1 ONLINE ONLINE maclean1
ora.maclean2.vip
1 ONLINE INTERMEDIATE maclean1 FAILED OVER
ora.oc4j
1 ONLINE OFFLINE STARTING
ora.prod.db
1 ONLINE OFFLINE Instance Shutdown,S
TARTING
2 ONLINE OFFLINE
ora.scan1.vip
1 ONLINE ONLINE maclean1
因为上面更新了ASM共享使用的SPFILE，所以其他节点上一般不会存在问题，直接重启后CRS即可正常工作。
由于11gR2 RAC+ASM的启动依赖较为复杂，虽然通过以上操作但你仍可能在修改ASM DISK PATH路径时遇到一些问题， 若无法解决请去该问题的原帖提问，谢谢配合！