oracle11g模拟OCR和voting disk损坏恢复实验

检查ocr备份
[root@rac1 ~]# ocrconfig -showbackup
PROT-24: Auto backups for the Oracle Cluster Registry are not available

rac1     2018/06/24 01:56:51     /tpsys/app/11.2.0/grid/cdata/rac-cluster/backup_20180624_015651.ocr
rac1     2018/06/24 01:55:41     /tpsys/app/11.2.0/grid/cdata/rac-cluster/backup_20180624_015541.ocr
rac2     2018/06/12 06:14:35     /tpsys/app/11.2.0/grid/cdata/rac-cluster/backup_20180612_061435.ocr

检查vote disk
[root@rac1 ~]# crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   0a80756b9eb44f9abfae577e0e8ed2dd (/dev/raw/raw5) [OCRDG]
 2. ONLINE   000808a364544f11bf48969149e2bf2a (/dev/raw/raw6) [OCRDG]
 3. ONLINE   8714d5dd04634f70bf3bbd8ae1a467ac (/dev/raw/raw7) [OCRDG]
Located 3 voting disk(s).

数据库版本：
[oracle@rac2 ~]$ sqlplus -V
SQL*Plus: Release 11.2.0.1.0 Production

创建表
create table ming180623 as select * from dba_objects;

备份asm spfile：
create pfile='/home/grid/asmpfile.ora' from spfile;

使用dd命令来破坏OCR
dd if=/dev/zero of=/dev/sdf bs=1M count=100
dd if=/dev/zero of=/dev/sdh bs=1M count=100

关闭crs
[root@rac1 ~]# crsctl stop crs -f

启动crs
[root@rac1 ~]# crsctl start crs
CRS-4123: Oracle High Availability Services has been started.

css日志：
2018-06-24 02:30:14.838: [    CSSD][3690985216]clssnmvDiskVerify: file is not a voting file, cannot recognize on-disk signature for a voting
2018-06-24 02:30:14.838: [    CSSD][3690985216]clssnmvDiskVerify: file is not a voting file, cannot recognize on-disk signature for a voting
2018-06-24 02:30:14.838: [    CSSD][3690985216]clssnmvDiskVerify: file is not a voting file, cannot recognize on-disk signature for a voting
2018-06-24 02:30:14.838: [    CSSD][3690985216]clssnmvDiskVerify: file is not a voting file, cannot recognize on-disk signature for a voting
2018-06-24 02:30:14.838: [    CSSD][3690985216]clssnmvDiskVerify: file is not a voting file, cannot recognize on-disk signature for a voting
2018-06-24 02:30:14.838: [    CSSD][3690985216]clssnmvDiskVerify: file is not a voting file, cannot recognize on-disk signature for a voting
2018-06-24 02:30:14.838: [    CSSD][3690985216]clssnmvDiskVerify: file is not a voting file, cannot recognize on-disk signature for a voting
2018-06-24 02:30:14.838: [    CSSD][3690985216]clssnmvDiskVerify: Successful discovery of 0 disks
2018-06-24 02:30:14.838: [    CSSD][3690985216]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery
2018-06-24 02:30:14.838: [    CSSD][3690985216]clssnmvFindInitialConfigs: No voting files found

crs日志：
2018-06-24 02:31:07.379: [ CSSCLNT][3746670368]clssscConnect: gipc request failed with 29 (0x16)
2018-06-24 02:31:07.379: [ CSSCLNT][3746670368]clsssInitNative: connect failed, rc 29
2018-06-24 02:31:07.379: [  CRSRTI][3746670368] CSS is not ready. Received status 3 from CSS. Waiting for good status .. 

2018-06-24 02:31:08.381: [ CSSCLNT][3746670368]clssscConnect: gipc request failed with 29 (0x16)
2018-06-24 02:31:08.381: [ CSSCLNT][3746670368]clsssInitNative: connect failed, rc 29
2018-06-24 02:31:08.381: [  CRSRTI][3746670368] CSS is not ready. Received status 3 from CSS. Waiting for good status .. 

2018-06-24 02:31:09.383: [ CSSCLNT][3746670368]clssscConnect: gipc request failed with 29 (0x16)
2018-06-24 02:31:09.383: [ CSSCLNT][3746670368]clsssInitNative: connect failed, rc 29
2018-06-24 02:31:09.384: [  CRSRTI][3746670368] CSS is not ready. Received status 3 from CSS. Waiting for good status .. 

css没有发现votedisk已经起不来了，crs自然也没起来。
[root@rac1 ~]# crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
CRS-4533: Event Manager is online

[root@rac1 ~]# crsctl query css votedisk
Unable to communicate with the Cluster Synchronization Services daemon.

关闭两个节点：
crsctl stop crs -f

在一个节点以独占模式启动：
[root@rac1 ~]# crsctl start crs -excl 
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
CRS-2679: Attempting to clean 'ora.diskmon' on 'rac1'
CRS-2681: Clean of 'ora.diskmon' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac1'
CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded

注意：
11.2.0.1:
# $CRS_HOME/bin/crsctl start crs -excl
11.2.0.2 and above:
# $CRS_HOME/bin/crsctl start crs -excl -nocrs

[root@rac1 ~]# crsctl stop resource ora.crsd -init

grid用户：
sqlplus / as sysasm
SQL> select NAME,STATE,GROUP_NUMBER from v$asm_diskgroup;
NAME                           STATE                             GROUP_NUMBER
------------------------------ --------------------------------- ------------
RAC_DATA                       DISMOUNTED                                   0

创建ocr磁盘组：
create diskgroup OCRDG NORMAL redundancy disk '/dev/raw/raw5','/dev/raw/raw6','/dev/raw/raw7' attribute 'COMPATIBLE.ASM' = '11.2';
NAME       STATE            TOTAL_MB    FREE_MB USABLE_FILE_MB
---------- -------------- ---------- ---------- --------------
RAC_DATA   DISMOUNTED              0          0              0
OCRDG      MOUNTED              3000       2841           1394

注意，如果新创建的磁盘组名字和之前不一样，那么两个节点的ocr.loc文件内容要去手动修改一下：
[grid@rac1 ~]$ more /etc/oracle/ocr.loc
#Device/file +OCRVOTING getting replaced by device +OCRDG 
ocrconfig_loc=+OCRDG
local_only=false

还原OCR：
[root@rac1 raw]# ocrconfig -restore /tpsys/app/11.2.0/grid/cdata/rac-cluster/backup_20180624_015651.ocr

回到asm里看一下，大小已经发生了变化：
SQL> /
NAME         STATE            TOTAL_MB    FREE_MB USABLE_FILE_MB
------------ -------------- ---------- ---------- --------------
RAC_DATA     DISMOUNTED              0          0              0
OCRDG        MOUNTED              3000       2298           1031

11.2.0.1需要启动一下crs资源，11.2.0.2跳过：
[root@rac1 raw]# crsctl start res ora.crsd -init
CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded

此时ocrcheck已经可以正常显示结果了，但是crsctl query css votedisk显示没有找到vote disk：
[root@rac1 raw]# crsctl query css votedisk
Located 0 voting disk(s).

重建vote disk：
[root@rac1 raw]# crsctl replace votedisk +OCRDG
Successful addition of voting disk a186c141f39f4f5fbf3338b77ade330c.
Successful addition of voting disk 0070856935154f18bf44c916cd903b4c.
Successful addition of voting disk 290bc69f5e054febbfa9ea3b6674fc8c.
Successfully replaced voting disk group with +OCRDG.
CRS-4266: Voting file(s) successfully replaced

[root@rac1 raw]# crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   a186c141f39f4f5fbf3338b77ade330c (/dev/raw/raw5) [OCRDG]
 2. ONLINE   0070856935154f18bf44c916cd903b4c (/dev/raw/raw6) [OCRDG]
 3. ONLINE   290bc69f5e054febbfa9ea3b6674fc8c (/dev/raw/raw7) [OCRDG]
Located 3 voting disk(s).

重建asm spfile：
create spfile='+OCRDG' from pfile='/home/grid/asmpfile.ora';

关闭crs：
crsctl stop crs -f

两个节点重启crs：
crsctl start crs

检查一下：
crsctl stat res -t
ocrcheck
crsctl query css votedisk
crsctl check cluster -all

来自 “ ITPUB博客 ” ，链接：http://blog.itpub.net/31480688/viewspace-2156570/，如需转载，请注明出处，否则将追究法律责任。

转载于:http://blog.itpub.net/31480688/viewspace-2156570/