orcle 11g rac crs状态正常，节点2数据库未启动
安装完oracle11g R2 rac后，在节点1上查看数据库状态：
[oracle@node1 ~]$ sqlplus / as sysdba SQL*Plus: Release 11.2.0.4.0 Production on Wed May 17 18:56:34 2017 Copyright (c) 1982, 2013, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP, Data Mining and Real Application Testing options SQL> SELECT inst_id , instance_number inst_no , instance_name inst_name , parallel , status , database_status db_status , active_state state , host_name host FROM gv$instance ORDER BY inst_id; INST_ID INST_NO INST_NAME PAR STATUS DB_STATUS ---------- ---------- ---------------- --- ------------ ----------------- STATE HOST --------- ---------------------------------------------------------------- 1 1 orcl1 YES OPEN ACTIVE NORMAL node1 SQL> SQL> quit # 发现没有节点2的数据库信息
在节点2上查看数据库状态:
[oracle@node2 ~]$ sqlplus / as sysdba SQL*Plus: Release 11.2.0.4.0 Production on Wed May 17 18:57:48 2017 Copyright (c) 1982, 2013, Oracle. All rights reserved. Connected to an idle instance. SQL> SQL> SELECT inst_id , instance_number inst_no , instance_name inst_name , parallel , status , database_status db_status , active_state state , host_name host FROM gv$instance ORDER BY inst_id; SELECT inst_id , instance_number inst_no , instance_name inst_name , parallel , status , database_status db_status , active_state state , host_name host FROM gv$instance ORDER BY inst_id * ERROR at line 1: ORA-01034: ORACLE not available Process ID: 0 Session ID: 0 Serial number: 0 SQL> quit
显示数据库不可用,查看数据库进程,发现数据库进程不在：
[oracle@node2 ~]$ ps -ef | grep ora_ oracle 8718 8048 0 18:58 pts/0 00:00:00 grep --color=auto ora_
查看CRS状态：
[grid@node2 ~]$ crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora.DATA.dg ora....up.type ONLINE ONLINE node1 ora....ER.lsnr ora....er.type ONLINE ONLINE node1 ora....N1.lsnr ora....er.type ONLINE ONLINE node1 ora.OCRVOTE.dg ora....up.type ONLINE ONLINE node1 ora.asm ora.asm.type ONLINE ONLINE node1 ora.cvu ora.cvu.type ONLINE ONLINE node1 ora.gsd ora.gsd.type OFFLINE OFFLINE ora....network ora....rk.type ONLINE ONLINE node1 ora....SM1.asm application ONLINE ONLINE node1 ora....E1.lsnr application ONLINE ONLINE node1 ora.node1.gsd application OFFLINE OFFLINE ora.node1.ons application ONLINE ONLINE node1 ora.node1.vip ora....t1.type ONLINE ONLINE node1 ora....SM2.asm application ONLINE ONLINE node2 ora....E2.lsnr application ONLINE ONLINE node2 ora.node2.gsd application OFFLINE OFFLINE ora.node2.ons application ONLINE ONLINE node2 ora.node2.vip ora....t1.type ONLINE ONLINE node2 ora.oc4j ora.oc4j.type ONLINE ONLINE node1 ora.ons ora.ons.type ONLINE ONLINE node1 ora.orcl.db ora....se.type ONLINE ONLINE node1 ora.scan1.vip ora....ip.type ONLINE ONLINE node1 [grid@node2 ~]$ crsctl status res NAME=ora.DATA.dg TYPE=ora.diskgroup.type TARGET=ONLINE , ONLINE STATE=ONLINE on node1, ONLINE on node2 NAME=ora.LISTENER.lsnr TYPE=ora.listener.type TARGET=ONLINE , ONLINE STATE=ONLINE on node1, ONLINE on node2 NAME=ora.LISTENER_SCAN1.lsnr TYPE=ora.scan_listener.type TARGET=ONLINE STATE=ONLINE on node1 NAME=ora.OCRVOTE.dg TYPE=ora.diskgroup.type TARGET=ONLINE , ONLINE STATE=ONLINE on node1, ONLINE on node2 NAME=ora.asm TYPE=ora.asm.type TARGET=ONLINE , ONLINE STATE=INTERMEDIATE on node1, ONLINE on node2 NAME=ora.cvu TYPE=ora.cvu.type TARGET=ONLINE STATE=ONLINE on node1 NAME=ora.gsd TYPE=ora.gsd.type TARGET=OFFLINE, OFFLINE STATE=OFFLINE, OFFLINE NAME=ora.net1.network TYPE=ora.network.type TARGET=ONLINE , ONLINE STATE=ONLINE on node1, ONLINE on node2 NAME=ora.node1.vip TYPE=ora.cluster_vip_net1.type TARGET=ONLINE STATE=ONLINE on node1 NAME=ora.node2.vip TYPE=ora.cluster_vip_net1.type TARGET=ONLINE STATE=ONLINE on node2 NAME=ora.oc4j TYPE=ora.oc4j.type TARGET=ONLINE STATE=ONLINE on node1 NAME=ora.ons TYPE=ora.ons.type TARGET=ONLINE , ONLINE STATE=ONLINE on node1, ONLINE on node2 NAME=ora.orcl.db TYPE=ora.database.type TARGET=ONLINE , ONLINE STATE=ONLINE on node1, OFFLINE ----这里显示ora.orcl.db在node2上为OFFLINE NAME=ora.scan1.vip TYPE=ora.scan_vip.type TARGET=ONLINE STATE=ONLINE on node1
在节点2手动启动数据库：
[grid@node2 ~]$ srvctl start database -d orcl PRCC-1014 : orcl 已在运行 PRCR-1004 : 资源 ora.orcl.db 已在运行 PRCR-1079 : 无法启动资源 ora.orcl.db CRS-5017: The resource action "ora.orcl.db start" encountered the following error: ORA-01078: failure in processing system parameters ORA-01565: error in identifying file '+DATA/orcl/spfileorcl.ora' ORA-17503: ksfdopn:2 Failed to open file +DATA/orcl/spfileorcl.ora ORA-12547: TNS:lost contact . For details refer to "(:CLSN00107:)" in "/u01/app/11.2.0/grid/log/node2/agent/crsd/oraagent_oracle/oraagent_oracle.log". CRS-2674: Start of 'ora.orcl.db' on 'node2' failed CRS-2528: Unable to place an instance of 'ora.orcl.db' as all possible servers are occupied by the resource
解决办法：
联想到Oracle 11gR2 RAC DBCA无法识别ASM磁盘组时曾修改过grid用户下$ORACLE_HOME/bin/oracle的属性，当时是因为权限问题导致dbca无法访问创建好的ASM磁盘组，查看ORA-17503错误貌似是权限的问题，按照相同的方法修改后问题解决。
[grid@node2 bin]$ pwd /u01/app/11.2.0/grid/bin [grid@node2 bin]$ ll oracle -rwxr-x--x 1 grid oinstall 209854149 5月 16 15:36 oracle [grid@node2 bin]$ chmod +s oracle [grid@node2 bin]$ ll oracle -rwsr-s--x 1 grid oinstall 209854149 5月 16 15:36 oracle
然后启动数据库：
[grid@node2 bin]$ srvctl start database -d orcl #查看crs状态恢复正常 [grid@node2 ~]$ crsctl status res NAME=ora.DATA.dg TYPE=ora.diskgroup.type TARGET=ONLINE , ONLINE STATE=ONLINE on node1, ONLINE on node2 NAME=ora.LISTENER.lsnr TYPE=ora.listener.type TARGET=ONLINE , ONLINE STATE=ONLINE on node1, ONLINE on node2 NAME=ora.LISTENER_SCAN1.lsnr TYPE=ora.scan_listener.type TARGET=ONLINE STATE=ONLINE on node1 NAME=ora.OCRVOTE.dg TYPE=ora.diskgroup.type TARGET=ONLINE , ONLINE STATE=ONLINE on node1, ONLINE on node2 NAME=ora.asm TYPE=ora.asm.type TARGET=ONLINE , ONLINE STATE=INTERMEDIATE on node1, ONLINE on node2 NAME=ora.cvu TYPE=ora.cvu.type TARGET=ONLINE STATE=ONLINE on node1 NAME=ora.gsd TYPE=ora.gsd.type TARGET=OFFLINE, OFFLINE STATE=OFFLINE, OFFLINE NAME=ora.net1.network TYPE=ora.network.type TARGET=ONLINE , ONLINE STATE=ONLINE on node1, ONLINE on node2 NAME=ora.node1.vip TYPE=ora.cluster_vip_net1.type TARGET=ONLINE STATE=ONLINE on node1 NAME=ora.node2.vip TYPE=ora.cluster_vip_net1.type TARGET=ONLINE STATE=ONLINE on node2 NAME=ora.oc4j TYPE=ora.oc4j.type TARGET=ONLINE STATE=ONLINE on node1 NAME=ora.ons TYPE=ora.ons.type TARGET=ONLINE , ONLINE STATE=ONLINE on node1, ONLINE on node2 NAME=ora.orcl.db TYPE=ora.database.type TARGET=ONLINE , ONLINE STATE=ONLINE on node1, ONLINE on node2 NAME=ora.scan1.vip TYPE=ora.scan_vip.type TARGET=ONLINE STATE=ONLINE on node1 #查看数据库进程 [grid@node2 ~]$ ps -ef | grep ora_ oracle 9271 1 0 19:09 ? 00:00:00 ora_pmon_orcl2 oracle 9273 1 0 19:09 ? 00:00:00 ora_psp0_orcl2 oracle 9275 1 1 19:09 ? 00:00:06 ora_vktm_orcl2 oracle 9279 1 0 19:09 ? 00:00:00 ora_gen0_orcl2 oracle 9281 1 0 19:09 ? 00:00:00 ora_diag_orcl2 oracle 9283 1 0 19:09 ? 00:00:00 ora_dbrm_orcl2 oracle 9285 1 0 19:09 ? 00:00:00 ora_ping_orcl2 oracle 9287 1 0 19:09 ? 00:00:00 ora_acms_orcl2 oracle 9289 1 0 19:09 ? 00:00:00 ora_dia0_orcl2 oracle 9291 1 0 19:09 ? 00:00:00 ora_lmon_orcl2 oracle 9293 1 0 19:09 ? 00:00:01 ora_lmd0_orcl2 oracle 9295 1 0 19:09 ? 00:00:01 ora_lms0_orcl2 oracle 9299 1 0 19:09 ? 00:00:00 ora_rms0_orcl2 oracle 9301 1 0 19:09 ? 00:00:00 ora_lmhb_orcl2 oracle 9303 1 0 19:09 ? 00:00:00 ora_mman_orcl2 oracle 9305 1 0 19:09 ? 00:00:00 ora_dbw0_orcl2 oracle 9307 1 0 19:09 ? 00:00:00 ora_lgwr_orcl2 oracle 9309 1 0 19:09 ? 00:00:00 ora_ckpt_orcl2 oracle 9311 1 0 19:09 ? 00:00:00 ora_smon_orcl2 oracle 9313 1 0 19:09 ? 00:00:00 ora_reco_orcl2 oracle 9315 1 0 19:09 ? 00:00:00 ora_rbal_orcl2 oracle 9317 1 0 19:09 ? 00:00:00 ora_asmb_orcl2 oracle 9319 1 0 19:09 ? 00:00:00 ora_mmon_orcl2 oracle 9323 1 0 19:09 ? 00:00:00 ora_mmnl_orcl2 oracle 9325 1 0 19:09 ? 00:00:00 ora_d000_orcl2 oracle 9327 1 0 19:09 ? 00:00:00 ora_mark_orcl2 oracle 9329 1 0 19:09 ? 00:00:00 ora_s000_orcl2 oracle 9335 1 0 19:09 ? 00:00:00 ora_lck0_orcl2 oracle 9337 1 0 19:09 ? 00:00:00 ora_rsmn_orcl2 oracle 9400 1 0 19:10 ? 00:00:00 ora_gtx0_orcl2 oracle 9402 1 0 19:10 ? 00:00:00 ora_rcbg_orcl2 oracle 9404 1 0 19:10 ? 00:00:00 ora_qmnc_orcl2 oracle 9433 1 0 19:10 ? 00:00:00 ora_cjq0_orcl2 oracle 9443 1 0 19:10 ? 00:00:00 ora_q000_orcl2 oracle 9445 1 0 19:10 ? 00:00:00 ora_q001_orcl2 oracle 9657 1 0 19:15 ? 00:00:00 ora_smco_orcl2 oracle 9659 1 0 19:15 ? 00:00:00 ora_w000_orcl2 oracle 9684 1 0 19:16 ? 00:00:00 ora_pz99_orcl2 grid 9702 9501 0 19:17 pts/1 00:00:00 grep --color=auto ora_ [grid@node2 ~]$ crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora.DATA.dg ora....up.type ONLINE ONLINE node1 ora....ER.lsnr ora....er.type ONLINE ONLINE node1 ora....N1.lsnr ora....er.type ONLINE ONLINE node1 ora.OCRVOTE.dg ora....up.type ONLINE ONLINE node1 ora.asm ora.asm.type ONLINE ONLINE node1 ora.cvu ora.cvu.type ONLINE ONLINE node1 ora.gsd ora.gsd.type OFFLINE OFFLINE ora....network ora....rk.type ONLINE ONLINE node1 ora....SM1.asm application ONLINE ONLINE node1 ora....E1.lsnr application ONLINE ONLINE node1 ora.node1.gsd application OFFLINE OFFLINE ora.node1.ons application ONLINE ONLINE node1 ora.node1.vip ora....t1.type ONLINE ONLINE node1 ora....SM2.asm application ONLINE ONLINE node2 ora....E2.lsnr application ONLINE ONLINE node2 ora.node2.gsd application OFFLINE OFFLINE ora.node2.ons application ONLINE ONLINE node2 ora.node2.vip ora....t1.type ONLINE ONLINE node2 ora.oc4j ora.oc4j.type ONLINE ONLINE node1 ora.ons ora.ons.type ONLINE ONLINE node1 ora.orcl.db ora....se.type ONLINE ONLINE node1 ora.scan1.vip ora....ip.type ONLINE ONLINE node1
再次检查数据库状态，恢复正常：
[oracle@node2 ~]$ sqlplus / as sysdba SQL*Plus: Release 11.2.0.4.0 Production on Wed May 17 19:40:37 2017 Copyright (c) 1982, 2013, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP, Data Mining and Real Application Testing options SQL> SELECT inst_id , instance_number inst_no , instance_name inst_name , parallel , status , database_status db_status , active_state state , host_name host FROM gv$instance ORDER BY inst_id; INST_ID INST_NO INST_NAME PAR STATUS DB_STATUS ---------- ---------- ---------------- --- ------------ ----------------- STATE HOST --------- ---------------------------------------------------------------- 1 1 orcl1 YES OPEN ACTIVE NORMAL node1 2 2 orcl2 YES OPEN ACTIVE NORMAL node2 SQL>