Oracle 11g RAC 执行root.sh时遭遇 CRS-0184/PRCR-1070

Oracle 11g RAC安装时，在第一个节点执行root.sh时遭遇了CRS-0184/PRCR-1070，Google了很多帖子也没有找到解决办法。呜呜，还是静下心来看日志！！最后的发现原来是一个不经意的小问题，如下面的描述。

 

1、故障现象

#安装环境
[root@linux1 ~]# more /etc/issue
Enterprise Linux Enterprise Linux Server release 5.5 (Carthage)
Kernel \r on an \m

[root@linux1 ~]# uname -a
Linux linux1.orasrv.com 2.6.18-194.el5PAE #1 SMP Mon Mar 29 20:19:03 EDT 2010 i686 athlon i386 GNU/Linux

[root@linux1 bin]# ./crsctl query crs activeversion
Oracle Clusterware active version on the cluster is [11.2.0.1.0]

#执行root时，出现了下面的错误，仅列出错误部分
CRS-0184: Cannot communicate with the CRS daemon.   #这个是主要的错误提示，不能同crs通信

PRCR-1070 : Failed to check if resource ora.asm is registered
Cannot communicate with crsd
add asm ... failed
clsr_start_dg return error at loc: 70 rc=0
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Create failed, or completed with errors.
create diskgroup DATA ... failed
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Add failed, or completed with errors.
Command return code of 1 (256) from command: /u01/app/11.2.0/grid/bin/crsctl add type ora.registry.acfs.type -basetype 
   ora.local_resource.type -file /u01/app/11.2.0/grid/crs/template/registry.acfs.type
add ora.registry.acfs.type ... failed
PRCR-1070 : Failed to check if resource ora.net1.network is registered
Cannot communicate with crsd
add scan=scan-cluster.orasrv.com ... failed
Preparing packages for installation...
cvuqdisk-1.0.7-1                                   #这里有个提示，需要安装cvuqdisk-1.0.7-1包
Configure Oracle Grid Infrastructure for a Cluster ... failed
Updating inventory properties for clusterware
Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB.   Actual 3498 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /u01/app/oraInventory
'UpdateNodeList' was successful.

#使用crsctl check时收到了crs-4535错误
[root@linux1 bin]# ./crsctl check cluster      
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online

2、分析日志

#查看grid的crsd部分的日志，注意该日志位于$CRS_HOME/log/node_name/crsd
[root@linux1 crsd]# tail -30 crsdOUT.log  
2013-07-16 16:27:21  
CRSD REBOOT
Unable to resolve address for localhost:2016
ONS runtime exiting
Fatal error: eONS: eonsapi.c: Aug  4 2009 21:09:36
2013-07-16 16:27:24  
Changing directory to /u01/app/11.2.0/grid/log/linux1/crsd
2013-07-16 16:27:24  
CRSD REBOOT
Unable to resolve address for localhost:2016
ONS runtime exiting
Fatal error: eONS: eonsapi.c: Aug  4 2009 21:09:36
2013-07-16 16:27:26  
Changing directory to /u01/app/11.2.0/grid/log/linux1/crsd
2013-07-16 16:27:26  
CRSD REBOOT
Unable to resolve address for localhost:2016
ONS runtime exiting
Fatal error: eONS: eonsapi.c: Aug  4 2009 21:09:36
2013-07-16 16:27:29  
Changing directory to /u01/app/11.2.0/grid/log/linux1/crsd
2013-07-16 16:27:29  
CRSD REBOOT
Unable to resolve address for localhost:2016
ONS runtime exiting
Fatal error: eONS: eonsapi.c: Aug  4 2009 21:09:36
2013-07-16 16:48:00  
Changing directory to /u01/app/11.2.0/grid/log/linux1/crsd
2013-07-16 16:48:00  
CRSD REBOOT
#上面的日志提示不能够解析localhost

#看看grid的主日志，也就是alert_nodename部分的描述
[root@linux1 linux1]# tail -100 alertlinux1.log | more
2013-07-16 16:27:13.688
[crsd(3913)]CRS-1201:CRSD started on node linux1.
2013-07-16 16:27:14.338
[/u01/app/11.2.0/grid/bin/oraagent.bin(3993)]CRS-5823:Could not initialize agent framework. Details at (:CRSAGF00120:) in
 /u01/app/11.2.0/grid/log/linux1/agent/crsd/oraagent_grid/oraagent_grid.log.
2013-07-16 16:27:14.932
[ohasd(2748)]CRS-2765:Resource 'ora.crsd' has failed on server 'linux1'.
# Meatlink [ID 1230899.1] 有关于该问题的描述，就是需要添加localhost到hosts文件
# Author : Robinson
# Blog   : http://blog.csdn.net/robinson_0612

3、解决故障

#查看本地的hosts文件，下面是更改后的hosts配置文件
[root@linux1 bin]# more /etc/hosts | grep 127.0
#127.0.0.1              linux1.orasrv.com linux1 localhost.localdomain localhost  #这个是安装OS时自动生成的
#127.0.0.1               linux1.orasrv.com linux1          #这个是安装rac之前配置的，看来这个写法有问题，一定要写localhost
127.0.0.1  localhost.orasrv.com localhost                  #添加一个新的关于localhost的解析entry

#尝试启动crs
[root@linux1 bin]# ./crsctl start res ora.crsd -init
CRS-2672: Attempting to start 'ora.crsd' on 'linux1'
CRS-2676: Start of 'ora.crsd' on 'linux1' succeeded

#crs成功启动
[root@linux1 bin]# ./crsctl check cluster
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online

[root@linux1 bin]# ./crsctl stat res -t -init
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS       
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
      1        ONLINE  ONLINE       linux1                   Started             
ora.crsd
      1        ONLINE  ONLINE       linux1                                       
ora.cssd
      1        ONLINE  ONLINE       linux1                                       
ora.cssdmonitor
      1        ONLINE  ONLINE       linux1                                       
ora.ctssd
      1        ONLINE  ONLINE       linux1                   ACTIVE:0            
ora.diskmon
      1        ONLINE  ONLINE       linux1                                       
ora.drivers.acfs
      1        ONLINE  ONLINE       linux1                                       
ora.evmd
      1        ONLINE  ONLINE       linux1                                       
ora.gipcd
      1        ONLINE  ONLINE       linux1                                       
ora.gpnpd
      1        ONLINE  ONLINE       linux1                                       
ora.mdnsd
      1        ONLINE  ONLINE       linux1      

 

更多参考

有关Oracle RAC请参考
     使用crs_setperm修改RAC资源的所有者及权限
     使用crs_profile管理RAC资源配置文件
     RAC 数据库的启动与关闭
     再说 Oracle RAC services
     Services in Oracle Database 10g
     Migrate datbase from single instance to Oracle RAC
     Oracle RAC 连接到指定实例
     Oracle RAC 负载均衡测试(结合服务器端与客户端)
     Oracle RAC 服务器端连接负载均衡(Load Balance)
     Oracle RAC 客户端连接负载均衡(Load Balance)
     ORACLE RAC 下非缺省端口监听配置(listener.ora tnsnames.ora)
     ORACLE RAC 监听配置 (listener.ora tnsnames.ora)
     配置 RAC 负载均衡与故障转移
     CRS-1006 , CRS-0215 故障一例 
     基于Linux (RHEL 5.5) 安装Oracle 10g RAC
     使用 runcluvfy 校验Oracle RAC安装环境

有关Oracle 网络配置相关基础以及概念性的问题请参考：
     配置非默认端口的动态服务注册
     配置sqlnet.ora限制IP访问Oracle
     Oracle 监听器日志配置与管理
     设置 Oracle 监听器密码(LISTENER)
     配置ORACLE 客户端连接到数据库

有关基于用户管理的备份和备份恢复的概念请参考
     Oracle 冷备份
     Oracle 热备份
     Oracle 备份恢复概念
     Oracle 实例恢复
     Oracle 基于用户管理恢复的处理
     SYSTEM 表空间管理及备份恢复
     SYSAUX表空间管理及恢复
     Oracle 基于备份控制文件的恢复(unsing backup controlfile)

有关RMAN的备份恢复与管理请参考
     RMAN 概述及其体系结构
     RMAN 配置、监控与管理
     RMAN 备份详解
     RMAN 还原与恢复
     RMAN catalog 的创建和使用
     基于catalog 创建RMAN存储脚本
     基于catalog 的RMAN 备份与恢复
     RMAN 备份路径困惑
     使用RMAN实现异机备份恢复(WIN平台)
     使用RMAN迁移文件系统数据库到ASM
     linux 下RMAN备份shell脚本
     使用RMAN迁移数据库到异机

有关ORACLE体系结构请参考
     Oracle 表空间与数据文件
     Oracle 密码文件
     Oracle 参数文件
     Oracle 联机重做日志文件(ONLINE LOG FILE)
     Oracle 控制文件(CONTROLFILE)
     Oracle 归档日志
     Oracle 回滚(ROLLBACK)和撤销(UNDO)
     Oracle 数据库实例启动关闭过程
     Oracle 10g SGA 的自动化管理
     Oracle 实例和Oracle数据库(Oracle体系结构)