CRS-4639: Could not contact Oracle High Availability Services解决方案集
在启动asm实例的时候报如下错误：
[grid@b1 ~]$ sqlplus / as sysasm
SQL*Plus: Release 11.2.0.1.0 Production on Thu Sep 12 18:14:13 2013
Copyright (c) 1982, 2009, Oracle. All rights reserved.
Connected to an idle instance.
SQL> startup
ORA-01078: failure in processing system parameters
ORA-29701: unable to connect to Cluster Synchronization Service
然后用crsctl check css检查的时候报如下错误：
[grid@b1 ~]$ crsctl check css
CRS-4639: Could not contact Oracle High Availability Services
CRS-4000: Command Check failed, or completed with errors.
解决CRS-4639: Could not contact Oracle High Availability Services过程如下：
[root@b1 grid]# cd /u01/app/11.2.0/grid/crs/install
[root@b1 install]# ./roothas.pl -deconfig -force -verbose
2013-09-12 19:25:05: Checking for super user privileges
2013-09-12 19:25:05: User has super user privileges
2013-09-12 19:25:05: Parsing the host name
Using configuration parameter file: ./crsconfig_params
CRS-4639: Could not contact Oracle High Availability Services
CRS-4000: Command Stop failed, or completed with errors.
CRS-4639: Could not contact Oracle High Availability Services
CRS-4000: Command Delete failed, or completed with errors.
Failure at scls_scr_getval with code 1
Internal Error Information:
Category: -2
Operation: opendir
Location: scrsearch1
Other: cant open scr home dir scls_scr_getval
System Dependent Information: 2
CRS-4544: Unable to connect to OHAS
CRS-4000: Command Stop failed, or completed with errors.
ACFS-9200: Supported
Successfully deconfigured Oracle Restart stack
[root@b1 install]# cd /u01/app/11.2.0/grid/
[root@b1 grid]# ./root.sh
Running Oracle 11g root.sh script...
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying dbhome to /usr/local/bin ...
The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying oraenv to /usr/local/bin ...
The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying coraenv to /usr/local/bin ...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2013-09-12 19:27:31: Checking for super user privileges
2013-09-12 19:27:31: User has super user privileges
2013-09-12 19:27:31: Parsing the host name
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
LOCAL ADD MODE
Creating OCR keys for user 'grid', privgrp 'oinstall'..
Operation successful.
CRS-4664: Node b1 successfully pinned.
Adding daemon to inittab
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
b1 2013/09/12 19:29:12 /u01/app/11.2.0/grid/cdata/b1/backup_20130912_192912.olr
Successfully configured Oracle Grid Infrastructure for a Standalone Server
Updating inventory properties for clusterware
Starting Oracle Universal Installer...
Checking swap space: must be greater than 500 MB. Actual 4094 MB Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /u01/app/oraInventory
'UpdateNodeList' was successful.
[grid@b1 ~]$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.cssd ora.cssd.type OFFLINE OFFLINE
ora.diskmon ora....on.type OFFLINE OFFLINE
[grid@b1 ~]$ crs_start -all
Attempting to start `ora.diskmon` on member `b1`
Attempting to start `ora.cssd` on member `b1`
Start of `ora.diskmon` on member `b1` succeeded.
Start of `ora.cssd` on member `b1` succeeded.
[grid@b1 ~]$ sqlplus / as sysasm
SQL*Plus: Release 11.2.0.1.0 Production on Thu Sep 12 19:34:50 2013
Copyright (c) 1982, 2009, Oracle. All rights reserved.
Connected to an idle instance.
SQL> startup
ASM instance started
Total System Global Area 283930624 bytes
Fixed Size 2212656 bytes
Variable Size 256552144 bytes
ASM Cache 25165824 bytes
ASM diskgroups mounted
ASM diskgroups volume enabled
----------->>第二种解决方法:
[grid@vm11gr2] /home/grid> sqlplus "/as sysasm"
Copyright (c) 1982, 2009, Oracle. All rights reserved.
Connected to an idle instance.
SQL> startup
ORA-01078: failure in processing system parameters
ORA-29701: unable to connect to Cluster Synchronization Service
SQL>
无法连接到CSS服务上.到操作系统上检查一下看看
[grid@vm11gr2] /home/grid> crsctl check css
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
[grid@vm11gr2] /home/grid>
[grid@vm11gr2] /home/grid> ps -ef|grep cssd
果然没有CSS的服务daemon进程,再看一下HAS(High Availability Service)的状态
[grid@vm11gr2] /home/grid> crsctl check has
CRS-4638: Oracle High Availability Services is online
[grid@vm11gr2] /home/grid> ps -ef|grep d.bin
grid 5886 1 0 10:06 ? 00:00:01 /u01/app/grid/product/11.2/grid/bin/ohasd.bin reboot
[grid@vm11gr2] /home/grid>
发现HAS的服务确实 启动了的,而ora.cssd和ora.diskmon这2个服务是依赖于HAS维护的.
进一步查看各资源的状态
[grid@vm11gr2] /home/grid> crs_stat -t
Name Type Target State Host
--------------------------------------------------------------
ora.FLASH_DATA.dg ora.diskgroup.type OFFLINE OFFLINE vm11gr2
ora.SYS_DATA.dg ora.diskgroup.type OFFLINE OFFLINE vm11gr2
ora.asm ora.asm.type OFFLINE OFFLINE vm11gr2
ora.cssd ora.cssd.type OFFLINE OFFLINE vm11gr2
ora.diskmon ora.diskmon.type OFFLINE OFFLINE vm11gr2
[grid@vm11gr2] /home/grid>
[grid@vm11gr2] /home/grid> crsctl status resource -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_ DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.FLASH_DATA.dg
OFFLINE OFFLINE vm11gr2
ora.SYS_DATA.dg
OFFLINE OFFLINE vm11gr2
ora.asm
OFFLINE OFFLINE vm11gr2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
1 OFFLINE OFFLINE
ora.diskmon
1 OFFLINE OFFLINE
再看一下ora.cssd和ora.diskmon的属性
[grid@vm11gr2] /home/grid> crs_stat -p ora.cssd
NAME=ora.cssd
TYPE=ora.cssd.type
ACTION_SCRIPT=
ACTIVE_PLACEMENT=0
AUTO_START=never
CHECK_INTERVAL=30
DESCRIPTION="Resource type for CSSD"
FAILOVER_DELAY=0
FAILURE_INTERVAL=3
FAILURE_THRESHOLD=5
HOSTING_MEMBERS=
PLACEMENT=balanced
RESTART_ATTEMPTS=5
SCRIPT_TIMEOUT=600
START_TIMEOUT=600
STOP_TIMEOUT=900
UPTIME_THRESHOLD=1m
[grid@vm11gr2] /home/grid> crs_stat -p ora.diskmon
NAME=ora.diskmon
TYPE=ora.diskmon.type
ACTION_SCRIPT=
ACTIVE_PLACEMENT=0
AUTO_START=never
CHECK_INTERVAL=20
DESCRIPTION="Resource type for Diskmon"
FAILOVER_DELAY=0
FAILURE_INTERVAL=3
FAILURE_THRESHOLD=5
HOSTING_MEMBERS=
PLACEMENT=balanced
RESTART_ATTEMPTS=10
SCRIPT_TIMEOUT=60
START_TIMEOUT=60
STOP_TIMEOUT=60
UPTIME_THRESHOLD=5s
[grid@vm11gr2] /home/grid>
到这里基本就找到了原因了,可以看到这两个资源的AUTO_START属性默认都设置为never,也就是说他们不会随着HAS服务的启动而自动启动的,
尽管默认情况下HAS服务是开机自动启动的.好了，那我们就手动启动一下吧:
[grid@vm11gr2] /home/grid> crsctl start resource ora.cssd
CRS-2672: Attempting to start 'ora.cssd' on 'vm11gr2'
CRS-2679: Attempting to clean 'ora.diskmon' on 'vm11gr2'
CRS-2681: Clean of 'ora.diskmon' on 'vm11gr2' succeeded
CRS-2672: Attempting to start 'ora.diskmon' on 'vm11gr2'
CRS-2676: Start of 'ora.diskmon' on 'vm11gr2' succeeded
CRS-2676: Start of 'ora.cssd' on 'vm11gr2' succeeded
[grid@vm11gr2] /home/grid>
注:ora.cssd和ora.diskmon这两个服务是有依赖关系的，启动哪个都会把两个都起来.
[grid@vm11gr2] /home/grid> crs_stat -t
Name Type Target State Host
--------------------------------------------------------------
ora.FLASH_DATA.dg ora.diskgroup.type OFFLINE OFFLINE vm11gr2
ora.SYS_DATA.dg ora.diskgroup.type OFFLINE OFFLINE vm11gr2
ora.asm ora.asm.type OFFLINE OFFLINE vm11gr2
ora.cssd ora.cssd.type ONLINE ONLINE vm11gr2
ora.diskmon ora.diskmon.type ONLINE ONLINE vm11gr2
[grid@vm11gr2] /home/grid>
CSS服务起来了，重启动asm instance
[grid@vm11gr2] /home/grid> sqlplus "/as sysasm"
SQL*Plus: Release 11.2.0.1.0 Production on Sun Oct 25 10:30:03 2009
Copyright (c) 1982, 2009, Oracle. All rights reserved.
Connected to an idle instance.
SQL> startup
ASM instance started
Total System Global Area 284565504 bytes
Fixed Size 1336036 bytes
Variable Size 258063644 bytes
ASM Cache 25165824 bytes
ASM diskgroups mounted
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - Production
With the Automatic Storage Management option
[grid@vm11gr2] /home/grid> crs_stat -t
Name Type Target State Host
--------------------------------------------------------------
ora.FLASH_DATA.dg ora.diskgroup.type ONLINE ONLINE vm11gr2
ora.SYS_DATA.dg ora.diskgroup.type ONLINE ONLINE vm11gr2
ora.asm ora.asm.type ONLINE ONLINE vm11gr2
ora.cssd ora.cssd.type ONLINE ONLINE vm11gr2
ora.diskmon ora.diskmon.type ONLINE ONLINE vm11gr2
[grid@vm11gr2] /home/grid>
tips
1)默认情况下HAS(High Availability Service)是自动启动的.通过如下命令可以取消和启用自动启动
crsctl disable has
crsctl enable has
2)HAS手动启动和停止
crsctl start has
crsctl stop has
3)查看HAS的状态
crsctl check has
4)如果想让ora.css和ora.diskmon服务随着HAS的启动而自动启动，那么你可以这两个服务的AUTO_START属性
crsctl modify resource "ora.cssd" -attr "AUTO_START=1"
or
crsctl modify resource "ora.diskmon" -attr "AUTO_START=1"
5)如果想取消ora.css和ora.diskmon的Auto start
crsctl modify resource "ora.cssd" -attr "AUTO_START=never"
crsctl modify resource "ora.diskmon" -attr "AUTO_START=never"
------>>11gR2启动ASM实例时遭遇ORA-29701
[grid@vm11gr2] /home/grid> sqlplus "/as sysasm" SQL*Plus: Release 11.2.0.1.0 Production on Sun Oct 25 10:16:21 2009 Copyright (c) 1982, 2009, Oracle. All rights reserved. Connected to an idle instance. SQL> startup ORA-01078: failure in processing system parameters ORA-29701: unable to connect to Cluster Synchronization Service SQL> 无法连接到CSS服务上.到操作系统上检查一下看看 [grid@vm11gr2] /home/grid> crsctl check css CRS-4530: Communications failure contacting Cluster Synchronization Services daemon [grid@vm11gr2] /home/grid> [grid@vm11gr2] /home/grid> ps -ef|grep cssd 果然没有CSS的服务daemon进程,再看一下HAS(High Availability Service)的状态 [grid@vm11gr2] /home/grid> crsctl check has CRS-4638: Oracle High Availability Services is online [grid@vm11gr2] /home/grid> ps -ef|grep d.bin grid 5886 1 0 10:06 ? 00:00:01 /u01/app/grid/product/11.2/grid/bin/ohasd.bin reboot [grid@vm11gr2] /home/grid> 发现HAS的服务确实启动了的,而ora.cssd和ora.diskmon这2个服务是依赖于HAS维护的. 进一步查看各资源的状态 [grid@vm11gr2] /home/grid> crs_stat -t Name Type Target State Host -------------------------------------------------------------- ora.FLASH_DATA.dg ora.diskgroup.type OFFLINE OFFLINE vm11gr2 ora.SYS_DATA.dg ora.diskgroup.type OFFLINE OFFLINE vm11gr2 ora.asm ora.asm.type OFFLINE OFFLINE vm11gr2 ora.cssd ora.cssd.type OFFLINE OFFLINE vm11gr2 ora.diskmon ora.diskmon.type OFFLINE OFFLINE vm11gr2 [grid@vm11gr2] /home/grid> [grid@vm11gr2] /home/grid> crsctl status resource -t -------------------------------------------------------------------------------- NAME TARGET STATE SERVER STATE_ DETAILS -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.FLASH_DATA.dg OFFLINE OFFLINE vm11gr2 ora.SYS_DATA.dg OFFLINE OFFLINE vm11gr2 ora.asm OFFLINE OFFLINE vm11gr2 -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.cssd 1 OFFLINE OFFLINE ora.diskmon 1 OFFLINE OFFLINE 再看一下ora.cssd和ora.diskmon的属性 [grid@vm11gr2] /home/grid> crs_stat -p ora.cssd NAME=ora.cssd TYPE=ora.cssd.type ACTION_SCRIPT= ACTIVE_PLACEMENT=0 AUTO_START=never CHECK_INTERVAL=30 DESCRIPTION="Resource type for CSSD" FAILOVER_DELAY=0 FAILURE_INTERVAL=3 FAILURE_THRESHOLD=5 HOSTING_MEMBERS= PLACEMENT=balanced RESTART_ATTEMPTS=5 SCRIPT_TIMEOUT=600 START_TIMEOUT=600 STOP_TIMEOUT=900 UPTIME_THRESHOLD=1m [grid@vm11gr2] /home/grid> crs_stat -p ora.diskmon NAME=ora.diskmon TYPE=ora.diskmon.type ACTION_SCRIPT= ACTIVE_PLACEMENT=0 AUTO_START=never CHECK_INTERVAL=20 DESCRIPTION="Resource type for Diskmon" FAILOVER_DELAY=0 FAILURE_INTERVAL=3 FAILURE_THRESHOLD=5 HOSTING_MEMBERS= PLACEMENT=balanced RESTART_ATTEMPTS=10 SCRIPT_TIMEOUT=60 START_TIMEOUT=60 STOP_TIMEOUT=60 UPTIME_THRESHOLD=5s [grid@vm11gr2] /home/grid> 到这里基本就找到了原因了,可以看到这两个资源的AUTO_START属性默认都设置为never,也就是说他们不会随着HAS服务的启动而自动启动的, 尽管默认情况下HAS服务是开机自动启动的.好了，那我们就手动启动一下吧: [grid@vm11gr2] /home/grid> crsctl start resource ora.cssd CRS-2672: Attempting to start 'ora.cssd' on 'vm11gr2' CRS-2679: Attempting to clean 'ora.diskmon' on 'vm11gr2' CRS-2681: Clean of 'ora.diskmon' on 'vm11gr2' succeeded CRS-2672: Attempting to start 'ora.diskmon' on 'vm11gr2' CRS-2676: Start of 'ora.diskmon' on 'vm11gr2' succeeded CRS-2676: Start of 'ora.cssd' on 'vm11gr2' succeeded [grid@vm11gr2] /home/grid> 注:ora.cssd和ora.diskmon这两个服务是有依赖关系的，启动哪个都会把两个都起来. [grid@vm11gr2] /home/grid> crs_stat -t Name Type Target State Host -------------------------------------------------------------- ora.FLASH_DATA.dg ora.diskgroup.type OFFLINE OFFLINE vm11gr2 ora.SYS_DATA.dg ora.diskgroup.type OFFLINE OFFLINE vm11gr2 ora.asm ora.asm.type OFFLINE OFFLINE vm11gr2 ora.cssd ora.cssd.type ONLINE ONLINE vm11gr2 ora.diskmon ora.diskmon.type ONLINE ONLINE vm11gr2 [grid@vm11gr2] /home/grid> CSS服务起来了，重启动asm instance [grid@vm11gr2] /home/grid> sqlplus "/as sysasm" SQL*Plus: Release 11.2.0.1.0 Production on Sun Oct 25 10:30:03 2009 Copyright (c) 1982, 2009, Oracle. All rights reserved. Connected to an idle instance. SQL> startup ASM instance started Total System Global Area 284565504 bytes Fixed Size 1336036 bytes Variable Size 258063644 bytes ASM Cache 25165824 bytes ASM diskgroups mounted SQL> exit Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - Production With the Automatic Storage Management option [grid@vm11gr2] /home/grid> crs_stat -t Name Type Target State Host -------------------------------------------------------------- ora.FLASH_DATA.dg ora.diskgroup.type ONLINE ONLINE vm11gr2 ora.SYS_DATA.dg ora.diskgroup.type ONLINE ONLINE vm11gr2 ora.asm ora.asm.type ONLINE ONLINE vm11gr2 ora.cssd ora.cssd.type ONLINE ONLINE vm11gr2 ora.diskmon ora.diskmon.type ONLINE ONLINE vm11gr2 [grid@vm11gr2] /home/grid> tips 1)默认情况下HAS(High Availability Service)是自动启动的.通过如下命令可以取消和启用自动启动 crsctl disable has crsctl enable has 2)HAS手动启动和停止 crsctl start has crsctl stop has 3)查看HAS的状态 crsctl check has 4)如果想让ora.css和ora.diskmon服务随着HAS的启动而自动启动，那么你可以这两个服务的AUTO_START属性 crsctl modify resource "ora.cssd" -attr "AUTO_START=1" or crsctl modify resource "ora.diskmon" -attr "AUTO_START=1" 5)如果想取消ora.css和ora.diskmon的Auto start crsctl modify resource "ora.cssd" -attr "AUTO_START=never" crsctl modify resource "ora.diskmon" -attr "AUTO_START=never" ================================================================================== 自己遇到的情况是：
[grid@vm11gr2] /home/grid> sqlplus "/as sysasm" SQL*Plus: Release 11.2.0.1.0 Production on Sun Oct 25 10:16:21 2009
Copyright (c) 1982, 2009, Oracle. All rights reserved.
Connected to an idle instance.
SQL> startup
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+RAC_DISK/RACDB/spfileRACDB.ora'
ORA-17503: ksfdopn:2 Failed to open file +RAC_DISK/RACDB/spfileRACDB.ora
ORA-15077: could not locate ASM instance serving a required diskgroup
ORA-29701: unable to connect to Cluster Synchronization Service ---->>感同身受篇:rac02数据库异常停机报错:
Error: received an ABORT event from the Cluster Synchronization Service
Errors in file /u01/app/oracle/diag/rdbms/sun/sun/trace/sun_lmon_25888.trc:
ORA-29701: unable to connect to Cluster Synchronization Service
LMON (ospid: 25888): terminating the instance due to error 29701
Wed Nov 19 16:17:51 2014
System state dump is made for local instance
System State dumped to trace file /u01/app/oracle/diag/rdbms/sun/sun/trace/sun_diag_25864.trc
Instance terminated by LMON, pid = 25888
Wed Nov 19 23:38:23 2014
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Interface type 1 eth1 172.16.2.0 configured from GPnP Profile for use as a cluster interconnect
Interface type 1 eth0 192.168.2.0 configured from GPnP Profile for use as a public interface
Picked latch-free SCN scheme 3
Using LOG_ARCHIVE_DEST_1 parameter default value as USE_DB_RECOVERY_FILE_DEST
Autotune of undo retention is turned on.
IMODE=BR
ILAT =88
LICENSE_MAX_USERS = 0
SYS auditing is disabled
NOTE: Volume support enabled
Starting up:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options.
--sun_lmon_25888.trc内容:
[oracle@rac02 trace]$ cat sun_lmon_25888.trc
Trace file /u01/app/oracle/diag/rdbms/bus/bus/trace/sun_lmon_25888.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
ORACLE_HOME = /u01/app/oracle/product/11.2.0/db_1
System name: Linux
Node name: rac02
Release: 2.6.18-308.el5
Version: #1 SMP Fri Jan 27 17:17:51 EST 2012
Machine: x86_64
Instance name: sun
Redo thread mounted by this instance: 1
Oracle process number: 11
Unix process pid: 25888, image: oracle@rac02 (LMON)
*** 2014-11-19 16:17:51.745
*** SESSION ID:(276.1) 2014-11-19 16:17:51.745
*** CLIENT ID:() 2014-11-19 16:17:51.745
*** SERVICE NAME:(SYS$BACKGROUND) 2014-11-19 16:17:51.745
*** MODULE NAME:() 2014-11-19 16:17:51.745
*** ACTION NAME:() 2014-11-19 16:17:51.745
*** TRACE FILE RECREATED AFTER BEING REMOVED ***
2014-11-19 16:17:51.744: [ CSSCLNT]clsssRecvMsg: got a disconnect from the server while waiting for message type 1
2014-11-19 16:17:51.745: [ CSSCLNT]clssgsGroupGetStatus: communications failed (0/3/-1)
2014-11-19 16:17:51.745: [ CSSCLNT]clssgsGroupGetStatus: returning 8
kgxgnpstat: received ABORT event from CLSS
kjxggpoll: received an abort event from DBALL_DB
Error: Cluster Synchronization Service aborts the instance (error 6)
LMON caught an error 29701 in the main loop
error 29701 detected in background process
ORA-29701: unable to connect to Cluster Synchronization Service
*** 2014-11-19 16:17:51.754
LMON (ospid: 25888): terminating the instance due to error 29701
ksuitm: waiting up to [5] seconds before killing DIAG(25864)
来自 “ ITPUB博客 ” ，链接：http://blog.itpub.net/29119536/viewspace-1314318/，如需转载，请注明出处，否则将追究法律责任。
转载于:http://blog.itpub.net/29119536/viewspace-1314318/