CRS-4639: Could not contact Oracle High Availability Services

P720:/home/grid&sqlplus / as sysasm

SQL*Plus: Release 11.2.0.3.0 Production on Fri Jan 4 11:24:19 2013

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup
ORA-01078: failure in processing system parameters
ORA-29701: unable to connect to Cluster Synchronization Service
SQL> exit
Disconnected

P720:/home/grid&crs_stat -t
CRS-0184: Cannot communicate with the CRS daemon.

P720:/home/grid&crs_start -all
CRS-0184: Cannot communicate with the CRS daemon.

P720:/home/grid&ps -ef|grep crs
    grid 6881296 8192068   0 11:29:03  pts/2  0:00 grep crs
P720:/g01/grid/asm/log/p720/cssd&

 

&crsctl enable has
CRS-4622: Oracle High Availability Services autostart is enabled.
&crsctl disable has
CRS-4621: Oracle High Availability Services autostart is disabled.

 

/g01/grid/asm/bin/crs_stat -t
/g01/grid/asm/bin/crs_start –all
/g01/grid/asm/bin/ocrcheck
/g01/grid/asm/bin/crsctl start resource ora.cssd

/g01/grid/asm/bin/crsctl start has   -> start all the clusterware services/ resources (including the database server and the listener);
/g01/grid/asm/bin/crsctl stop has    -> stop all the clusterware services/ resources (including the database server and the listener);
/g01/grid/asm/bin/crsctl check has   -> to check if ohasd is running/ stopped
/g01/grid/asm/bin/crsctl config has

# /g01/grid/asm/bin/crsctl start has
CRS-4124: Oracle High Availability Services startup failed.
CRS-4000: Command Start failed, or completed with errors.
#  /etc/init.ohasd run

P720:/g01/grid&find . -name aler*.log -print
./asm/log/diag/asmcmd/user_root/P720/alert/alert.log
./asm/log/p720/alertp720.log
./diag/asm/+asm/+ASM/trace/alert_+ASM.log
P720:/g01/grid&

# ll /etc/*init*
ksh: ll:  not found.
# ls -l /etc/init*
lrwxrwxrwx    1 root     system           14 Dec 27 14:52 /etc/init -> /usr/sbin/init
-rwxr-x---    1 root     system         8740 Dec 27 19:19 /etc/init.ohasd
-rw-r--r--    1 root     system         3727 Jan 04 11:06 /etc/inittab
-rw-r--r--    1 root     system         3727 Dec 27 19:19 /etc/inittab.crs
-rw-r--r--    1 root     system         3666 Dec 27 19:19 /etc/inittab.no_crs
P720:/home/grid&crsctl start cluster -all
CRS-4013: This command is not supported in a single-node configuration.
CRS-4000: Command Start failed, or completed with errors.
P720:/home/grid&
P720:/home/grid&crsctl check css
CRS-4639: Could not contact Oracle High Availability Services
CRS-4000: Command Check failed, or completed with errors.
P720:/home/grid&

#  lslpp -l | grep ssh
  openssh.base.client     5.2.0.5300  COMMITTED  Open Secure Shell Commands
  openssh.base.server     5.2.0.5300  COMMITTED  Open Secure Shell Server
  openssh.license         5.2.0.5300  COMMITTED  Open Secure Shell License
  openssh.man.en_US       5.2.0.5300  COMMITTED  Open Secure Shell
  openssh.msg.en_US       5.2.0.5300  COMMITTED  Open Secure Shell Messages -
  openssh.base.client     5.2.0.5300  COMMITTED  Open Secure Shell Commands
  openssh.base.server     5.2.0.5300  COMMITTED  Open Secure Shell Server
# startsrc -g ssh
0513-059 The sshd Subsystem has been started. Subsystem PID is 8978464.
#  lssrc -s sshd
Subsystem         Group            PID          Status
 sshd             ssh              8978464      active
#   

If OLR backup does not exist for any reason, perform deconfig and rerun root.sh is required to recreate OLR, as root user:
# $GRID_HOME/crs/install/rootcrs.pl -deconfig -force
# /g01/grid/asm/crs/install/rootcrs.pl -verbose -deconfig -force

sh -x /g01/grid/asm/bin/crsctl check has

# /g01/grid/asm/crs/install/roothas.pl -deconfig -force -verbose
# /g01/grid/asm/root.sh

./runcluvfy.sh stage -pre crsinst -n p720

P720:/g01/grid/asm/log/p720/ohasd&ohasd start
2013-01-04 16:00:50 
Changing directory to /g01/grid/asm/log/p720/ohasd

 

没有办法，只有重建CSS:
# lssrc -s sshd
Subsystem         Group            PID          Status
 sshd             ssh              8454222      active
# /g01/grid/asm/crs/install/rootcrs.pl -verbose -deconfig -force
Using configuration parameter file: /g01/grid/asm/crs/install/crsconfig_params
Usage: srvctl <command> <object> [<options>]
    commands: enable|disable|start|stop|status|add|remove|modify|getenv|setenv|unsetenv|config|upgrade
    objects: database|service|asm|diskgroup|listener|home|ons
For detailed help on each command and object and its options use:
  srvctl <command> -h or
  srvctl <command> <object> -h
PRKO-2012 : nodeapps object is not supported in Oracle Restart
CRS-2613: Could not find resource 'ora.registry.acfs'.
CRS-4000: Command Stop failed, or completed with errors.
CRS-2613: Could not find resource 'ora.drivers.acfs'.
CRS-4000: Command Modify failed, or completed with errors.
CRS-2613: Could not find resource 'ora.drivers.acfs'.
CRS-4000: Command Delete failed, or completed with errors.
CRS-4013: This command is not supported in a single-node configuration.
CRS-4000: Command Stop failed, or completed with errors.
################################################################
# You must kill processes or reboot the system to properly #
# cleanup the processes started by Oracle clusterware          #
################################################################
ACFS-9313: No ADVM/ACFS installation detected.
Successfully deconfigured Oracle clusterware stack on this node
# /g01/grid/asm/crs/install/roothas.pl -deconfig -force -verbose
Using configuration parameter file: /g01/grid/asm/crs/install/crsconfig_params
CRS-4047: No Oracle Clusterware components configured.
CRS-4000: Command Stop failed, or completed with errors.
CRS-4047: No Oracle Clusterware components configured.
CRS-4000: Command Delete failed, or completed with errors.
CRS-4047: No Oracle Clusterware components configured.
CRS-4000: Command Stop failed, or completed with errors.
You must kill ohasd processes or reboot the system to properly
cleanup the processes started by Oracle clusterware
ACFS-9313: No ADVM/ACFS installation detected.
Either /etc/oracle/olr.loc does not exist or is not readable
Make sure the file exists and it has read and execute access
Failure in execution (rc=-1, 512, A file or directory in the path name does not exist.) for command /etc/ohasd deinstall
Successfully deconfigured Oracle Restart stack
# /g01/grid/asm/root.sh
Performing root user operation for Oracle 11g

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /g01/grid/asm

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /g01/grid/asm/crs/install/crsconfig_params
LOCAL ADD MODE
Creating OCR keys for user 'grid', privgrp 'oinstall'..
Operation successful.
LOCAL ONLY MODE
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'system'..
Operation successful.
CRS-4664: Node p720 successfully pinned.
Adding Clusterware entries to inittab

p720     2013/01/04 23:11:39     /g01/grid/asm/cdata/p720/backup_20130104_231139.olr
Successfully configured Oracle Grid Infrastructure for a Standalone Server
#

#!/bin/sh
startsrc -g ssh

P720:/home/grid&crs_stat -t
Name           Type           Target    State     Host       
------------------------------------------------------------
ora.cssd       ora.cssd.type  OFFLINE   OFFLINE              
ora.diskmon    ora....on.type OFFLINE   OFFLINE              
ora.evmd       ora.evm.type   ONLINE    ONLINE    p720       
ora.ons        ora.ons.type   OFFLINE   OFFLINE         

P720:/home/grid&crsctl start resource ora.cssd
CRS-2672: Attempting to start 'ora.cssd' on 'p720'
CRS-2672: Attempting to start 'ora.diskmon' on 'p720'
CRS-2676: Start of 'ora.diskmon' on 'p720' succeeded
CRS-2676: Start of 'ora.cssd' on 'p720' succeeded
P720:/home/grid&crs_stat -t
Name           Type           Target    State     Host       
------------------------------------------------------------
ora.cssd       ora.cssd.type  ONLINE    ONLINE    p720       
ora.diskmon    ora....on.type OFFLINE   OFFLINE              
ora.evmd       ora.evm.type   ONLINE    ONLINE    p720       
ora.ons        ora.ons.type   OFFLINE   OFFLINE              
P720:/home/grid&

css已经启动上来了，下面注册实例和监听：

crsctl start resource ora.diskmon 
srvctl status database -d sgerp5
srvctl modify database -d sgerp5 -a "DATA"
alter diskgroup data dismount;
drop diskgroup data including contents;
crs  asm comp  start catalog with      set newname for datafile  'xxxxxx/xxx/xxxx' to '+xxx/xxxxxx'
srvctl add database -d sgerp5 -o /u01/app/product/11.2
srvctl config database -d sgerp5
srvctl add database
crsctl stat res -t

用srvctl增加asm,database,instance：
srvctl add asm -n rac1 -i ASM1 -o /app/oracle/product/10.2.0/db_1
srvctl add asm -n rac2 -i ASM2 -o /app/oracle/product/10.2.0/db_1
oracle:add.asm.权限
./srvctl add asm -n rac1 -i +ASM1 -o $ORACLE_HOME
./srvctl add asm -n rac2 -i +ASM2 -o $ORACLE_HOME
oracle:
srvctl add database -d racdb -o $ORACLE_HOME  -p /opt/ora10g/admin/+ASM/pfile/init.ora
srvctl add instance -d racdb -i racdb1 -n rac1

srvctl add database -d racdb -o /app/oracle/product/10.2.0/db_1

srvctl add instance -d racdb -i racdb1 -n rac1
srvctl add instance -d racdb -i racdb2 -n rac2

P720:/home/oracle$srvctl -help
Usage: srvctl <command> <object> [<options>]
    commands: enable|disable|start|stop|status|add|remove|modify|getenv|setenv|unsetenv|config|upgrade
    objects: database|service|asm|diskgroup|listener|home|ons
For detailed help on each command and object and its options use:
  srvctl <command> -h or
  srvctl <command> <object> -h
 
P720:/home/oracle$srvctl add -h

The SRVCTL add command adds the configuration and the Oracle Restart application to the OCR for the cluster database, named instances, named services, or for the named nodes.

Usage: srvctl add database -d <db_unique_name> -o <oracle_home> [-m <domain_name>] [-p <spfile>] [-r {PRIMARY | PHYSICAL_STANDBY | LOGICAL_STANDBY | SNAPSHOT_STANDBY}] [-s <start_options>] [-t <stop_options>] [-n <db_name>] [-i <inst_name>] [-y {AUTOMATIC | MANUAL | NORESTART}] [-a "<diskgroup_list>"]
Usage: srvctl add service -d <db_unique_name> -s <service_name> [-l [PRIMARY][,PHYSICAL_STANDBY][,LOGICAL_STANDBY][,SNAPSHOT_STANDBY]] [-y {AUTOMATIC | MANUAL}][-q {true|false}] [-j {SHORT|LONG}] [-B {NONE|SERVICE_TIME|THROUGHPUT}][-e {NONE|SESSION|SELECT}] [-m {NONE|BASIC}][-z <failover_retries>] [-w <failover_delay>] [-t <edition>] [-f]
Usage: srvctl add asm [-l <lsnr_name>] [-p <spfile>] [-d <asm_diskstring>]

Usage: srvctl add listener [-l <lsnr_name>] [-s] [-p "[TCP:]<port>[, ...][/IPC:<key>][/NMP:<pipe_name>][/TCPS:<s_port>] [/SDP:<port>]"] [-o <oracle_home>]
Usage: srvctl add ons [-e <em-port>] [-l <ons-local-port>]  [-r <ons-remote-port>] [-t <host>[:<port>][,<host>[:<port>]...]] [-v]
For detailed help on each command and object and its options use:
  srvctl <command> <object> -h
P720:/home/oracle$srvctl add listener -h

grid用户操作:先注册监听，后注册ASM实例:
srvctl add listener -l LISTENER  -p 1521 -o $ORACLE_HOME
P720:/home/oracle$srvctl add asm -l ora.asm -p "+DATA/asm/asmparameterfile/registry.253.803157833" -d "/dev/rdisk*"
PRCR-1031 : Resource attribute NAME value ora.ORA.ASM.lsnr is not valid
P720:/home/oracle$
启动：
srvctl start listener -l listener
srvctl add asm -l LISTENER -p "+DATA/asm/asmparameterfile/registry.253.803157833" -d "/dev/rdisk*"

P720:/home/grid&crs_stat -t
Name           Type           Target    State     Host       
------------------------------------------------------------
ora.DATA.dg    ora....up.type ONLINE    ONLINE    p720       
ora....ER.lsnr ora....er.type ONLINE    ONLINE    p720       
ora.asm        ora.asm.type   ONLINE    ONLINE    p720
ora.cssd       ora.cssd.type  ONLINE    ONLINE    p720       
ora.diskmon    ora....on.type OFFLINE   OFFLINE              
ora.evmd       ora.evm.type   ONLINE    ONLINE    p720       
ora.ons        ora.ons.type   OFFLINE   OFFLINE