一次关于Oracle RAC 11g ASM磁盘组剩余空间不足导致数据库无法启动的解决

前段时间在VM ESX上安装了两台Linux Server(版本:RHEL5U4 64Bit )做了Oracle RAC 11g，测试了一段时间RAC 的TAF特性，周一上班后重启两个节点，发现Oracle DB无法启动，两个节点分别报如下错误：
节点1：
[oracle@rac01 ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Mon Nov 28 15:38:20 2011

Copyright (c) 1982, 2011, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options

SQL> select open_mode from v$database;

OPEN_MODE
--------------------
MOUNTED

SQL> alter database open;
alter database open
*
ERROR at line 1:
ORA-16038: log 4 sequence# 56 cannot be archived
ORA-19504: failed to create file ""
ORA-00312: online log 4 thread 2: '+FRA/ora11g/onlinelog/group_4.271.767987875'
ORA-00312: online log 4 thread 2: '+FRA/ora11g/onlinelog/group_4.272.767987877'

预警文件中出现如下错误：
*************************************************************
ARC2: Error 19504 Creating archive log file to '+FRA'
Errors in file /u01/product/diag/rdbms/ora11g/ora11g1/trace/ora11g1_arc1_5550.trc:
ORA-19816: WARNING: Files may exist in db_recovery_file_dest that are not known to database.
ORA-17502: ksfdcre:4 Failed to create file +FRA
ORA-15041: diskgroup "FRA" space exhausted
*************************************************************
WARNING: A file of type ARCHIVED LOG may exist in
db_recovery_file_dest that is not known to the database.
Use the RMAN command CATALOG RECOVERY AREA to re-catalog
any such files. If files cannot be cataloged, then manually
delete them using OS command. This is most likely the
result of a crash during file creation.
*************************************************************
节点2：
[root@rac02 ~]# su - oracle
[oracle@rac02 ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Mon Nov 28 15:21:39 2011

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup;
ORACLE instance started.

Total System Global Area  425897984 bytes
Fixed Size                  2229024 bytes
Variable Size             268438752 bytes
Database Buffers          150994944 bytes
Redo Buffers                4235264 bytes
Database mounted.
ORA-03113: end-of-file on communication channel
Process ID: 5250
Session ID: 34 Serial number: 3

预警文件中出现如下错误：
Errors in file /u01/product/diag/rdbms/ora11g/ora11g2/trace/ora11g2_ora_6964.trc:
ORA-19816: WARNING: Files may exist in db_recovery_file_dest that are not known to database.
ORA-17502: ksfdcre:4 Failed to create file +FRA
ORA-15041: diskgroup "FRA" space exhausted

Oracle DB的归档日志路径设置在ASM磁盘组“FRA"下，究其原因是由于"FRA"剩余空间不足，导致归档无法生成。使用ASM的磁盘管理工具：ASMCMD ，可以直接查看ASM磁盘组的内容并对其做相应的操作。这里我们可以删除磁盘组”FRA"下的归档（或其它不再需要的文件），操作过程如下
1) 以oracle grid infrastructure软件所有者的身份登录到节点1，设置ASM实例名： Normal 0 0 2 false false false MicrosoftInternetExplorer4 export ORACLE_SID=+ASM1
2)运行asmcmd工具：asmcmd
3)查看剩余空间：lsdg
ASMCMD> lsdg 
State    Type    Rebal  Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  EXTERN  N         512   4096  1048576     10236     9840                0            9840              0             Y  CRS/
MOUNTED  EXTERN  N         512   4096  1048576     20473    20378                0           20378              0             N  DATA/
MOUNTED  EXTERN  N         512   4096  1048576     10236        6                0               6              0             N  FRA/
4)进入相应路径删除归档：ASMCMD> cd +FRA/ORA11G/ARCHIVELOG
ASMCMD> ls
2011_11_23/
2011_11_24/
2011_11_25/
2011_11_26/
ASMCMD> rm -rf *
5)退出asmcmd工具后，即可正常开启Oracle DB

来自 “ ITPUB博客 ” ，链接：http://blog.itpub.net/25574072/viewspace-712245/，如需转载，请注明出处，否则将追究法律责任。

转载于:http://blog.itpub.net/25574072/viewspace-712245/