ORA-00600: internal error code, arguments: [kcratr1_lostwrt], [], [], [], []错误!

出现该错误是因为系统强制关机造成的!
症状为数据库无法打开!
解决办法:
1>shutdown
2>startup mount
3>recover database
4>alter database open
案例1：

数据库掉电，重启发现如下错误：

SQL> select open_mode from v$database;

OPEN_MODE
----------
MOUNTED

奇怪，于是先把数据库关闭，然后重新启动，报错如下：
SQL> startup
ORACLE instance started.

Total System Global Area 1375731712 bytes
Fixed Size 1260780 bytes
Variable Size 603980564 bytes
Database Buffers 754974720 bytes
Redo Buffers 15515648 bytes
Database mounted.
ORA-00600: internal error code, arguments: [kcratr1_lastbwr], [], [], [], [], [], [], []

www.ixdba.net

查看ALERT日志，发现
Errors in file /opt/oracle/admin/billdb/udump/billdb_ora_7186.trc:
ORA-00600: internal error code, arguments: [kcratr1_lastbwr], [], [], [], [], [], [], []
ORA-600 signalled during: ALTER DATABASE OPEN...

打开上面提到的trace文件
*** SERVICE NAME:() 2007-10-11 10:08:49.493
*** SESSION ID:(989.3) 2007-10-11 10:08:49.493
Successfully allocated 2 recovery slaves
Using 543 overflow buffers per recovery slave
Thread 1 checkpoint: logseq 67179, block 2, scn 2132065607
cache-low rba: logseq 67179, block 5453
on-disk rba: logseq 67179, block 5828, scn 2132066974
Starting CRASH recovery for thread 1 sequence 67180 block 1
Thread 1 current log 5
Scanning log 5 thread 1 sequence 67179
Scanning log 6 thread 1 sequence 67178
Cannot find online redo log for thread 1 sequence 67180
start recovery at logseq 67179, block 5453, scn 0
----- Redo read statistics for thread 1 -----
Read rate (ASYNC): 252Kb in 0.03s => 8.22 Mb/sec
Total physical reads: 252Kb
Longest record: 9Kb, moves: 0/341 (0%)
Change moves: 1/24 (4%), moved: 0Mb
Longest LWN: 35Kb, moves: 0/110 (0%), moved: 0Mb
Last redo scn: 0x0000.7f14c2af (2132066991)
----------------------------------------------
******** WRITE VERIFICATION FAILED ********
File 15 Block 89 (rdba 0x3c00059)
BWR version: 0x0000.7f14c25e.01 flg: 0x04
Disk version: 0x0000.7f14c12d.01 flag: 0x04
*** 2007-10-11 10:08:49.541
ksedmp: internal or fatal error
ORA-00600: internal error code, arguments: [kcratr1_lastbwr], [], [], [], [], [], [], []
Current SQL statement for this session:
ALTER DATABASE OPEN

上面提示说找不到67180的日志了，去日志目录看看明明在的，于是跑到metalink上去，查到如下结果：

Changes
There was a disk problem that caused the database to crash.
Cause
Oracle is unable to perform instance recover but it works when is invoked manually.
Solution
Mount the database and issue a recover statement

接着执行恢复：

SQL> startup mount;

SQL> recover database;

SQL> alter database open

于是照搬上面的步骤，手工恢复后问题解决

以下是官方解释：

Unbreakable Oracle 10g Release 2 : What if you have ORA-600 kcratr1_lastbwr ?
This an interesting story that happened yesterday on one of our customer site. An engineer powered off the wrong rack of equipment containing a Sun Fire X4600 running Oracle 10g Release 2.  Almost no transactions were performed at time so when the system came up the customer expected the database to be up and running very quickly.

In reality this is what happened :

Completed: ALTER DATABASE   MOUNT
Tue Nov  7 11:19:42 2006
ALTER DATABASE OPEN
Tue Nov  7 11:19:42 2006
Beginning crash recovery of 1 threads
parallel recovery started with 16 processes
Tue Nov  7 11:19:44 2006
Started redo scan
Tue Nov  7 11:19:44 2006
Errors in file /xxx/oracle/oracle/product/10.2.0/db_1/admin/xxx/udump/xxx_ora_947.trc:
ORA-00600: internal error code, arguments: [kcratr1_lastbwr], [], [], [], [], [], [], []
Tue Nov  7 11:19:44 2006
Aborting crash recovery due to error 600
Tue Nov  7 11:19:44 2006
Errors in file /xxx/oracle/oracle/product/10.2.0/db_1/admin/xxxtest/udump/xxxtest_ora_947.trc:
ORA-00600: internal error code, arguments: [kcratr1_lastbwr], [], [], [], [], [], [], []
ORA-600 signalled during: ALTER DATABASE OPEN...


Not too pretty ! Checking the ASM configuration and the IO subsystem showed nothing wrong. So what to do if you do not have a backup handy ?

Well, here is the idea .... what would we do if we had a backup that was inconsistent ?
The recover database command will start an Oracle process which will roll forward all transactions stored in the restored archived logs necessary to make the database consistent again. The recovery process must run up to a point that corresponds with the time just before the error occurred after which the log sequence must be reset to prevent any further system changes from being applied to the database.
So we tried :

startup mount

ALTER DATABASE   MOUNT
Tue Nov  7 11:54:03 2006
Starting background process ASMB
ASMB started with pid=61, OS id=1070
Starting background process RBAL
RBAL started with pid=67, OS id=1074
Tue Nov  7 11:54:13 2006
SUCCESS: diskgroup xxxTESTDATA was mounted
Tue Nov  7 11:54:17 2006
Setting recovery target incarnation to 2
Tue Nov  7 11:54:17 2006
Successful mount of redo thread 1, with mount id 2364224219
Tue Nov  7 11:54:17 2006
Database mounted in Exclusive Mode
Completed: ALTER DATABASE   MOUNT
Tue Nov  7 11:54:32 2006


recover database

ALTER DATABASE RECOVER  database 
Tue Nov  7 11:54:32 2006
Media Recovery Start
parallel recovery started with 16 processes
Tue Nov  7 11:54:33 2006
Recovery of Online Redo Log: Thread 1 Group 3 Seq 4 Reading mem 0
  Mem# 0 errs 0: +xxxTESTDATA/xxxtest/onlinelog/group_3.263.605819131
Tue Nov  7 11:59:25 2006
Media Recovery Complete (xxxtest)
Tue Nov  7 11:59:27 2006
Completed: ALTER DATABASE RECOVER  database 


alter database open

Tue Nov  7 12:03:01 2006
alter database open
Tue Nov  7 12:03:01 2006
Beginning crash recovery of 1 threads
parallel recovery started with 16 processes
Tue Nov  7 12:03:01 2006
Started redo scan
Tue Nov  7 12:03:01 2006
Completed redo scan
273 redo blocks read, 0 data blocks need recovery
Tue Nov  7 12:03:01 2006
Started redo application at
Thread 1: logseq 4, block 12858574
Tue Nov  7 12:03:01 2006
Recovery of Online Redo Log: Thread 1 Group 3 Seq 4 Reading mem 0
  Mem# 0 errs 0: +xxxTESTDATA/xxxtest/onlinelog/group_3.263.605819131
Tue Nov  7 12:03:01 2006
Completed redo application
Tue Nov  7 12:03:01 2006
Completed crash recovery at
Thread 1: logseq 4, block 12858847, scn 824040
0 data blocks read, 0 data blocks written, 273 redo blocks read
Tue Nov  7 12:03:02 2006
Thread 1 advanced to log sequence 5
Thread 1 opened at log sequence 5
  Current log# 1 seq# 5 mem# 0: +xxxTESTDATA/xxxtest/onlinelog/group_1.261.605819081
Successful open of redo thread 1
Tue Nov  7 12:03:02 2006
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Tue Nov  7 12:03:02 2006
SMON: enabling cache recovery
Tue Nov  7 12:03:03 2006
Successfully onlined Undo Tablespace 1.
Tue Nov  7 12:03:03 2006
SMON: enabling tx recovery
Tue Nov  7 12:03:03 2006
Database Characterset is UTF8
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
QMNC started with pid=56, OS id=1128
Tue Nov  7 12:03:05 2006
Completed: alter database open

And we are up and running ! The real thing that Oracle should work on is the quality and clarity of their error messages.
At this point this is quite poor ...

Unbreakable database, maybe. Automatic (and simple) , not yet.

案例二：

如果在执行recover时刻出现

SQL> recover database;

ORA-00283: recovery session canceled due to errors

ORA-12801: error signaled in parallel query server P002

ORA-10562: Error occurred while applying redo to data block (file# 1, block#

4568)

ORA-10564: tablespace SYSTEM

ORA-01110: data file 1: '/opt/oracle/oradata/orcl/system01.dbf'

ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 576

ORA-00600: internal error code, arguments: [6101], [

此时，检查日志信息如下：

Mon Nov 19 15:38:50 2007

ALTER DATABASE RECOVER  database

Mon Nov 19 15:38:50 2007

Media Recovery Start

parallel recovery started with 3 processes

Mon Nov 19 15:38:50 2007

Recovery of Online Redo Log: Thread 1 Group 3 Seq 16 Reading mem 0

  Mem# 0 errs 0: /opt/oracle/oradata/orcl/redo03.log

Mon Nov 19 15:38:50 2007

Errors in file /opt/oracle/admin/orcl/bdump/orcl_p002_7917.trc:

ORA-00600: internal error code, arguments: [6101], [0], [17], [0], [], [], [], []

Mon Nov 19 15:38:50 2007

Errors in file /opt/oracle/admin/orcl/bdump/orcl_p000_7913.trc:

ORA-00600: internal error code, arguments: [3020], [2], [882], [8389490], [], [], [], []

ORA-10567: Redo is inconsistent with data block (file# 2, block# 882)

ORA-10564: tablespace UNDOTBS1

ORA-01110: data file 2: '/opt/oracle/oradata/orcl/undotbs01.dbf'

ORA-10560: block type 'KTU UNDO BLOCK'

Mon Nov 19 15:38:51 2007

Errors in file /opt/oracle/admin/orcl/bdump/orcl_p000_7913.trc:

ORA-00600: internal error code, arguments: [3020], [2], [882], [8389490], [], [], [], []

ORA-10567: Redo is inconsistent with data block (file# 2, block# 882)

ORA-10564: tablespace UNDOTBS1

ORA-01110: data file 2: '/opt/oracle/oradata/orcl/undotbs01.dbf'

ORA-10560: block type 'KTU UNDO BLOCK'

Mon Nov 19 15:38:51 2007

Errors in file /opt/oracle/admin/orcl/bdump/orcl_p002_7917.trc:

ORA-10562: Error occurred while applying redo to data block (file# 1, block# 4568)

ORA-10564: tablespace SYSTEM

ORA-01110: data file 1: '/opt/oracle/oradata/orcl/system01.dbf'

ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 576

ORA-00600: internal error code, arguments: [6101], [0], [17], [0], [], [], [], []

Mon Nov 19 15:38:54 2007

Errors in file /opt/oracle/admin/orcl/bdump/orcl_p001_7915.trc:

ORA-00600: internal error code, arguments: [kddummy_blkchk], [1], [1658], [6101], [], [], [], []

Mon Nov 19 15:38:54 2007

Errors in file /opt/oracle/admin/orcl/bdump/orcl_p001_7915.trc:

ORA-10562: Error occurred while applying redo to data block (file# 1, block# 1658)

ORA-10564: tablespace SYSTEM

ORA-01110: data file 1: '/opt/oracle/oradata/orcl/system01.dbf'

ORA-10561: block type 'TRANSACTION MANAGED DATA BLOCK', data object# 237

ORA-00607: Internal error occurred while making a change to a data block

ORA-00600: internal error code, arguments: [kddummy_blkchk], [1], [1658], [6101], [], [], [], []

Mon Nov 19 15:38:54 2007

Media Recovery failed with error 12801

ORA-283 signalled during: ALTER DATABASE RECOVER  database  ...

从上面信息中抓取了一个信息：

ORA-10562: Error occurred while applying redo to data block (file# 1, block# 1658)

针对这个错误解决如下：

ORA-10562: Error occurred while applying redo to data block (file# string, block# string)
Cause: See other errors on error stack.
Action: Investigate why the error occurred and how important is the data block. Media and standby database recovery usually can continue if user allows recovery to corrupt this data block。

从日志信息可以基本判断问题如下：

当前在线日志损坏，导致undo回滚段出现问题，又由于系统突然掉电，系统表空间在重启实例后要进行实例恢复，当前在线日志损坏，系统表空间无法进行recover，因而出现了上面的错误。

找到了问题，解决访问就有了。

由于没有备份，数据库也运行在非归档模式下，所以恢复如下步骤：

SQL>startup mount

SQL>recover database using backup controlfile until cancel;

SQL>alter database open resetlogs;

SQL> startup mount

SQL> alter system set “_allow_resetlogs_corruption”=true scope=spfile;

SQL>shutdown immediate

SQL> startup mount

SQL> alter database open resetlogs;

SQL> startup

我的基本恢复步骤：

SQL>startup mount

SQL>recover database using backup controlfile until cancel;

Cancel

SQL>alter database open resetlogs;

此时会提示system表空间需要恢复，但是由于当前日志损坏，无法进行恢复，所以需要加入隐含参数，oracle才不会监测scn的一致性，才能打开数据库。

重启数据库加入隐含参数

SQL> startup mount

SQL> alter system set “_allow_resetlogs_corruption”=true scope=spfile;

SQL> shutdown immediate

SQL> startup mount

SQL> alter database open resetlogs;

alter database open resetlogs

*

ERROR at line 1:

ORA-01092: ORACLE instance terminated. Disconnection forced

不管这些，再次登录sqlplus起动数据库

SQL*Plus: Release 10.2.0.1.0 - Production on Fri Nov 16 08:03:43 2007

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

Connected to:

Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production

With the Partitioning, OLAP and Data Mining options

SQL>startup

数据库正常打开，exp备份需要的数据，然后重新建库，导入数据。