ORA-01033: ORACLE initialization or shutdown in progress---惜分飞

客户反馈数据库使用plsql dev登录报ORA-01033: ORACLE initialization or shutdown in progress的错误
 

---

出现该错误一般是由于数据库没有正常open成功,查看oracle 告警日志发现

Mon Jan 22 16:55:50 2024 Database mounted in Exclusive Mode Lost write protection disabled Completed: alter database mount exclusive alter database open Beginning crash recovery of 1 threads  parallel recovery started with 15 processes Started redo scan Completed redo scan  read 139 KB redo, 70 data blocks need recovery Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_7792.trc  (incident=20565): ORA-00600: ??????, ??: [kcratr_nab_less_than_odr], [1], [1916], [28210], [28222], [], [], [], [], [], [], [] Incident details in: d:\app\administrator\diag\rdbms\orcl\orcl\incident\incdir_20565\orcl_ora_7792_i20565.trc Mon Jan 22 16:55:57 2024 Trace dumping is performing id=[cdmp_20240122165557] Aborting crash recovery due to error 600 Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_7792.trc: ORA-00600: ??????, ??: [kcratr_nab_less_than_odr], [1], [1916], [28210], [28222], [], [], [], [], [], [], [] Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_7792.trc: ORA-00600: ??????, ??: [kcratr_nab_less_than_odr], [1], [1916], [28210], [28222], [], [], [], [], [], [], [] ORA-600 signalled during: alter database open...

这个错误比较常见,解决起来比较简单,参考:
kcratr_nab_less_than_odr
12c启动报kcratr_nab_less_than_odr
又一例ORA-600 kcratr_nab_less_than_odr
在恢复过程中中还遇到了ORA-00700 kcrf_split_brain_error错误,但是没有影响数据库open

Mon Jan 22 20:13:55 2024 alter database open Beginning crash recovery of 1 threads Started redo application at  Thread 1: logseq 1916, block 27931 Recovery of Online Redo Log: Thread 1 Group 2 Seq 1916 Reading mem 0   Mem# 0: D:\TEMP\ORCL\REDO02.LOG Completed redo application of 0.00MB Completed crash recovery at  Thread 1: logseq 1916, block 28210, scn 43957072  0 data blocks read, 0 data blocks written, 139 redo k-bytes read Errors in file d:\app\xifenfei\diag\rdbms\orcl\orcl\trace\orcl_ora_6104.trc  (incident=15729): ORA-00700: 软内部错误, 参数: [kcrf_split_brain_error], [1], [1916], [28222], [28209], [4], [], [], [], [], [], [] Incident details in: d:\app\xifenfei\diag\rdbms\orcl\orcl\incident\incdir_15729\orcl_ora_6104_i15729.trc Mon Jan 22 20:13:56 2024 Trace dumping is performing id=[cdmp_20240122201356] Mon Jan 22 20:13:56 2024 Thread 1 advanced to log sequence 1917 (thread open) Thread 1 opened at log sequence 1917   Current log# 3 seq# 1917 mem# 0: D:\TEMP\ORCL\REDO03.LOG Successful open of redo thread 1 Mon Jan 22 20:13:56 2024 SMON: enabling cache recovery Verifying file header compatibility for 11g tablespace encryption.. Verifying 11g file header compatibility for tablespace encryption completed SMON: enabling tx recovery Database Characterset is AL32UTF8 replication_dependency_tracking turned off (no async multimaster replication found) WARNING: AQ_TM_PROCESSES is set to 0. System operation                     might be adversely affected. Completed: alter database open

---

至此数据库open成功但是dbv检测system有很多坏块需要分析处理

C:\Users\XIFENFEI>dbv file=d:/temp/orcl/system01.dbf DBVERIFY: Release 11.2.0.1.0 - Production on 星期一 1月 22 21:07:18 2024 Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved. DBVERIFY - 开始验证: FILE = D:\TEMP\ORCL\SYSTEM01.DBF 页 106278 流入 - 很可能是介质损坏 Corrupt block relative dba: 0x00419f26 (file 1, block 106278) Fractured block found during dbv: Data in bad block:  type: 6 format: 2 rdba: 0x00419f26  last change scn: 0x0000.01410f78 seq: 0x2 flg: 0x04  spare1: 0x0 spare2: 0x0 spare3: 0x0  consistency value in tail: 0x00000000  check value in block header: 0xbf11  computed block checksum: 0xaf18 页 106279 标记为损坏 Corrupt block relative dba: 0x00419f27 (file 1, block 106279) Bad header found during dbv: Data in bad block:  type: 178 format: 3 rdba: 0xc8c9c1b6  last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00  spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf  consistency value in tail: 0x00000000  check value in block header: 0xbbb2  block checksum disabled 页 106280 标记为损坏 Corrupt block relative dba: 0x00419f28 (file 1, block 106280) Bad header found during dbv: Data in bad block:  type: 178 format: 3 rdba: 0xc8c9c1b6  last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00  spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf  consistency value in tail: 0x00000000  check value in block header: 0xbbb2  block checksum disabled 页 106281 标记为损坏 Corrupt block relative dba: 0x00419f29 (file 1, block 106281) Bad header found during dbv: Data in bad block:  type: 178 format: 3 rdba: 0xc8c9c1b6  last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00  spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf  consistency value in tail: 0x00000000  check value in block header: 0xbbb2  block checksum disabled 页 106282 标记为损坏 Corrupt block relative dba: 0x00419f2a (file 1, block 106282) Bad header found during dbv: Data in bad block:  type: 178 format: 3 rdba: 0xc8c9c1b6  last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00  spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf  consistency value in tail: 0x00000000  check value in block header: 0xbbb2  block checksum disabled 页 106283 标记为损坏 Corrupt block relative dba: 0x00419f2b (file 1, block 106283) Bad header found during dbv: Data in bad block:  type: 178 format: 3 rdba: 0xc8c9c1b6  last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00  spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf  consistency value in tail: 0x00000000  check value in block header: 0xbbb2  block checksum disabled 页 106284 标记为损坏 Corrupt block relative dba: 0x00419f2c (file 1, block 106284) Bad header found during dbv: Data in bad block:  type: 178 format: 3 rdba: 0xc8c9c1b6  last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00  spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf  consistency value in tail: 0x00000000  check value in block header: 0xbbb2  block checksum disabled 页 106285 标记为损坏 Corrupt block relative dba: 0x00419f2d (file 1, block 106285) Bad header found during dbv: Data in bad block:  type: 178 format: 3 rdba: 0xc8c9c1b6  last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00  spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf  consistency value in tail: 0x00000000  check value in block header: 0xbbb2  block checksum disabled 页 106286 标记为损坏 Corrupt block relative dba: 0x00419f2e (file 1, block 106286) Bad header found during dbv: Data in bad block:  type: 178 format: 3 rdba: 0xc8c9c1b6  last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00  spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf  consistency value in tail: 0x2c310602  check value in block header: 0xbbb2  block checksum disabled 页 143094 流入 - 很可能是介质损坏 Corrupt block relative dba: 0x00422ef6 (file 1, block 143094) Fractured block found during dbv: Data in bad block:  type: 6 format: 2 rdba: 0x00422ef6  last change scn: 0x0000.028f863b seq: 0x2 flg: 0x04  spare1: 0x0 spare2: 0x0 spare3: 0x0  consistency value in tail: 0x00000000  check value in block header: 0xda23  computed block checksum: 0x4210 页 143095 标记为损坏 Corrupt block relative dba: 0x00422ef7 (file 1, block 143095) Bad header found during dbv: Data in bad block:  type: 178 format: 3 rdba: 0xc8c9c1b6  last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00  spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf  consistency value in tail: 0x00000000  check value in block header: 0xbbb2  block checksum disabled 页 143096 标记为损坏 Corrupt block relative dba: 0x00422ef8 (file 1, block 143096) Bad header found during dbv: Data in bad block:  type: 178 format: 3 rdba: 0xc8c9c1b6  last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00  spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf  consistency value in tail: 0x00000000  check value in block header: 0xbbb2  block checksum disabled 页 143097 标记为损坏 Corrupt block relative dba: 0x00422ef9 (file 1, block 143097) Bad header found during dbv: Data in bad block:  type: 178 format: 3 rdba: 0xc8c9c1b6  last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00  spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf  consistency value in tail: 0x00000000  check value in block header: 0xbbb2  block checksum disabled 页 143098 标记为损坏 Corrupt block relative dba: 0x00422efa (file 1, block 143098) Bad header found during dbv: Data in bad block:  type: 178 format: 3 rdba: 0xc8c9c1b6  last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00  spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf  consistency value in tail: 0x00000000  check value in block header: 0xbbb2  block checksum disabled 页 143099 标记为损坏 Corrupt block relative dba: 0x00422efb (file 1, block 143099) Bad header found during dbv: Data in bad block:  type: 178 format: 3 rdba: 0xc8c9c1b6  last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00  spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf  consistency value in tail: 0x00000000  check value in block header: 0xbbb2  block checksum disabled 页 143100 标记为损坏 Corrupt block relative dba: 0x00422efc (file 1, block 143100) Bad header found during dbv: Data in bad block:  type: 178 format: 3 rdba: 0xc8c9c1b6  last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00  spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf  consistency value in tail: 0x00000000  check value in block header: 0xbbb2  block checksum disabled 页 143101 标记为损坏 Corrupt block relative dba: 0x00422efd (file 1, block 143101) Bad header found during dbv: Data in bad block:  type: 178 format: 3 rdba: 0xc8c9c1b6  last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00  spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf  consistency value in tail: 0x00000000  check value in block header: 0xbbb2  block checksum disabled 页 143102 标记为损坏 Corrupt block relative dba: 0x00422efe (file 1, block 143102) Bad header found during dbv: Data in bad block:  type: 178 format: 3 rdba: 0xc8c9c1b6  last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00  spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf  consistency value in tail: 0x8a780602  check value in block header: 0xbbb2  block checksum disabled DBVERIFY - 验证完成 检查的页总数: 152320 处理的页总数 (数据): 115189 失败的页总数 (数据): 0 处理的页总数 (索引): 13086 失败的页总数 (索引): 0 处理的页总数 (其他): 9741 处理的总页数 (段)  : 1 失败的总页数 (段)  : 0 空的页总数: 14286 标记为损坏的总页数: 18 流入的页总数: 2 加密的总页数        : 0 最高块 SCN            : 44036082 (0.44036082)

通过分析aud$的extent,确认这些坏块全部属于该对象

SQL> select block_id,blocks from dba_extents where segment_name='AUD$';   BLOCK_ID     BLOCKS ---------- ---------- …………     102016       1024     103040       1024     104064       1024     105088       1024     106112       1024 …………     141056       1024     142080       1024     143104       1024 已选择124行。

处理方法比较简单,直接truncate aud$表即可