BBED修复DB启动问题

在网上看到一个案例UNDO tablespace被误删除,并且shutdown abort数据库.然后再启动数据库都会报下面的错：

Undo initialization finished serial:0 start:161968634 end:161968684 diff:50 (0 seconds)
Verifying file header compatibility for 11g tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
SMON: enabling tx recovery
Database Characterset is UTF8
Errors in file /u01/app/diag/rdbms/primary/primary/trace/primary_ora_13747.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-00376: file 3 cannot be read at this time
ORA-01110: data file 3: ‘/u01/app/oradata/undotbs1.dbf’
Errors in file /u01/app/diag/rdbms/primary/primary/trace/primary_ora_13747.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-00376: file 3 cannot be read at this time
ORA-01110: data file 3: ‘/u01/app/oradata/undotbs1.dbf’
Error 604 happened during db open, shutting down database
USER (ospid: 13747): terminating the instance due to error 604
Instance terminated by USER, pid = 13747
ORA-1092 signalled during: alter database open…
opiodr aborting process unknown ospid (13747) as a result of ORA-1092
Wed Jul 31 18:06:08 2013
ORA-1092 : opitsk aborting process

下面是尝试解决方法：

1.发现在执行recursive SQL level 1的时候报错,所以我们需要通过10046看看在执行什么sql报错的

alter session set events  ’10046 trace name context forever,level 12′;
alter session set db_file_multiblock_read_count=1;

alter database open;然后会生成一个tracefile

2.tracefile内容

PARSING IN CURSOR #47510118633584 len=102 dep=1 uid=0 oct=3 lid=0 tim=1375265166576545 hv=3967354608 ad=’687e34d8′ sqlid=’axmdf8vq7k1rh’
select increment$$,minvalue,maxvalue,cycle#,order$$,cache,highwater,audit$$,flags from seq$$ where obj#=:1
END OF STMT
PARSE #47510118633584:c=8999,e=8834,p=2,cr=27,cu=0,mis=1,r=0,dep=1,og=4,plh=0,tim=1375265166576543
BINDS #47510118633584:
Bind#0
 oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
 oacflg=00 fl2=0001 frm=00 csi=00 siz=24 off=0
 kxsbbbfp=2b35cfbe7b78  bln=22  avl=03  flg=05
 value=1172
EXEC #47510118633584:c=1000,e=1213,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,plh=2203911306,tim=1375265166577898
WAIT #47510118633584: nam=’db file sequential read’ ela= 134 file#=1 block#=705 blocks=1 obj#=79 tim=1375265166578720
WAIT #47510118633584: nam=’db file sequential read’ ela= 20 file#=1 block#=665 blocks=1 obj#=74 tim=1375265166579003
WAIT #47510118633584: nam=’control file sequential read’ ela= 45 file#=0 block#=1 blocks=1 obj#=74 tim=1375265166579478
WAIT #47510118633584: nam=’control file sequential read’ ela= 134 file#=0 block#=16 blocks=1 obj#=74 tim=1375265166579747
WAIT #47510118633584: nam=’control file sequential read’ ela= 93 file#=0 block#=18 blocks=1 obj#=74 tim=1375265166579921
WAIT #47510118633584: nam=’control file sequential read’ ela= 10 file#=0 block#=23 blocks=1 obj#=74 tim=1375265166580002
WAIT #47510118633584: nam=’control file sequential read’ ela= 8 file#=0 block#=1 blocks=1 obj#=74 tim=1375265166580171
WAIT #47510118633584: nam=’control file sequential read’ ela= 6 file#=0 block#=16 blocks=1 obj#=74 tim=1375265166580215
WAIT #47510118633584: nam=’control file sequential read’ ela= 7 file#=0 block#=18 blocks=1 obj#=74 tim=1375265166580235
WAIT #47510118633584: nam=’control file sequential read’ ela= 9 file#=0 block#=32 blocks=1 obj#=74 tim=1375265166580257
DDE rules only execution for: ORA 1110
―�C START Event Driven Actions Dump ―-
―- END Event Driven Actions Dump ―-
―�C START DDE Actions Dump ―�C
Executing SYNC actions
―�C START DDE Action: ‘DB_STRUCTURE_INTEGRITY_CHECK’ (Async) ―�C
Successfully dispatched
―�C END DDE Action: ‘DB_STRUCTURE_INTEGRITY_CHECK’ (SUCCESS, 1 csec) ―�C
Executing ASYNC actions
―�C END DDE Actions Dump (total 1 csec) ―�C
FETCH #47510118633584:c=2999,e=11520,p=2,cr=2,cu=0,mis=0,r=0,dep=1,og=4,plh=2203911306,tim=1375265166589486
STAT #47510118633584 id=1 cnt=0 pid=0 pos=1 obj=74 op=’TABLE ACCESS BY INDEX ROWID SEQ$$ (cr=0 pr=0 pw=0 time=153 us)’
STAT #47510118633584 id=2 cnt=1 pid=1 pos=1 obj=79 op=’INDEX UNIQUE SCAN I_SEQ1 (cr=1 pr=1 pw=0 time=579 us)’
ORA-00604: error occurred at recursive SQL level 1
ORA-00376: file 3 cannot be read at this time
ORA-01110: data file 3: ‘/u01/app/oradata/undotbs1.dbf’
ORA-00604: error occurred at recursive SQL level 1
ORA-00376: file 3 cannot be read at this time
ORA-01110: data file 3: ‘/u01/app/oradata/undotbs1.dbf’

在上面你可以看到在读到file1,block 705,665的时候报错,应该是里面有未提交的事物,下面就可以通过bbed来简单的修复

3.通过bbed来修复,需先找一个原datafile 1的文件

BBED> info
File#  Name                                                        Size(blks)
―�C  ―-                                                        ―――-
    1  /u01/app/oradata/system.dbf                                      64000
    2  /u01/app/oradatabak/system.dbf                                   64000

BBED> copy dba 2,665 to dba 1,665
File: /u01/app/oradata/system.dbf (1)
Block: 665              Offsets:    0 to  511           Dba:0×00400299
――――――――――――――――――――――――
06a20000 99024000 0c8f0600 00000106 64710000 01000000 4a000000 0a8f0600
00000000 02000200 00000000 02000900 e7000000 9400c000 57000e00 01200000
0b8f0600 04000d00 e2000000 a804c000 3f001b00 01200000 0c8f0600 00016200
ffffd600 e6015103 51030000 6200571f 191fd11e 8f1ee903 041eb81d 6c1d2c04
211dd71c 8d1c431c 021cc11b 7f1b351b ea1aa91a 681a1f1a d6198d19 4319f918
b0186618 1c18d317 91174b17 0b17c516 7b163116 e7159d15 51151015 cf148414
3914ee13 a3136113 1713cd12 82123712 ed11a211 58110e11 c4107a10 3010e60f
9c0f5b0f 1a0fd90e 980e570e 160ee601 870d7302 f30ca90c 5f0cbe02 0903820b
390bef0a a90a680a 270ae609 a1095809 0e09c408 7a083008 e6079c07 52070807
be067406 2a06e005 96054c05 0205b704 6c040000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 0000002c 000a03c2
333102c1 0202c102 0fce6464 64646464 64646464 64646464 01800180 02c20203
c2020220 2d2d2d2d 2d2d2d2d 2d2d2d2d 2d2d2d2d 2d2d2d2d 2d2d2d2d 2d2d2d2d
2d2d2d2d 01802c00 0a03c233 2f02c102 02c1020f ce646464 64646464 64646464
64646401 80018002 c20203c2 0202202d 2d2d2d2d 2d2d2d2d 2d2d2d2d 2d2d2d2d
2d2d2d2d 2d2d2d2d 2d2d2d2d 2d2d2d01 802c000a 03c2332e 02c10202 c1020fce

<32 bytes per line>

BBED> copy dba 2,705 to dba 1,705
File: /u01/app/oradata/system.dbf (1)
Block: 705              Offsets:    0 to  511           Dba:0x004002c1
――――――――――――――――――――――――
06a20000 c1024000 cf6e0500 00000106 12b40000 02000000 4f000000 ce6e0500
00000000 02000200 00000000 00000000 00000000 00000000 00000000 00000000
00000000 07001100 d2000000 6a23c000 38000500 01200000 cf6e0500 00008001
00000000 86003001 fd18cd17 00000000 00000000 00000000 06000000 601f0000
551f491f 3d1f311f 251f191f 0d1f011f f51ee91e dd1ed11e c51eb91e ad1ea11e
951e891e 7d1e711e 651e591e 4d1e411e 351e291e 1d1e111e 051ef91d ed1de11d
d51dc91d bd1db11d a51d991d 8d1d811d 751d691d 5d1d511d 451d391d 2d1d211d
151d091d fd1cf11c e51cd91c cd1cc11c b51ca91c 9d1c911c 851c791c 6d1c611c
551c491c 3d1c311c 251c191c 0d1c011c f51be91b dd1bd11b c51bb91b ad1ba11b
951b891b 7d1b711b 651b591b 4d1b411b 351b291b 1d1b111b 051bf91a ed1ae11a
d51ac91a bd1ab11a a51a991a 8d1a811a 751a691a 5c1a4f1a 421a351a 281a1b1a
0e1a011a f419e719 da19cd19 c019b319 a6199919 8c197f19 72196519 58194b19
3e193119 24191719 0a19fd18 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000

<32 bytes per line>

BBED> sum apply;
Check value for File 1, Block 705:
current = 0xb412, required = 0xb412

BBED> verify
DBVERIFY �C Verification starting
FILE = /u01/app/oradata/system.dbf
BLOCK = 705
DBVERIFY �C Verification complete

Total Blocks Examined         : 1
Total Blocks Processed (Data) : 0
Total Blocks Failing   (Data) : 0
Total Blocks Processed (Index): 1
Total Blocks Failing   (Index): 0
Total Blocks Empty            : 0
Total Blocks Marked Corrupt   : 0
Total Blocks Influx           : 0
Message 531 not found;  product=RDBMS; facility=BBED
BBED> copy dba 2,665 to dba 1,665
File: /u01/app/oradata/system.dbf (1)
Block: 665              Offsets:    0 to  511           Dba:0×00400299
――――――――――――――――――――――――
06a20000 99024000 0c8f0600 00000106 64710000 01000000 4a000000 0a8f0600
00000000 02000200 00000000 02000900 e7000000 9400c000 57000e00 01200000
0b8f0600 04000d00 e2000000 a804c000 3f001b00 01200000 0c8f0600 00016200
ffffd600 e6015103 51030000 6200571f 191fd11e 8f1ee903 041eb81d 6c1d2c04
211dd71c 8d1c431c 021cc11b 7f1b351b ea1aa91a 681a1f1a d6198d19 4319f918
b0186618 1c18d317 91174b17 0b17c516 7b163116 e7159d15 51151015 cf148414
3914ee13 a3136113 1713cd12 82123712 ed11a211 58110e11 c4107a10 3010e60f
9c0f5b0f 1a0fd90e 980e570e 160ee601 870d7302 f30ca90c 5f0cbe02 0903820b
390bef0a a90a680a 270ae609 a1095809 0e09c408 7a083008 e6079c07 52070807
be067406 2a06e005 96054c05 0205b704 6c040000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 0000002c 000a03c2
333102c1 0202c102 0fce6464 64646464 64646464 64646464 01800180 02c20203
c2020220 2d2d2d2d 2d2d2d2d 2d2d2d2d 2d2d2d2d 2d2d2d2d 2d2d2d2d 2d2d2d2d
2d2d2d2d 01802c00 0a03c233 2f02c102 02c1020f ce646464 64646464 64646464
64646401 80018002 c20203c2 0202202d 2d2d2d2d 2d2d2d2d 2d2d2d2d 2d2d2d2d
2d2d2d2d 2d2d2d2d 2d2d2d2d 2d2d2d01 802c000a 03c2332e 02c10202 c1020fce

<32 bytes per line>

BBED> sum apply
Check value for File 1, Block 665:
current = 0×7164, required = 0×7164

BBED> verify
DBVERIFY �C Verification starting
FILE = /u01/app/oradata/system.dbf
BLOCK = 665
DBVERIFY �C Verification complete

Total Blocks Examined         : 1
Total Blocks Processed (Data) : 1
Total Blocks Failing   (Data) : 0
Total Blocks Processed (Index): 0
Total Blocks Failing   (Index): 0
Total Blocks Empty            : 0
Total Blocks Marked Corrupt   : 0
Total Blocks Influx           : 0
Message 531 not found;  product=RDBMS; facility=BBED
BBED> exit

[oracle@dg1 lib]$$ sqlplus ‘/as sysdba’;

SQL*Plus: Release 11.2.0.2.0 Production on Wed Jul 31 18:31:11 2013

Copyright (c) 1982, 2010, Oracle.  All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 �C 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> shutdown immediate;
ORA-01109: database not open
Database dismounted.
ORACLE instance shut down.
SQL>
SQL>
SQL> startup
ORACLE instance started.

Total System Global Area  217157632 bytes
Fixed Size                  2225064 bytes
Variable Size             159386712 bytes
Database Buffers           50331648 bytes
Redo Buffers                5214208 bytes
Database mounted.
Database opened.
SQL>

修改后就可以直接打开,你也可以dump出来去修改itl的信息.