Oracle_ORA-00600: internal error code, arguments: [kdsgrp1]
今日值班，收到数据库预警文件告警(alert)日志中出现ORA-00600: internal error code, arguments: [kdsgrp1]错误，相关错误内容如下：
LNS: Standby redo logfile selected for thread 1 sequence 276475 for destination LOG_ARCHIVE_DEST_2
Fri Dec 25 13:59:10 2015
Archived Log entry 849751 added for thread 1 sequence 276474 ID 0x13caeb69 dest 1:
Fri Dec 25 13:59:22 2015
Errors in file /oracle_log/diag/rdbms/oraxulq1/oxx1a/trace/oxx1a_ora_56034106.trc:
ORA-00308: cannot open archived log '/arch2/xulq1b-826331085-2-0000226892.arc'
ORA-27037: unable to obtain file status
IBM AIX RISC System/6000 Error: 2: No such file or directory
Additional information: 3
Errors in file /oracle_log/diag/rdbms/oraxulq1/oxx1a/trace/oxx1a_ora_56034106.trc (incident=3720455):
ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], [], [], [], [], []
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Fri Dec 25 13:59:22 2015
Sweep [inc][3720455]: completed
Fri Dec 25 14:00:52 2015
Errors in file /oracle_log/diag/rdbms/oraxulq1/oxx1a/trace/oxx1a_ora_56034106.trc:
ORA-00308: cannot open archived log '/arch2/xulq1b-826331085-2-0000226892.arc'
ORA-27037: unable to obtain file status
IBM AIX RISC System/6000 Error: 2: No such file or directory
Additional information: 3
Errors in file /oracle_log/diag/rdbms/oraxulq1/oxx1a/trace/oxx1a_ora_56034106.trc (incident=3720456):
ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], [], [], [], [], []
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Fri Dec 25 14:00:53 2015
Sweep [inc][3720456]: completed
Fri Dec 25 14:01:40 2015
Thread 1 advanced to log sequence 276476 (LGWR switch)
Current log# 2 seq# 276476 mem# 0: /dev/rolv_redo_a21
Current log# 2 seq# 276476 mem# 1: /dev/rolv_redo_a22
Fri Dec 25 14:01:40 2015
LNS: Standby redo logfile selected for thread 1 sequence 276476 for destination LOG_ARCHIVE_DEST_2
Fri Dec 25 14:01:47 2015
Archived Log entry 849755 added for thread 1 sequence 276475 ID 0x13caeb69 dest 1:
查看trace文件
[oracle@xulq] /oracle> more /oracle_log/diag/rdbms/oraxulq1/oxulq1b/trace/oxulq1b_ora_53542940.trc
Trace file /oracle_log/diag/rdbms/oraxulq1/oxulq1b/trace/oxulq1b_ora_53542940.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options
ORACLE_HOME = /oracle/app/oracle/product/11.2.0/dbhome_1
System name: AIX
Node name: c4oxulq1b
Release: 1
Version: 6
Machine: 00F80CD14C00
Instance name: oxulq1b
Redo thread mounted by this instance: 2
Oracle process number: 4934
Unix process pid: 53542940, image: oracle@c4oxulq1b
*** 2015-12-25 14:36:23.581
*** SESSION ID:(17854.56245) 2015-12-25 14:36:23.581
*** CLIENT ID:() 2015-12-25 14:36:23.581
*** SERVICE NAME:(SYS$USERS) 2015-12-25 14:36:23.581
*** MODULE NAME:(/tbcs/tnode2/bosscron/bin/backcreditrc@tin1b (TN) 2015-12-25 14:36:23.581
*** ACTION NAME:() 2015-12-25 14:36:23.581
* kdsgrp1-1: *************************************************
row 0x7280dc54.0 continuation at
0x7280dc54.0 file# 458 block# 56404 slot 0 not found
KDSTABN_GET: 0 ..... ntab: 1
curSlot: 0 ..... nrows: 11
kdsgrp - dump CR block dba=0x7280dc54
Block header dump: 0x7280dc54
Object id on Block? Y
seg/obj: 0xe98fe6 csc: 0xc05.feabb68f itc: 80 flg: E typ: 1 - DATA
brn: 0 bdba: 0x7280dc0a ver: 0x01 opc: 0
inc: 0 exflg: 0
Itl Xid Uba Flag Lck Scn/Fsc
0x01 0x0281.01b.0033cf4f 0x238167aa.bd57.19 C--- 0 scn 0x0c05.d8c34e52
0x02 0x0088.00b.00c12881 0x2389f370.3ae9.50 C--- 0 scn 0x0c05.e2bb4c68
0x03 0x004c.00c.00d109dc 0x238add8e.bb06.08 C--- 0 scn 0x0c05.eb36142a
0x04 0x0500.01a.000f5237 0x940070e3.88cd.17 C--- 0 scn 0x0c05.e2c4257c
0x05 0x00e3.014.0041e63b 0x944aad3e.b2e4.1a C--- 0 scn 0x0c05.eb36a12d
0x06 0x001d.01d.013ca0bb 0x94499a1a.18f8.0b C--- 0 scn 0x0c05.f10324f2
0x07 0x00c1.00e.0091d205 0x4c0298a6.c398.16 C--- 0 scn 0x0c05.f101b17e
0x08 0x0b33.020.0003e076 0x009c364c.2962.05 C--- 0 scn 0x0c05.f1374405
0x09 0x008d.003.0069141b 0x0082d4bc.c69f.2f C--- 0 scn 0x0c05.f138741b
0x0a 0x0036.00c.00c21731 0x9408811f.33ce.03 C--- 0 scn 0x0c05.f1a12590
0x0b 0x0b1a.015.0002c17e 0x00987498.1df7.1a C--- 0 scn 0x0c05.f1a249e2
0x0c 0x00e6.021.003fe335 0x94064cde.e342.1d C--- 0 scn 0x0c05.f1b82533
0x0d 0x0076.01f.006c544b 0x9401bade.8e1c.19 C--- 0 scn 0x0c05.f3132bf2
0x0e 0x04f8.00c.000bd41c 0x93d3af73.78b9.27 C--- 0 scn 0x0c05.f1b8e9d0
0x0f 0x009d.01c.0064bb29 0x0084d2d5.4eba.0c C--- 0 scn 0x0c05.f313886f
0x10 0x009a.00d.007bb467 0x93c6408e.9aa8.2d C--- 0 scn 0x0c05.f38ecf66
0x11 0x0b60.002.0002c43d 0x93c76f0b.2072.35 C--- 0 scn 0x0c05.f38fc155
0x12 0x02a2.00f.0018cd3e 0x238ed5b2.0536.38 C--- 0 scn 0x0c05.f47337eb
0x13 0x000b.018.013e93cb 0x008e979a.0f7a.03 C--- 0 scn 0x0c05.f4764933
0x14 0x008c.012.005fb8c6 0x23809b1e.48b6.39 C--- 0 scn 0x0c05.f58b5c71
0x15 0x0090.002.00ab3123 0x93d2b38a.0c7a.16 C--- 0 scn 0x0c05.f58ceb88
0x16 0x00e6.007.003fe51c 0x9406818a.e348.38 C--- 0 scn 0x0c05.f7149923
Finished processing ORADEBUG command (#2) 'event 10046 trace name context forever, level 12'
*** 2015-12-25 14:50:35.716
Received ORADEBUG command (#3) 'tracefile_name' from process 'Unix process pid: 40830820, image: <none>'
*** 2015-12-25 14:50:35.716
Finished processing ORADEBUG command (#3) 'tracefile_name'
WAIT #4573544456: nam='SQL*Net message from client' ela= 1000550 driver id=1413697536 #bytes=1 p3=0 obj#=-1 tim=82639769478657
CLOSE #4573544456:c=0,e=6,dep=0,type=3,tim=82639769478774
=====================
PARSING IN CURSOR #4573391400 len=345 dep=0 uid=150 oct=3 lid=150 tim=82639769479837 hv=2794797623 ad='700000de5aa9648' sqlid='cw84a6am9afjr'
SELECT TO_CHAR(ACCOUNTID) ACCOUNTID,INTIME,OLDSTATUS,NEWSTATUS, TO_CHAR(SEQNO) SEQNO, CREDIT*100 CREDIT,BALANCE*100 BALANCE,
NVL(OPER_TYPE,'0') OPER_TYPE, TO_CHAR(SUBSID) SUBSID FROM xulq_test WHERE REGION =:REGION AND ACCOUNTID=:ACCOUNTID AND
TASKNO=:TASKNO AND REFRESHED=0 ORDER BY INTIME,SEQNO ASC
根据MOS的文档，发生ORA-600[kdsgrp1]错误主要有以下几种原因：
1. Bug 6772911 - OERI[12700] OERI[qertbFetchByRowID] OERI[kdsgrp1] due to bad CR rollback of INDEX block (Doc ID 6772911.8)
2. Bug 9457185 - Intermittent ORA-600 [kdsgrp1] during CR read (Doc ID 9457185.8)
3. Bug 8771916 - OERI [kdsgrp1] during CR read (Doc ID 8771916.8)
4. Getting Ora-600 [kdsgrp1] On A Select (Doc ID 429746.1)
该错误会有类似如下的trace信息：
*** SESSION ID:(461.5270) 2007-05-01 23:43:18.775
row 028796f1.125 continuation at
file# 10 block# 497393 slot 294 not found
尝试通过rebuild表 xulq_test上的索引来解决。rebuild 完成后，问题未出现。