一、故障现象:
1、在做了redo log当前日志组被破坏恢复的测试后
2、启动数据库后出现ORA-600 【4193】的错误
3、数据库被强制关闭
查看告警日志:
[oracle@ocm1 ~]$ tail -f /u01/app/oracle/diag/rdbms/enmoedu/enmoedu/trace/alert_enmoedu.log
Block recovery completed at rba 5.111.16, scn 0.1430641
Errors in file /u01/app/oracle/diag/rdbms/enmoedu/enmoedu/trace/enmoedu_pmon_10635.trc (incident=36027):
ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/enmoedu/enmoedu/incident/incdir_36027/enmoedu_pmon_10635_i36027.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Tue Dec 13 12:54:04 2016
Dumping diagnostic data in directory=[cdmp_20161213125404], requested by (instance=1, osid=10635 (PMON)), summary=[incident=36027].
Errors in file /u01/app/oracle/diag/rdbms/enmoedu/enmoedu/trace/enmoedu_pmon_10635.trc:
ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], []
PMON (ospid: 10635): terminating the instance due to error 472
System state dump requested by (instance=1, osid=10635 (PMON)), summary=[abnormal instance termination].
System State dumped to trace file /u01/app/oracle/diag/rdbms/enmoedu/enmoedu/trace/enmoedu_diag_10653.trc
Dumping diagnostic data in directory=[cdmp_20161213125405], requested by (instance=1, osid=10635 (PMON)), summary=[abnormal instance termination].
Instance terminated by PMON, pid = 10635
查看trace文件:
[oracle@ocm1 ~]$more /u01/app/oracle/diag/rdbms/enmoedu/enmoedu/incident/incdir_38544/enmoedu_mmon_3181_i38544.trc
......
----- START DDE Action: 'dumpKernelDiagState' (Sync) -----
------- Kernel Diag Dump -------
dbkcBSExt: 0
dbkedDefDump info:
Internal err count: 4
Error Flags: 0x0
Exception: FALSE
Bootstrapping info:
Flags: 0x17
Options: 0x806
Diag Dest: /u01/app/oracle
DB Unique name: enmoedu
Instance Name: enmoedu
------- END Kernel Diag Dump -------
----- END DDE Action: 'dumpKernelDiagState' (SUCCESS, 0 csec) -----
----- START DDE Action: 'xdb_dump_buckets' (Sync) -----
----- END DDE Action: 'xdb_dump_buckets' (FAILURE, 0 csec) -----
----- START DDE Action: 'dumpKGERing' (Sync) -----
----- END DDE Action: 'dumpKGERing' (SUCCESS, 0 csec) -----
----- START DDE Action: 'dumpKGEState' (Sync) -----
kgepgtfr 0x7fffb09068d0
kgepgtba 0x7fffb09107a8
kgepgter 5
kgepgpar kgepgbpa 0xbaf48c5
kgepgepa 0xbaf5064
kgepgtfd 21
kgepgdmc 0
kgepgflg 0x8
kgepg_stkgfr (nil)
kgepgkgsmp 0xbaf3fa0
kgepgspm 4
kgepg_ba_set_in_eh 0x7fffb09114b0
kgepg_kgecatch_set_in_eh_ba (nil)
kge_ba_set_in_eh_funcloc 0x9b975bc
kge_ba_set_in_eh_fileloc 0x9b97890
------------------- start error stack dump with barriers
ORA-00603: ORACLE server session terminated by fatal error
ORA-24557: error 600 encountered while handling error 600; exiting server process
ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], []
ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], []
ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], []
------------------- end error stack dump with barriers
----- END DDE Action: 'dumpKGEState' (SUCCESS, 0 csec) -----
----- START DDE Action: 'kpuActionDefault' (Sync) -----
Begin OCI Current State Dump
End OCI Current State Dump
Begin OCI Call Context Dump
End OCI Call Context Dump
Begin Process state dump.
ttcdrvdmplocation: msg-0 ln-0 reporting 0
HST is NULL or no two task connection
End Process state dump.
----- END DDE Action: 'kpuActionDefault' (SUCCESS, 0 csec) -----
----- END DDE Actions Dump (total 1 csec) -----
End of Incident Dump
根据MOS介绍,此故障一般和undo segment有关
二、解决方法:
1、通过spfile生成pfile
01:55:31 SYS@ enmoedu>create pfile from spfile;
File created.
2、编辑pfile文件
[oracle@ocm1 dbs]$ vi initenmoedu.ora
#*.undo_tablespace='UNDOTBS1'
undo_management = 'MANUAL'
rollback_segments = 'SYSTEM'
3、通过pfile启动Instance
01:58:48 SYS@ enmoedu>startup mount pfile='$ORACLE_HOME/dbs/initenmoedu.ora';
ORACLE instance started.
Total System Global Area 521936896 bytes
Fixed Size 2229944 bytes
Variable Size 360712520 bytes
Database Buffers 155189248 bytes
Redo Buffers 3805184 bytes
Database mounted.
Elapsed: 00:00:00.00
02:00:07 SYS@ enmoedu>show parameter undo
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
undo_management string MANUAL
undo_retention integer 900
undo_tablespace string
4、打开数据库
02:00:16 SYS@ enmoedu>alter database open;
Database altered.
此时打开数据库正常:
[oracle@ocm1 ~]$ tail -f /u01/app/oracle/diag/rdbms/enmoedu/enmoedu/trace/alert_enmoedu.log
alter database open
Beginning crash recovery of 1 threads
Started redo scan
Completed redo scan
read 43 KB redo, 36 data blocks need recovery
Started redo application at
Thread 1: logseq 7, block 3
Recovery of Online Redo Log: Thread 1 Group 1 Seq 7 Reading mem 0
Mem# 0: /u01/app/oracle/oradata/enmoedu/redo01.log
Completed redo application of 0.03MB
Completed crash recovery at
Thread 1: logseq 7, block 90, scn 1491526
36 data blocks read, 36 data blocks written, 43 redo k-bytes read
Wed Dec 14 02:00:26 2016
Thread 1 advanced to log sequence 8 (thread open)
Thread 1 opened at log sequence 8
Current log# 2 seq# 8 mem# 0: /u01/app/oracle/oradata/enmoedu/redo02.log
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Wed Dec 14 02:00:26 2016
SMON: enabling cache recovery
Undo initialization finished serial:0 start:479574 end:479584 diff:10 (0 seconds)
Verifying file header compatibility for 11g tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
SMON: enabling tx recovery
Database Characterset is ZHS16GBK
No Resource Manager plan active
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
Wed Dec 14 02:00:27 2016
QMNC started with pid=20, OS id=3522
Completed: alter database open
Wed Dec 14 02:00:28 2016
Starting background process CJQ0
Wed Dec 14 02:00:28 2016
CJQ0 started with pid=26, OS id=3554
Wed Dec 14 02:00:30 2016
db_recovery_file_dest_size of 2048 MB is 0.00% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
Wed Dec 14 02:00:51 2016
Errors in file /u01/app/oracle/diag/rdbms/enmoedu/enmoedu/trace/enmoedu_j001_3562.trc:
ORA-01552: cannot use system rollback segment for non-system tablespace 'TEMP'
5、删除原有的undo tablespace创建新的undo tablespace
02:00:27 SYS@ enmoedu>drop tablespace undotbs1 including contents and datafiles;
Tablespace dropped.
02:03:32 SYS@ enmoedu>create undo tablespace undotbs1
02:03:39 2 datafile '/u01/app/oracle/oradata/enmoedu/undotbs01.dbf' size 100m
02:03:50 3 autoextend on;
Tablespace created.
6、关闭数据库,重新通过spfle启动
02:04:02 SYS@ enmoedu>shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
02:05:45 SYS@ enmoedu>startup
ORACLE instance started.
Total System Global Area 521936896 bytes
Fixed Size 2229944 bytes
Variable Size 360712520 bytes
Database Buffers 155189248 bytes
Redo Buffers 3805184 bytes
Database mounted.
Database opened.
查看告警日志,数据库启动正常,问题解决!
[oracle@ocm1 ~]$ tail -f /u01/app/oracle/diag/rdbms/enmoedu/enmoedu/trace/alert_enmoedu.log
ALTER DATABASE OPEN
Thread 1 opened at log sequence 8
Current log# 2 seq# 8 mem# 0: /u01/app/oracle/oradata/enmoedu/redo02.log
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
SMON: enabling cache recovery
[3870] Successfully onlined Undo Tablespace 2.
Undo initialization finished serial:0 start:808234 end:808274 diff:40 (0 seconds)
Verifying file header compatibility for 11g tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
SMON: enabling tx recovery
Database Characterset is ZHS16GBK
No Resource Manager plan active
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
Wed Dec 14 02:05:55 2016
QMNC started with pid=20, OS id=3874
Completed: ALTER DATABASE OPEN
Wed Dec 14 02:05:56 2016
db_recovery_file_dest_size of 2048 MB is 0.00% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
Starting background process CJQ0
Wed Dec 14 02:05:56 2016
CJQ0 started with pid=22, OS id=3902