Database 11g: Quick Steps to Package and Send Critical Error Diagnostic Information to Support

Applies to:

Oracle Server - Enterprise Edition - Version 11.1.0.6 and later
Information in this document applies to any platform.

Goal

Starting with Oracle11g, alert and trace information is generated in a new format and stored in the Automatic Diagnostic Repository (ADR).

Refer to Document 422893.1 to know the ADR in detail.

This article explains how to find and send the alert.log and relevant trace files to Oracle Global Software Support when a critical error occurs in database, using the ADRCI tool.

Fix

Oracle11g provides tools and methods to automatically identify and gather trace files related to a problem (a critical error in the database environment) or an incident (an occurrence of a problem). This facility is known as the Incident Packaging Services (IPS). The Support Workbench (GUI) and ADRCI (command line) utilities are the two interfaces to the Incident Packaging Services. Every occurrence of a critical error (problem) in the database creates an incident.

Steps in identifying and sending files to support.

1. You are notified of a critical error reported in the database environment:

SQL> select * from atab;
select * from atab
*
ERROR at line 1:
ORA-01578: ORACLE data block corrupted (file # 6, block # 11)
ORA-01110: data file 6: '/opt/oracle/oradata/db11g/tt.dbf'

2. Check for the error reported in the instance alert file and in the ADR.

To check for the error reported in the ADR:

With the 11g environment being set, from the OS command prompt invoke the ADRCI utility:

$ adrci

Let ADRCI know which Oracle environment to investigate:

adrci> show home
--show home will list all the ADR homes. Set the proper database ADR home for use with ADRCI.
adrci> set homepath  database_home

Check for the problems reported:

adrci> show problem

ADR Home = /opt/oracle/diag/rdbms/db11g/db11g:
*************************************************************************
PROBLEM_ID   PROBLEM_KEY    LAST_INCIDENT   LASTINC_TIME
------------ -------------- --------------- ---------------------------------
1            ORA 1578       18104           2009-06-01 22:06:19.501207 +10:00
1 rows fetched

You can have multiple problems reported. Create a package for the one which needs to be investigated.

3. Check for the incidents reported for the problem which needs to be packaged

Each occurrence of the problem is called an 'incident'. The instance alert file will show all the incidents reported. Each incident will be associated with a unique incident id in the ADR.

From the ADRCI prompt, execute the SHOW INCIDENT command for the problem to be investigated, by specifying the associated problem key. You can get the problem key from the SHOW PROBLEM command:

adrci> show incident -p "problem_key='ORA 1578'"

ADR Home = /opt/oracle/diag/rdbms/db11g/db11g:
*************************************************************************
INCIDENT_ID     PROBLEM_KEY              CREATE_TIME
--------------- ------------------------ ----------------------------------
18147           ORA 1578                 2009-06-01 22:02:08.805002 +10:00
1 rows fetched

You can see multiple incidents reported for the same problem.

Note the incident number for which you need to create a package.

4. Package the trace files invoking IPS.

Generate the package using the IPS PACK command and store the generated package in a directory of your choice. The example below shows the package for the incident will be created in the /tmp directory.

adrci> ips pack incident 18147 in /tmp
Generated package 9 in file /tmp/ORA1578_20090602113045_COM_1.zip, mode complete

If at this step the ADRCI utility complains that the incident was flood-controlled and that no package can be generated for it, then instead of choosing the most recent incident to be packaged, choose the first incident that occurred after an instance startup.

Examples of IPS PACK include:

ips pack problem 100 in /tmp
--Generates the package for the problem id 100 in /tmp

ips pack incident 6439 in /tmp
--Generates the package for the incident id 6439 in /tmp

ips pack problemkey "ORA 1578"
--Generates the package for the problem with the problem_key 'ORA 1578'

ips pack seconds 8
--Generates the package with the incidents occurred in last 8 seconds.

ips pack time '2007-05-01 10:00:00.00' to '2007-05-01 23:00:00.00'
--Generates the package with the incidents occurred between the times '2007-05-01 10:00:00.00' and '2007-05-01 23:00:00.00'

Upload the ZIP file generated to the Service Request you created on the My Oracle Support website . This file will include all the trace files, instance alert file and other diagnostic information for the critical error. Make sure that the ZIP file is uploaded WITHOUT any post processing (such as TAR).