AMDU and CHECK NOREPAIR report conflicting corruption results for ASM Diskgroup (Doc ID 1535055.1)

In this Document

Symptoms

Cause

Solution

References

APPLIES TO:

Oracle Database - Enterprise Edition - Version 11.1.0.7 and later
Information in this document applies to any platform.

SYMPTOMS

Why would AMDU report ASM metadata corruption when CHECK NOREPAIR does not?

From ASM's alert log:
~~~~~~~~~~~~~~~~~~~~~~
SQL> alter diskgroup FRAEDTRNPA check all norepair 
NOTE: starting check of diskgroup FRAEDTRNPA
Fri Nov 09 10:19:43 2012
GMON checking disk 3 for group 29 at 167 for pid 30, osid 27582
GMON checking disk 4 for group 29 at 168 for pid 30, osid 27582
SUCCESS:  check of diskgroup FRAEDTRNPA found no errors  <======================== No errors from CHECK NOREPAIR
SUCCESS: alter diskgroup

From AMDU's report.txt:
~~~~~~~~~~~~~~~~~~~~~~~
...
  *** Slept for 6 seconds waiting for heartbeats *** <=================  Indication the ASM diskgroup is mounted
...
---------------------- SUMMARY FOR DISKGROUP FRAEDTRNPA ----------------------
           Allocated AU's: 6487
                Free AU's: 95911
       AU's read for dump: 169
       Block images saved: 37649
        Map lines written: 169
          Heartbeats seen: 0
   Corrupt metadata blocks: 1024  <==================================================== AMDU reports corruption
        Corrupt AT blocks: 0

CAUSE

Ref:

Bug 15855599  : FOR ASM DISKGROUP AMDU REPORTS CORRUPTION 'CHECK ALL NOREPAIR' REPORTS NO ERRORS

Bug 16263812  : ASM DISKGROUP AMDU REPORTS CORRUPTION 'CHECK ALL NOREPAIR' REPORTS NO ERRORS

SOLUTION

AMDU is a dump tool, designed to obtain ASM metadata details primarily on dismounted ASM disk groups.

Although AMDU can be run on mounted ASM disk groups, by design it is not aware if there are any dirty ASM metadata blocks buffered in the ASM instance buffer cache.

Therefore, if AMDU dumps are collected while the disk group is mounted and active (ie, ongoing I/O), there is a possibility AMDU could interpret the dirty ASM metadata blocks cached in memory and not yet written to disk (in-flight), as inconsistent and thus report corrupted metadata blocks.

If CHECK NOREPAIR reports no corruption on a disk group that is mounted, then AMDU ought no longer report corruption once the asm disk group is dismounted normal (ie, not 'dismount force). If this is a clustered environment, the disk group must be dismounted across all clustered ASM instances.

Important: As a precautionary step or best practice, before attempting to dismount the ASM disk group, make sure RMAN back ups on the database are current, and stored in a safe place.

REFERENCES

BUG:16263812  - ASM DISKGROUP AMDU REPORTS CORRUPTION 'CHECK ALL NOREPAIR' REPORTS NO ERRORS
NOTE:1485597.1  - ASM tools used by Support : KFOD, KFED, AMDU