vmax Degraded lun

Issue

VPLEX Online help indicates the following cause for a degrade connectivity status:

Degraded

A LUN exported from the array does not have two paths to each director.

On a CLARiiON:

If you have collect diagnostics, look at the debugTowerDump files. Search for "scsi view array" and you should see an output similar to below:

COMMAND scsi view array * all

DIRECTOR /engines/engine-1-1/directors/director-1-1-A

a: EMC~CLARiiON~APM00111000XXX name APM00111000XXX vend "DGC     "
    prod "VRAID           " rev "0532" type EMC~CLARiiON
    c: APM00111000XXX.SPA
    c: APM00111000XXX.SPB
    u: VPD83T3:600601607ee028006115390b2d25e311
    u: VPD83T3:600601607ee02800bf8e45f52c25e311
    u: VPD83T3:50060160bde007a9
    l: "x fcp i 0x5000144280728a13 t 0x5006016c3de007a9"
    l: "x fcp i 0x5000144280728a12 t 0x500601683de007a9"
    l: "x fcp i 0x5000144280728a13 t 0x500601643de007a9"
    l: "x fcp i 0x5000144280728a12 t 0x500601603de007a9"
    p: "x fcp i 0x5000144280728a13 t 0x500601643de007a9 l 0x0000000000000000" AAO
    p: "x fcp i 0x5000144280728a12 t 0x500601683de007a9 l 0x0000000000000000" DL0
    p: "x fcp i 0x5000144280728a13 t 0x500601643de007a9 l 0x0001000000000000" AAO
    p: "x fcp i 0x5000144280728a12 t 0x500601603de007a9 l 0x0001000000000000" AAO
    p: "x fcp i 0x5000144280728a13 t 0x5006016c3de007a9 l 0x0000000000000000" DL0
    p: "x fcp i 0x5000144280728a12 t 0x500601603de007a9 l 0x0000000000000000" AAO

Note the paths which end in a DL0. These represent Default LUN 0 because VPLEX does not see storage on these paths and assigns a DL0 (similar to a LUNZ). With proper backend connectivity, all paths should show as AAN and AAO.

a: EMC~CLARiiON~APM00051002XXX name APM00051002XXX vend "DGC     "
    prod "RAID 5          " rev "0226" type EMC~CLARiiON
        c: APM00051002XXX.SPB
        c: APM00051002XXX.SPA
        u: VPD83T3:60060160fd221100f5e474acaadfe211
        l: "x fcp i 0x5000144260072011 t 0x50060169106026fe"
        l: "x fcp i 0x5000144260072010 t 0x50060169106026fe"
        l: "x fcp i 0x5000144260072010 t 0x50060160106026fe"
        l: "x fcp i 0x5000144260072011 t 0x50060160106026fe"
        l: "x fcp i 0x5000144260072011 t 0x50060161106026fe"
        l: "x fcp i 0x5000144260072010 t 0x50060161106026fe"
        l: "x fcp i 0x5000144260072011 t 0x50060168106026fe"
        l: "x fcp i 0x5000144260072010 t 0x50060168106026fe"
        p: "x fcp i 0x5000144260072010 t 0x50060161106026fe l 0x0000000000000000" AAN
        p: "x fcp i 0x5000144260072010 t 0x50060169106026fe l 0x0000000000000000" AAO
        p: "x fcp i 0x5000144260072010 t 0x50060160106026fe l 0x0000000000000000" AAN
        p: "x fcp i 0x5000144260072010 t 0x50060168106026fe l 0x0000000000000000" AAO
        p: "x fcp i 0x5000144260072011 t 0x50060161106026fe l 0x0000000000000000" AAN
        p: "x fcp i 0x5000144260072011 t 0x50060169106026fe l 0x0000000000000000" AAO
        p: "x fcp i 0x5000144260072011 t 0x50060160106026fe l 0x0000000000000000" AAN
        p: "x fcp i 0x5000144260072011 t 0x50060168106026fe l 0x0000000000000000" AAO


On a Symmetrix:

Array and logical-units show as degraded, but all storage-volumes are healthy.

EMC-SYMMETRIX-190100XXX      degraded      -    0x50060482d52cc607,  474
                                                                                          0x50060482d52cc608,
                                                                                          0x50060482d52cc617,
                                                                                          0x50060482d52cc618,
                                                                                          0x50060482d52cc627,
                                                                                          0x50060482d52cc628,

/clusters/cluster-1/storage-elements/storage-arrays/EMC-SYMMETRIX-190100XXX/logical-units:
Name                                                                         Connectivity   Active/AAO   Passive/AAN  Visibility    LUNs                            ALUA
----------------------------------------                      Status             Controllers     Controllers      ----------  -------------------         Support
----------------------------------------                      ------------    -----------      -----------      ----------  -------------------         -------
VPD83T3:60060480000190100248533030344442  degraded      190100XXX                          active        0x004f000000000000   none
VPD83T3:60060480000190100248533030344445  degraded      190100XXX                          active        0x0050000000000000   none
VPD83T3:60060480000190100248533030344530  degraded      190100XXX                          active        0x0051000000000000   none
VPD83T3:60060480000190100248533030344534  degraded      190100XXX                          active        0x0052000000000000   none
VPD83T3:60060480000190100248533030344537  degraded      190100XXX                          active        0x0053000000000000   none
VPD83T3:60060480000190100248533030344543  degraded      190100XXX                          active        0x0055000000000000   none
VPD83T3:60060480000190100248533030344630  degraded      190100XXX                          active        0x0056000000000000   none

                                             

VPlexcli:/> connectivity validate-be

Storage volumes that are dead or unreachable:

Storage volumes which do not meet the high availability requirement for storage volume paths*:

Cluster    Director             Array                                
Storage volumes which do not meet the high availability requirement for storage volume paths*
----------    --------------         ------------------------------------  ---------------------------------------------------------------------------------------------
cluster-1  director-1-1-B  (190100XXX, EMC~SYMMETRIX~190100XXX)        VPD83T3:60060480000190100248533031364442                                                                                               VPD83T3:60060480000190100248533030354634
VPD83T3:60060480000190100248533031364438
VPD83T3:60060480000190100248533030363138
VPD83T3:60060480000190100248533031333441
VPD83T3:60060480000190100248533031373731
VPD83T3:60060480000190100248533030353543
VPD83T3:60060480000190100248533030353141
VPD83T3:60060480000190100248533031333439
                                                                                                                    ...
Summary
Cluster cluster-1
    5 storage-volumes which are dead or unreachable.
    104 storage-volumes which do not meet the high availability requirement for storage volume paths*.
    Director director-1-1-B
        Storage array: ('190100XXX', 'EMC~SYMMETRIX~190100XXX') has 104 storage-volumes which do not meet the high availability requirement for storage volume paths*.
    0 storage-volumes which are not visible from all directors.
    0 storage-volumes which have more than supported (4) active paths from same director.
    *To meet the high availability requirement for storage volume paths each storage volume must be accessible from each of the directors through 2 or more VPlex backend ports, and 2 or more Array target ports, and there should be 2 or more ITLs.
Environment     

EMC Hardware: VPLEX Series
EMC Hardware: VPLEX VS1
EMC Hardware: VPLEX VS2
EMC Hardware: VPLEX-Local
EMC Hardware: VPLEX-Metro
EMC Hardware: VPLEX-Geo

EMC Software: GeoSynchrony 4.0
EMC Software: GeoSynchrony 4.0.1
EMC Software: GeoSynchrony 4.1
EMC Software: GeoSynchrony 4.2
EMC Software: GeoSynchrony 4.2 Patch1
EMC Software: GeoSynchrony 5.0
EMC Software: GeoSynchrony 5.0.1
EMC Software: GeoSynchrony 5.0.1 Patch 1
EMC Software: GeoSynchrony 5.0.1 Patch 2
EMC Software: GeoSynchrony 5.1
EMC Software: GeoSynchrony 5.1 Patch 1
EMC Software: GeoSynchrony 5.1 Patch 2
EMC Software: GeoSynchrony 5.1 Patch 3
EMC Software: GeoSynchrony 5.1 Patch 4
EMC Software: GeoSynchrony 5.2
EMC Software: GeoSynchrony 5.2 Patch 1
EMC Software: GeoSynchrony 5.2 Service Pack 1

EMC Hardware: Symmetrix Series
EMC Hardware: VNX Series
EMC Hardware: CLARiiON Series

All supported third party array types
Cause     

VPLEX Online help indicates the following cause for a degrade connectivity status:

Degraded
    

A LUN exported from the array does not have two paths to each
director.

However, a LUN will not enter a degraded status if only one path exists from the array to the VPLEX director,

For example, if the zoning is not present, or if the VPLEX director or array ports are not enabled.

This condition is caused when there are multiple paths from the array to a given VPLEX director, but the LUN is not mapped/masked to the necessary VPLEX ports.

It is possible to hit this issue on any array type, but is less common on array types where all initiators are registered to a single host entry. This article uses a Symmetrix as an example.

In the case of active/passive arrays, confirm that all initiators are registered and in the storage-group. Any VPLEX initiators which are unregistered and not in a storage-group will cause the array status to be degraded
Change     

User had made zoning changes in the fabric which requires the initiators to be manually unregistered on the array (change the mapping and masking), then once the rezoning is done and the initiators have logged back onto the VNX they will be on different ports and so needed to be registered and added back to the storage-group and these two steps were missed.
Resolution     

To verify this issue, pick one of the degraded devices and check the mapping and masking. For a Symmetrix, use the following commands:

VPlexcli:/clusters/cluster-1/storage-elements/storage-arrays/EMC-SYMMETRIX-190100XXX/logical-units>

VPD83T3:60060480000190100248533031333434  degraded      190100XXX                 active      0x006e000000000000   none
 

[root@sanadm]# symdev -sid 0XXX show -wwn 60060480000190100248533031333434

       Device Physical Name     : Not Visible

   Device Symmetrix Name    : 1344
   Device Serial ID         : N/A
   Symmetrix ID             : 000190100XXX

   Attached BCV Device      : N/A

[root@sanadm]# symmaskdb -sid 0XXX list assignment -dev 1344

Symmetrix ID : 000190100XXX

Device  Identifier        Type   Dir:P
------  ----------------  -----  ----------------
1344    50001442806cbd12  FIBRE  FA-8A:0
        50001442804c2f12  FIBRE  FA-8A:0
        50001442904c2f12  FIBRE  FA-8A:0
        50001442806cbd13  FIBRE  FA-9A:0
        50001442906cbd13  FIBRE  FA-9A:0
        50001442804c2f13  FIBRE  FA-9A:0
        50001442904c2f13  FIBRE  FA-9A:0


Note:  The pwwns listed under the Identifier column. These are the VPLEX initiator ports. Check these against the available VPLEX initiator ports, by running 'll /engines/**/ports' from vplexcli. Ensure that two ports from each director are present in the symmaskdb output. In this example, director-1-1-B port B1-FC02 is missing from the symmaskdb output. This matches the output of 'connectivity validate-be' which indicates that director-1-1-B did not have the required number of paths.

VPlexcli:/> ll engines/**/ports

/engines/engine-1-1/directors/director-1-1-A/hardware/ports:
Name     Address             Role       Port Status
-------  ------------------  ---------  -----------
A0-FC00  0x50001442806cbd00  front-end  up
A0-FC01  0x50001442806cbd01  front-end  up
A0-FC02  0x50001442806cbd02  front-end  up
A0-FC03  0x50001442806cbd03  front-end  up
A1-FC00  50001442806cbd10    back-end   up
A1-FC01  50001442806cbd11    back-end   up
A1-FC02  50001442806cbd12    back-end   up
A1-FC03  50001442806cbd13    back-end   up
A3-FC00  0x50001442806cbd30  local-com  up
A3-FC01  0x50001442806cbd31  local-com  up
A3-FC02  0x0000000000000000  -          down
A3-FC03  0x0000000000000000  -          down


/engines/engine-1-1/directors/director-1-1-B/hardware/ports:
Name     Address             Role       Port Status
-------  ------------------  ---------  -----------
B0-FC00  0x50001442906cbd00  front-end  up
B0-FC01  0x50001442906cbd01  front-end  up
B0-FC02  0x50001442906cbd02  front-end  up
B0-FC03  0x50001442906cbd03  front-end  up
B1-FC00  50001442906cbd10    back-end   up
B1-FC01  50001442906cbd11    back-end   up
B1-FC02  50001442906cbd12    back-end   up <---
B1-FC03  50001442906cbd13    back-end   up
B3-FC00  0x50001442906cbd30  local-com  up
B3-FC01  0x50001442906cbd31  local-com  up
B3-FC02  0x0000000000000000  -          down
B3-FC03  0x0000000000000000  -          down


/engines/engine-1-2/directors/director-1-2-A/hardware/ports:
Name     Address             Role       Port Status
-------  ------------------  ---------  -----------
A0-FC00  0x50001442804c2f00  front-end  up
A0-FC01  0x50001442804c2f01  front-end  up
A0-FC02  0x50001442804c2f02  front-end  up
A0-FC03  0x50001442804c2f03  front-end  up
A1-FC00  50001442804c2f10    back-end   up
A1-FC01  50001442804c2f11    back-end   up
A1-FC02  50001442804c2f12    back-end   up
A1-FC03  50001442804c2f13    back-end   up
A3-FC00  0x50001442804c2f30  local-com  up
A3-FC01  0x50001442804c2f31  local-com  up
A3-FC02  0x0000000000000000  -          down
A3-FC03  0x0000000000000000  -          down


/engines/engine-1-2/directors/director-1-2-B/hardware/ports:
Name     Address             Role       Port Status
-------  ------------------  ---------  -----------
B0-FC00  0x50001442904c2f00  front-end  up
B0-FC01  0x50001442904c2f01  front-end  up
B0-FC02  0x50001442904c2f02  front-end  up
B0-FC03  0x50001442904c2f03  front-end  up
B1-FC00  50001442904c2f10    back-end   up
B1-FC01  50001442904c2f11    back-end   up
B1-FC02  50001442904c2f12    back-end   up
B1-FC03  50001442904c2f13    back-end   up
B3-FC00  0x50001442904c2f30  local-com  up
B3-FC01  0x50001442904c2f31  local-com  up
B3-FC02  0x0000000000000000  -          down
B3-FC03  0x0000000000000000  -          down


For VNX or CLARiiON, log in to Unisphere, go to the initiators tab and look for any VPLEX initiators which are logged into the array, but not registered AND in the VPLEX storage-group.
        
To resolve the issue, add the missing port(s) to the mapping/masking/storage-group on the array. Once this is done, run an array re-discover for VPLEX to pick up the new path.