netapp fas3220更换硬盘

Netapp存储无法正常工作导致小机数据库无法连接

1.使用sysconfig -r查看系统状态硬盘状态

SBJYJ-02> sysconfig –r 
Aggregate aggr0 (online, raid_dp, degraded, hybrid_enabled) (block checksums)
  Plex /aggr0/plex0 (online, normal, active)
    RAID group /aggr0/plex0/rg0 (degraded, block checksums)

      RAID Disk Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
      --------- ------          ------------- ---- ---- ---- ----- --------------    --------------
      dparity   0b.10.1         0b    10  1   SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
      parity    0b.20.1         0b    20  1   SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      0b.20.17        0b    20  17  SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      0b.10.3         0b    10  3   SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      0b.20.3         0b    20  3   SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      4c.30.3         4c    30  3   SA:A   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      0b.10.5         0b    10  5   SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      0b.20.5         0b    20  5   SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      4c.30.5         4c    30  5   SA:A   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      0b.10.7         0b    10  7   SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      FAILED                  N/A                        560000/ -
      data      4c.30.7         4c    30  7   SA:A   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      0b.10.9         0b    10  9   SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      0b.20.9         0b    20  9   SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      4c.30.9         4c    30  9   SA:A   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      0b.10.11        0b    10  11  SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      0b.20.11        0b    20  11  SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 

    RAID group /aggr0/plex0/rg1 (double degraded, block checksums)

      RAID Disk Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
      --------- ------          ------------- ---- ---- ---- ----- --------------    --------------
      dparity   4c.30.11        4c    30  11  SA:A   0   SAS 15000 560000/1146880000 560208/1147307688 
      parity    0b.20.13        0b    20  13  SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      0b.10.13        0b    10  13  SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      4c.30.13        4c    30  13  SA:A   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      0b.20.15        0b    20  15  SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      FAILED                  N/A                        560000/ -
      data      4c.30.15        4c    30  15  SA:A   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      0b.10.23        0b    10  23  SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      4c.30.1         4c    30  1   SA:A   0   SAS 15000 560000/1146880000 560879/1148681096 
      data      4c.30.17        4c    30  17  SA:A   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      0b.20.19        0b    20  19  SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      FAILED                  N/A                        560000/ -
      data      4c.30.19        4c    30  19  SA:A   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      0b.20.21        0b    20  21  SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      0b.10.21        0b    10  21  SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
      data      0b.10.15        0b    10  15  SA:B   0   SAS 15000 560000/1146880000 560879/1148681096 
      data      0b.20.23        0b    20  23  SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 


Pool1 spare disks (empty)

Pool0 spare disks

RAID Disk       Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
---------       ------          ------------- ---- ---- ---- ----- --------------    --------------
Spare disks for block checksum
spare           0a.00.1         0a    0   1   SA:B   0   SSD   N/A 190532/390209536  190782/390721968 
spare           0a.00.3         0a    0   3   SA:B   0   SSD   N/A 190532/390209536  190782/390721968 

Broken disks

RAID Disk       Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
---------       ------          ------------- ---- ---- ---- ----- --------------    --------------
failed          0b.10.17        0b    10  17  SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
failed          0b.10.19        0b    10  19  SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
failed          0b.20.7         0b    20  7   SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
failed          4c.30.23        4c    30  23  SA:A   0   SAS 15000 560000/1146880000 560208/1147307688 

Partner disks

RAID Disk       Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
---------       ------          ------------- ---- ---- ---- ----- --------------    --------------
partner         0b.10.22        0b    10  22  SA:B   0   SAS 15000 560000/1146880000 560879/1148681096 
partner         4c.30.21        4c    30  21  SA:A   0   SAS 15000 0/0               560879/1148681096 
partner         4c.30.22        4c    30  22  SA:A   0   SAS 15000 0/0               560879/1148681096 
partner         4c.20.10        4c    20  10  SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         0b.10.2         0b    10  2   SA:B   0   SAS 15000 0/0               560208/1147307688 
partner         0b.10.16        0b    10  16  SA:B   0   SAS 15000 0/0               560208/1147307688 
partner         0b.10.20        0b    10  20  SA:B   0   SAS 15000 0/0               560208/1147307688 
partner         0b.10.0         0b    10  0   SA:B   0   SAS 15000 0/0               560208/1147307688 
partner         0b.10.18        0b    10  18  SA:B   0   SAS 15000 0/0               560208/1147307688 
partner         0b.10.8         0b    10  8   SA:B   0   SAS 15000 0/0               560208/1147307688 
partner         4c.30.14        4c    30  14  SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.20.12        4c    20  12  SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.30.6         4c    30  6   SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.20.6         4c    20  6   SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.30.20        4c    30  20  SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.20.2         4c    20  2   SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.20.16        4c    20  16  SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.30.0         4c    30  0   SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.30.16        4c    30  16  SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.20.8         4c    20  8   SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.30.2         4c    30  2   SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         0b.10.12        0b    10  12  SA:B   0   SAS 15000 0/0               560208/1147307688 
partner         0b.10.14        0b    10  14  SA:B   0   SAS 15000 0/0               560208/1147307688 
partner         0b.10.10        0b    10  10  SA:B   0   SAS 15000 0/0               560208/1147307688 
partner         0b.10.4         0b    10  4   SA:B   0   SAS 15000 0/0               560208/1147307688 
partner         0b.10.6         0b    10  6   SA:B   0   SAS 15000 0/0               560208/1147307688 
partner         4c.20.14        4c    20  14  SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.20.22        4c    20  22  SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.30.12        4c    30  12  SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.20.20        4c    20  20  SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.20.18        4c    20  18  SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.30.10        4c    30  10  SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.30.18        4c    30  18  SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.20.0         4c    20  0   SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.20.4         4c    20  4   SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.30.4         4c    30  4   SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4c.30.8         4c    30  8   SA:A   0   SAS 15000 0/0               560208/1147307688 
partner         4a.00.2         4a    0   2   SA:A   0   SSD   N/A 0/0               190782/390721968 
partner         4a.00.0         4a    0   0   SA:A   0   SSD   N/A 0/0               190782/390721968

看到存储做了raid dp 初步断定硬盘损坏,数据不受影响
由以上日志可看出,坏盘一共四块,坏在两个raid组中,vol status -r 命令查看热备盘已用完,无法进行坏硬盘更换,而下方硬盘没有被控制器二所接管,无法看到硬盘具体状态。

2.更换控制器二所属的硬盘
①将坏掉的硬盘拔出,等待30秒(防止磁盘断电后还在转动,防止磁盘造成物理损坏为数据恢复增加困难),插入新硬盘(确认黄灯亮、绿灯不闪烁)
②插入硬盘后disk show -v 查看新硬盘是否分配了owner(控制器)

DISK       OWNER                    POOL   SERIAL NUMBER         DR HOME            CHKSUM
------------ -------------            -----  -------------    -------------            -------  
4a.00.0      esad  (22312421)    Pool0  S142NEAD806058        SBJYJ-01  (2017242430)    Block
4a.00.2      SBJYJ-01  (2017242430)    Pool0  S142NEAD803850        SBJYJ-01  (2017242430)    Block

发现新插入的硬盘不属于此控制器,带有其他控制器的信息或raid信息
使用disk assign  -f    -s   强制分配给一个控制器 *慎用,分配完成后使用disk show –v查看是否分配成功
使用aggr destroy   删除一个AGGR *慎用

③使用vol status -r 查看硬盘状态,如果硬盘为Bad Label,执行步骤(1);如果硬盘已经进入spare disks中,并且磁盘最后标注了not zerod执行步骤(2)
(1)	在vol status –r中看到带有bad label标签的盘,但是已经将新的硬盘安装上
先priv set advanced 进入高级模式,使用 disk unfail -s <硬盘id 0b.**.**> 去除Bad标签,再退出高级模式priv set 
RAID Disk       Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
---------       ------          ------------- ---- ---- ---- ----- --------------    --------------
failed          0b.10.19        0b    10  19  SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
failed          0b.20.7         0b    20  7   SA:B   0   SAS 15000 560000/1146880000 560208/1147307688 
failed          4c.30.23        4c    30  23  SA:A   0   SAS 15000 560000/1146880000 560208/1147307688 
bad label       0b.10.17        0b    10  17  SA:B   0   SAS 15000 560000/1146880000 560879/1148681096
(2)	当热备盘中硬盘后跟not zerod,

RAID Disk       Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
---------       ------          ------------- ---- ---- ---- ----- --------------    --------------
Spare disks for block checksum
spare           0a.00.1         0a    0   1   SA:B   0   SSD   N/A 190532/390209536  190782/390721968   not zerod
spare           0a.00.3         0a    0   3   SA:B   0   SSD   N/A 190532/390209536  190782/390721968 

使用命令disk zero spares 初始化所有spare disk

拉起控制器一
控制器一因硬盘损坏,进行保护硬盘,自己关闭了控制器,防止用户继续访问存储造成硬盘的继续损坏导致数据丢失
使用console线直连存储,会进入LOADER>模式,使用help命令查看可用命令,使用boot_ontap命令强制控制器启动(若无法启动,则控制器可能损坏),拉起控制器后进入控制器更换硬盘即可

总结
存储进行重构、初始化需要一段时间,当存储硬盘重构、初始化完成后raid组自动将降级取消,存储正常运行

参考命令
sysconfig -v 查看存储状态
sysconfig -r 查看存储硬盘状态
sysconfig -a 查看系统信息详情
vol status -v 查看volume状态
vol status -f 检查是否有故障硬盘
disk show 查看磁盘分配信息
disk show -v 查看硬盘所属控制器
storage show disk -p 查看硬盘位置
disk zero spares 初始化所有spare disk
environment status 检查电源、风扇状态
rdfile /etc/messages 检查最新的日志
cf status 检查控制器状态
df 检查逻辑卷磁盘使用率
environment status 查看环境信息
license 查看许可信息
ifconfig -a 查看网络配置
aggr status 查看raid组信息
aggr status -r 查看raid组详情
 df -Vh 查看卷空间
df -Ah 查看aggr空间

disk assign  -f  -s  强制分配给一个控制器 *慎用
disk replace start disk_name spare_disk_name 使用spare disk 替换一块磁盘 *慎用
disk replace stop disk_name 停止替换硬盘 *慎用
disk sanitize start disk_name 将磁盘上面所有数据移除 *慎用
disk sanitize abort disk_name 停止 *慎用
aggr destroy aggrname  删除一个AGGR *慎用
disk remove onwership disk_name 删除硬盘owner *慎用

你可能感兴趣的:(存储)