Netbackup code 98错误实战解析

前两天nbu的几个备份策略突然报错,具体错误代码error requesting media (tpreq) (98), 错误的具体信息是这样的

2012/09/13 17:13:18 - Info bphdb(pid=6828388) Backup started          
2012/09/13 17:13:18 - Info bphdb(pid=6840628) Backup started          
2012/09/13 17:13:18 - Info bptm(pid=5632) start           
2012/09/13 17:13:18 - Info bptm(pid=5632) using 65536 data buffer size       
2012/09/13 17:13:20 - Info bptm(pid=5632) setting receive network buffer to 263168 bytes     
2012/09/13 17:13:20 - Info bptm(pid=5632) using 30 data buffers        
2012/09/13 17:13:20 - Info bptm(pid=5632) start backup          
2012/09/13 17:13:20 - Info bptm(pid=5632) backup child process is pid 4608.4996      
2012/09/13 17:13:20 - Info bptm(pid=5632) Waiting for mount of media id 0044L4 (copy 1) on server S4PB02.
2012/09/13 17:13:20 - Info bptm(pid=4608) start           
2012/09/13 17:13:20 - mounting 0044L4
2012/09/13 17:13:24 - Error bptm(pid=5632) error requesting media, TpErrno = Robot operation failed    
2012/09/13 17:13:24 - Warning bptm(pid=5632) media id 0044L4 load operation reported an error    
2012/09/13 17:13:24 - current media 0044L4 complete, requesting next resource Any
2012/09/13 17:13:28 - current media -- complete, awaiting next media Any Reason: Media is in use, Media Server: S4PB02,
  Robot Number: 0, Robot Type: TLD, Media ID: N/A, Drive Name: N/A,
  Volume Pool: SAP-BACKUP-POOL, Storage Unit: S4PB02-hcart-robot-tld-0, Drive Scan Host: N/A

2012/09/13 17:13:30 - end writing
error requesting media (tpreq) (98)

从以上的报错信息提取两行关键信息:
Error bptm(pid=5632) error requesting media, TpErrno = Robot operation failed
Warning bptm(pid=5632) media id 0044L4 load operation reported an error
该错误提示机械臂操作失败，0044L4磁带载入作业报告了一个错误。

首先查看当前备份策略运行状态截图

当前有多个作业在跑,而工作的驱动器只有一个(带黄圈的),正常情况下是两个驱动器可以同时跑备份作业.

首先使用scan命令检查带库状态
C:\Program Files\Veritas\Volmgr\bin>scan
************************************************************
*********************** SDT_TAPE    ************************
*********************** SDT_CHANGER ************************
************************************************************
------------------------------------------------------------
Device Name  : ""
Passthru Name: ""
Volume Header: ""
Port: 3; Bus: 0; Target: 1; LUN: 1
Inquiry    : "IBM     3573-TL         B.20"
Vendor ID  : "IBM     "
Product ID : "3573-TL         "
Product Rev: "B.20"
Serial Number: "00X4U78N3182_LL0"
WWN          : ""
WWN Id Type  : 0
Device Identifier: "IBM     3573-TL         00X4U78N3182_LL0"
Device Type    : SDT_CHANGER
NetBackup Robot Type: 8
Removable      : Yes
Device Supports: SCSI-5
Number of Drives : 2
Number of Slots  : 45
Number of Media Access Ports: 3
Drive 1 Serial Number      : "9310088198"
Drive 2 Serial Number      : "1310242183"
Flags : 0x0
Reason: 0x0
------------------------------------------------------------
Device Name  : ""
Passthru Name: ""
Volume Header: ""
Port: 3; Bus: 0; Target: 2; LUN: 1
Inquiry    : "IBM     3573-TL         B.20"
Vendor ID  : "IBM     "
Product ID : "3573-TL         "
Product Rev: "B.20"
Serial Number: "00X4U78N3182_LL0"
WWN          : ""
WWN Id Type  : 0
Device Identifier: "IBM     3573-TL         00X4U78N3182_LL0"
Device Type    : SDT_CHANGER
NetBackup Robot Type: 8
Removable      : Yes
Device Supports: SCSI-5
Number of Drives : 2
Number of Slots  : 45
Number of Media Access Ports: 3
Drive 1 Serial Number      : "9310088198"
Drive 2 Serial Number      : "1310242183"
Flags : 0x0
Reason: 0x0

发现状态正常,然后使用tpconfig命令检查驱动器状态
C:\Program Files\Veritas\Volmgr\bin>tpconfig  -d
Id  DriveName           Type   Residence
      SCSI coordinates/Path                                            Status
****************************************************************************
1   IBM.ULT3580-TD4.001  hcart  TLD(0)  DRIVE=2
      {3,0,2,0}                                                        UP
2   IBM.ULT3580-TD4.003  hcart  TLD(0)  DRIVE=1
      {3,0,1,0}                                                        UP

Currently defined robotics are:
  TLD(0)     SCSI coordinates = {3,0,1,1}

EMM Server = S4PB02

发现驱动器状态正常,那么为什么只有一个驱动器正常呢,难道卡带了.试着在磁带机中移动一盘磁带看,发现可正常移入移出,应该不是卡带的问题.

现场检查备份服务器和磁带机后部的两个驱动器的光纤卡和指示灯闪烁状态，发现光纤卡一切正常。有工程师建议说重启带库,在这里我们要注意一个观察一个问题,如果你发现有某块光纤卡无信号灯闪烁，而此时有多个备份作业在跑那么链路是不正常的。需要重启带库，带库重启后再次扫描带机有可能恢复正常。或者取消所有备份作业，关闭磁带机将那块光纤卡无信号灯闪烁的驱动器拔出来重新插接进去也可能正常了。）不过在这里光纤卡和指示灯状态正常,链路也正常，那么在mounting磁带时出错，很有可能的就是磁带的槽位变了，我们可以通过update volume configuration更新卷配置让驱动臂找到正确的磁带位置来解决这个问题。

我们来验证一下

更新完OK，故障解决。备份恢复正常,两个驱动器都可以工作了.