Pytorch官方微调Mask-RCNN遇到的坑

官网地址

已经成功做好了这个实验，但是不得不吐槽一下官方的说明书，做实验的前提不交代好，尤其是这句话：

In references/detection/, we have a number of helper functions to simplify training and evaluating detection models. Here, we will use references/detection/engine.py, references/detection/utils.py and references/detection/transforms.py. Just copy them to your folder and use them here.

让人百思不得其解，references/detection/在哪？没有交代，这里附上链接：
https://github.com/pytorch/vision/tree/master/references/detection

做实验你会发现，光这3个文件还不够，因为这3个文件还依赖其他文件，依赖的文件其实就在/references/detection目录下，我们把coco_eval.py，coco_utils.py一并拷过来，好了，我们把这5个文件都拷到当前文件夹下，结构如下所示：

到了这一步，你以为一切都好了，运行后又出现bug，你会发现coco_eval.py和coco_utils.py都依赖于pycocotools，接下来就去安装pycocotools。
方法一：在window下使用如下命令，前提要安装了git

pip install git+https://github.com/philferriere/cocoapi.git#subdirectory=PythonAPI

方法二：https://github.com/philferriere/cocoapi下载源码，并进行解压。切换到anncoda的虚拟环境，并切换到 cocoapi\PythonAPI目录。运行以下指令：

# install pycocotools locally
python setup.py build_ext --inplace
 
# install pycocotools to the Python site-packages
python setup.py build_ext install

以上都在window环境下，本人用方法二解决了pycocotools的安装，在安装过程中需要VS环境，装一个就好了。

随便提一下，模型文件在程序里下载太慢，我们自己下载会快很多，链接：
https://download.pytorch.org/models/maskrcnn_resnet50_fpn_coco-bf2d0c1e.pth
加载模型有两种方法，第一、将其放到指定的缓存下；第二、自己来加载模型，代码如下：

    # load an instance segmentation model pre-trained pre-trained on COCO
    # no need to download the backbone if pretrained is set
    pretrained_backbone = False
    backbone = resnet_fpn_backbone('resnet50', pretrained_backbone)
    model = MaskRCNN(backbone, num_classes=91)
    state_dict = torch.load("X:/Downloads/maskrcnn_resnet50_fpn_coco-bf2d0c1e.pth")
    model.load_state_dict(state_dict)
    
    #用以上代码替换该行代码
    #model = torchvision.models.detection.maskrcnn_resnet50_fpn(pretrained=True)

最后运行程序时，又遇到一个bug：
TypeError: ‘numpy.float64’ object cannot be interpreted as an integer

主要因为numpy版本太高导致的，我当前是最新版本1.18.1，安装numpy==1.16.0即可解决这个问题。
pip install numpy==1.16.0

总共10个epoch，最后一个epoch结果如下：

Epoch: [9] [ 0/60] eta: 0:01:20 lr: 0.000005 loss: 0.1457 (0.1457) loss_classifier: 0.0221 (0.0221) loss_box_reg: 0.0074 (0.0074) loss_mask:
0.1059 (0.1059) loss_objectness: 0.0011 (0.0011) loss_rpn_box_reg: 0.0092 (0.0092) time: 1.3427 data: 0.7483 max mem: 3827
Epoch: [9] [10/60] eta: 0:00:36 lr: 0.000005 loss: 0.1534 (0.1581) loss_classifier: 0.0227 (0.0255) loss_box_reg: 0.0074 (0.0129) loss_mask:
0.1105 (0.1122) loss_objectness: 0.0005 (0.0006) loss_rpn_box_reg: 0.0077 (0.0068) time: 0.7343 data: 0.0691 max mem: 3827
Epoch: [9] [20/60] eta: 0:00:27 lr: 0.000005 loss: 0.1534 (0.1573) loss_classifier: 0.0227 (0.0249) loss_box_reg: 0.0075 (0.0119) loss_mask:
0.1105 (0.1125) loss_objectness: 0.0003 (0.0008) loss_rpn_box_reg: 0.0074 (0.0071) time: 0.6563 data: 0.0013 max mem: 3827
Epoch: [9] [30/60] eta: 0:00:20 lr: 0.000005 loss: 0.1487 (0.1530) loss_classifier: 0.0205 (0.0233) loss_box_reg: 0.0085 (0.0109) loss_mask:
0.1018 (0.1110) loss_objectness: 0.0003 (0.0008) loss_rpn_box_reg: 0.0051 (0.0071) time: 0.6521 data: 0.0014 max mem: 3827
Epoch: [9] [40/60] eta: 0:00:13 lr: 0.000005 loss: 0.1528 (0.1558) loss_classifier: 0.0205 (0.0237) loss_box_reg: 0.0073 (0.0113) loss_mask:
0.1077 (0.1129) loss_objectness: 0.0003 (0.0008) loss_rpn_box_reg: 0.0053 (0.0071) time: 0.6462 data: 0.0014 max mem: 3827
Epoch: [9] [50/60] eta: 0:00:06 lr: 0.000005 loss: 0.1555 (0.1604) loss_classifier: 0.0227 (0.0257) loss_box_reg: 0.0094 (0.0121) loss_mask:
0.1198 (0.1146) loss_objectness: 0.0004 (0.0007) loss_rpn_box_reg: 0.0067 (0.0073) time: 0.6517 data: 0.0013 max mem: 3827
Epoch: [9] [59/60] eta: 0:00:00 lr: 0.000005 loss: 0.1520 (0.1601) loss_classifier: 0.0227 (0.0253) loss_box_reg: 0.0098 (0.0120) loss_mask:
0.1155 (0.1149) loss_objectness: 0.0004 (0.0007) loss_rpn_box_reg: 0.0067 (0.0072) time: 0.6718 data: 0.0013 max mem: 3827
Epoch: [9] Total time: 0:00:40 (0.6712 s / it)
creating index…
index created!
Test: [ 0/50] eta: 0:00:43 model_time: 0.1556 (0.1556) evaluator_time: 0.0140 (0.0140) time: 0.8673 data: 0.6937 max mem: 3827
Test: [49/50] eta: 0:00:00 model_time: 0.1177 (0.1196) evaluator_time: 0.0040 (0.0049) time: 0.1286 data: 0.0007 max mem: 3827
Test: Total time: 0:00:07 (0.1441 s / it)
Averaged stats: model_time: 0.1177 (0.1196) evaluator_time: 0.0040 (0.0049)
Accumulating evaluation results…
DONE (t=0.01s).
Accumulating evaluation results…
DONE (t=0.01s).
IoU metric: bbox
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.806
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.994
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.964
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.766
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.811
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.342
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.839
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.839
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.787
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.843
IoU metric: segm
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.767
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.993
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.953
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.587
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.777
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.328
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.795
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.797
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.675
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.805
That’s it!