谷歌云服务器CoLab跑Faster_RCNN记录
本文参考：使用colab训练faster-rcnn_LCCFlccf的博客-CSDN博客
首先下载配置谷歌云盘下载配置不做介绍。
1.配置环境
!apt-get install -y -qq software-properties-common python-software-properties module-init-tools
!add-apt-repository -y ppa:alessandro-strada/ppa 2>&1 > /dev/null
!apt-get update -qq 2>&1 > /dev/null
!apt-get -y install -qq google-drive-ocamlfuse fuse
from google.colab import auth
auth.authenticate_user()
from oauth2client.client import GoogleCredentials
creds = GoogleCredentials.get_application_default()
import getpass
!google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} < /dev/null 2>&1 | grep URL
vcode = getpass.getpass()
!echo {vcode} | google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret}
2.克隆Faster_RCNN源代码
!git clone -b pytorch-1.0 https://github.com/jwyang/faster-rcnn.pytorch.git 
3.进入Faster_ RCNN文件夹
import os
os.chdir('faster-rcnn.pytorch')
!ls4.下载预训练权重
!mkdir data
os.chdir('data')
!mkdir pretrained_model
os.chdir('pretrained_model')
# 下载预训练模型res101
!wget https://filebox.ece.vt.edu/~jw2yang/faster-rcnn/pretrained-base-models/resnet101_caffe.pth
# 下载预训练模型vgg16
!wget https://filebox.ece.vt.edu/~jw2yang/faster-rcnn/pretrained-base-models/vgg16_caffe.pth5.下载数据集
os.chdir('../') #返回上一级目录即data/下
# 下载数据集
!wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtrainval_06-Nov-2007.tar
!wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtest_06-Nov-2007.tar
!wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCdevkit_08-Jun-2007.tar
# 解压缩
!tar xvf VOCtrainval_06-Nov-2007.tar
!tar xvf VOCtest_06-Nov-2007.tar
!tar xvf VOCdevkit_08-Jun-2007.tar
# 建立软连接
!ln -s $VOCdevkit VOCdevkit2007 #注意！如果上面解压缩得到的文件夹名字为"VOCdevdit"，要将其改为“VOCdevdit2007"，否则后面会报错。6.删除faster-rcnn.pytorch下的VOCdevkit2007，并修改文件夹VOCdevkit为VOCdevkit2007
!rm -rf VOCdevkit2007在进行下一步代码编译之前由于CUDA版本预GCC不对应的问题会出现如下错误：
error: command '/usr/local/cuda/bin/nvcc' failed with exit status 1解决方法运行下面代码：
!pip install -U torch==1.4 torchvision==0.5 -f https://download.pytorch.org/whl/cu101/torch_stable.html7.回到上一级路径，然后进入lib中进行编译
os.chdir('../lib')
!python setup.py build develop8.编译成功后按照下图训练参数进行训练：
输入如下代码会出现报错：
解决：
!pip install scipy==1.2.1参考训练代码：
!CUDA_VISIBLE_DEVICES=0 python trainval_net.py \
--dataset pascal_voc \
--net res101 \
--bs 4 \
--nw 0 \
--lr 0.004 \
--lr_decay_step 8 \
--epochs 10 \
--cuda训练中的输出：
Called with args:
Namespace(batch_size=4, checkepoch=1, checkpoint=0, checkpoint_interval=10000, checksession=1, class_agnostic=False, cuda=True, dataset='pascal_voc', disp_interval=100, large_scale=False, lr=0.004, lr_decay_gamma=0.1, lr_decay_step=8, mGPUs=False, max_epochs=10, net='res101', num_workers=0, optimizer='sgd', resume=False, save_dir='models', session=1, start_epoch=1, use_tfboard=False)
Using config:
{'ANCHOR_RATIOS': [0.5, 1, 2],
'ANCHOR_SCALES': [8, 16, 32],
'CROP_RESIZE_WITH_MAX_POOL': False,
'CUDA': False,
'DATA_DIR': '/content/faster-rcnn.pytorch/data',
'DEDUP_BOXES': 0.0625,
'EPS': 1e-14,
'EXP_DIR': 'res101',
'FEAT_STRIDE': [16],
'GPU_ID': 0,
'MATLAB': 'matlab',
'MAX_NUM_GT_BOXES': 20,
'MOBILENET': {'DEPTH_MULTIPLIER': 1.0,
'FIXED_LAYERS': 5,
'REGU_DEPTH': False,
'WEIGHT_DECAY': 4e-05},
'PIXEL_MEANS': array([[[102.9801, 115.9465, 122.7717]]]),
'POOLING_MODE': 'align',
'POOLING_SIZE': 7,
'RESNET': {'FIXED_BLOCKS': 1, 'MAX_POOL': False},
'RNG_SEED': 3,
'ROOT_DIR': '/content/faster-rcnn.pytorch',
'TEST': {'BBOX_REG': True,
'HAS_RPN': True,
'MAX_SIZE': 1000,
'MODE': 'nms',
'NMS': 0.3,
'PROPOSAL_METHOD': 'gt',
'RPN_MIN_SIZE': 16,
'RPN_NMS_THRESH': 0.7,
'RPN_POST_NMS_TOP_N': 300,
'RPN_PRE_NMS_TOP_N': 6000,
'RPN_TOP_N': 5000,
'SCALES': [600],
'SVM': False},
'TRAIN': {'ASPECT_GROUPING': False,
'BATCH_SIZE': 128,
'BBOX_INSIDE_WEIGHTS': [1.0, 1.0, 1.0, 1.0],
'BBOX_NORMALIZE_MEANS': [0.0, 0.0, 0.0, 0.0],
'BBOX_NORMALIZE_STDS': [0.1, 0.1, 0.2, 0.2],
'BBOX_NORMALIZE_TARGETS': True,
'BBOX_NORMALIZE_TARGETS_PRECOMPUTED': True,
'BBOX_REG': True,
'BBOX_THRESH': 0.5,
'BG_THRESH_HI': 0.5,
'BG_THRESH_LO': 0.0,
'BIAS_DECAY': False,
'BN_TRAIN': False,
'DISPLAY': 20,
'DOUBLE_BIAS': False,
'FG_FRACTION': 0.25,
'FG_THRESH': 0.5,
'GAMMA': 0.1,
'HAS_RPN': True,
'IMS_PER_BATCH': 1,
'LEARNING_RATE': 0.001,
'MAX_SIZE': 1000,
'MOMENTUM': 0.9,
'PROPOSAL_METHOD': 'gt',
'RPN_BATCHSIZE': 256,
'RPN_BBOX_INSIDE_WEIGHTS': [1.0, 1.0, 1.0, 1.0],
'RPN_CLOBBER_POSITIVES': False,
'RPN_FG_FRACTION': 0.5,
'RPN_MIN_SIZE': 8,
'RPN_NEGATIVE_OVERLAP': 0.3,
'RPN_NMS_THRESH': 0.7,
'RPN_POSITIVE_OVERLAP': 0.7,
'RPN_POSITIVE_WEIGHT': -1.0,
'RPN_POST_NMS_TOP_N': 2000,
'RPN_PRE_NMS_TOP_N': 12000,
'SCALES': [600],
'SNAPSHOT_ITERS': 5000,
'SNAPSHOT_KEPT': 3,
'SNAPSHOT_PREFIX': 'res101_faster_rcnn',
'STEPSIZE': [30000],
'SUMMARY_INTERVAL': 180,
'TRIM_HEIGHT': 600,
'TRIM_WIDTH': 600,
'TRUNCATED': False,
'USE_ALL_GT': True,
'USE_FLIPPED': True,
'USE_GT': False,
'WEIGHT_DECAY': 0.0001},
'USE_GPU_NMS': True}
Loaded dataset `voc_2007_trainval` for training
Set proposal method: gt
Appending horizontally-flipped training examples...
wrote gt roidb to /content/faster-rcnn.pytorch/data/cache/voc_2007_trainval_gt_roidb.pkl
done
Preparing training data...
done
before filtering, there are 10022 images...
after filtering, there are 10022 images...
10022 roidb entries
Loading pretrained weights from data/pretrained_model/resnet101_caffe.pth
[session 1][epoch 1][iter 0/2505] loss: 4.3865, lr: 4.00e-03
fg/bg=(85/427), time cost: 0.601814
rpn_cls: 0.6829, rpn_box: 0.2242, rcnn_cls: 3.0302, rcnn_box 0.4492
[session 1][epoch 1][iter 100/2505] loss: 1.5043, lr: 4.00e-03
fg/bg=(117/395), time cost: 77.200093
rpn_cls: 0.1848, rpn_box: 0.0734, rcnn_cls: 0.6639, rcnn_box 0.5441
[session 1][epoch 1][iter 200/2505] loss: 1.2434, lr: 4.00e-03
fg/bg=(118/394), time cost: 81.200940
rpn_cls: 0.1488, rpn_box: 0.1067, rcnn_cls: 0.6809, rcnn_box 0.5596
[session 1][epoch 1][iter 300/2505] loss: 1.1935, lr: 4.00e-03
fg/bg=(107/405), time cost: 80.614398
rpn_cls: 0.2032, rpn_box: 0.0593, rcnn_cls: 0.5152, rcnn_box 0.5732
[session 1][epoch 1][iter 400/2505] loss: 1.1234, lr: 4.00e-03
fg/bg=(112/400), time cost: 80.967055
rpn_cls: 0.2763, rpn_box: 0.1483, rcnn_cls: 0.5449, rcnn_box 0.4684
[session 1][epoch 1][iter 500/2505] loss: 1.1293, lr: 4.00e-03
fg/bg=(89/423), time cost: 80.701656
rpn_cls: 0.0986, rpn_box: 0.1710, rcnn_cls: 0.4989, rcnn_box 0.4123
[session 1][epoch 1][iter 600/2505] loss: 1.0828, lr: 4.00e-03
fg/bg=(59/453), time cost: 81.680025
rpn_cls: 0.1715, rpn_box: 0.0200, rcnn_cls: 0.1562, rcnn_box 0.2416
[session 1][epoch 1][iter 700/2505] loss: 1.0852, lr: 4.00e-03
fg/bg=(91/421), time cost: 80.847974
rpn_cls: 0.2103, rpn_box: 0.1290, rcnn_cls: 0.4282, rcnn_box 0.3773
[session 1][epoch 1][iter 800/2505] loss: 1.0542, lr: 4.00e-03
fg/bg=(102/410), time cost: 80.808020
rpn_cls: 0.1616, rpn_box: 0.1210, rcnn_cls: 0.3271, rcnn_box 0.3437
[session 1][epoch 1][iter 900/2505] loss: 1.0191, lr: 4.00e-03
fg/bg=(128/384), time cost: 80.726070
rpn_cls: 0.1724, rpn_box: 0.0435, rcnn_cls: 0.3047, rcnn_box 0.4088
[session 1][epoch 1][iter 1000/2505] loss: 0.9602, lr: 4.00e-03
fg/bg=(99/413), time cost: 80.964316
rpn_cls: 0.1359, rpn_box: 0.1641, rcnn_cls: 0.2922, rcnn_box 0.4030
[session 1][epoch 1][iter 1100/2505] loss: 1.0293, lr: 4.00e-03
fg/bg=(128/384), time cost: 81.418617
rpn_cls: 0.1714, rpn_box: 0.0918, rcnn_cls: 0.5062, rcnn_box 0.5381
[session 1][epoch 1][iter 1200/2505] loss: 0.9298, lr: 4.00e-03
fg/bg=(128/384), time cost: 80.932757
rpn_cls: 0.1507, rpn_box: 0.0604, rcnn_cls: 0.4468, rcnn_box 0.5164
[session 1][epoch 1][iter 1300/2505] loss: 0.8705, lr: 4.00e-03
fg/bg=(110/402), time cost: 81.297547
rpn_cls: 0.0586, rpn_box: 0.0663, rcnn_cls: 0.3055, rcnn_box 0.3561
[session 1][epoch 1][iter 1400/2505] loss: 0.9164, lr: 4.00e-03
fg/bg=(125/387), time cost: 81.203549
rpn_cls: 0.1953, rpn_box: 0.1411, rcnn_cls: 0.3142, rcnn_box 0.4131
[session 1][epoch 1][iter 1500/2505] loss: 0.8912, lr: 4.00e-03
fg/bg=(92/420), time cost: 80.496142
rpn_cls: 0.1335, rpn_box: 0.1330, rcnn_cls: 0.1567, rcnn_box 0.2395
[session 1][epoch 1][iter 1600/2505] loss: 0.8711, lr: 4.00e-03
fg/bg=(119/393), time cost: 80.958639
rpn_cls: 0.0733, rpn_box: 0.0217, rcnn_cls: 0.1653, rcnn_box 0.3126
[session 1][epoch 1][iter 1700/2505] loss: 0.8627, lr: 4.00e-03
fg/bg=(109/403), time cost: 80.712723
rpn_cls: 0.2050, rpn_box: 0.0961, rcnn_cls: 0.1754, rcnn_box 0.2821
[session 1][epoch 1][iter 1800/2505] loss: 0.8574, lr: 4.00e-03
fg/bg=(70/442), time cost: 80.907473
rpn_cls: 0.1356, rpn_box: 0.0348, rcnn_cls: 0.2234, rcnn_box 0.2077
[session 1][epoch 1][iter 1900/2505] loss: 0.8235, lr: 4.00e-03
fg/bg=(128/384), time cost: 80.964377
rpn_cls: 0.1156, rpn_box: 0.0996, rcnn_cls: 0.2020, rcnn_box 0.3494
[session 1][epoch 1][iter 2000/2505] loss: 0.8421, lr: 4.00e-03
fg/bg=(91/421), time cost: 81.235305
rpn_cls: 0.1262, rpn_box: 0.1005, rcnn_cls: 0.2875, rcnn_box 0.2914
[session 1][epoch 1][iter 2100/2505] loss: 0.8335, lr: 4.00e-03
fg/bg=(108/404), time cost: 80.114589
rpn_cls: 0.2443, rpn_box: 0.2750, rcnn_cls: 0.2798, rcnn_box 0.3018
[session 1][epoch 1][iter 2200/2505] loss: 0.7941, lr: 4.00e-03
fg/bg=(116/396), time cost: 80.939714
rpn_cls: 0.0630, rpn_box: 0.0444, rcnn_cls: 0.2918, rcnn_box 0.4340
[session 1][epoch 1][iter 2300/2505] loss: 0.7769, lr: 4.00e-03
fg/bg=(93/419), time cost: 79.992247
rpn_cls: 0.0638, rpn_box: 0.0387, rcnn_cls: 0.2485, rcnn_box 0.2362
[session 1][epoch 1][iter 2400/2505] loss: 0.7878, lr: 4.00e-03
fg/bg=(122/390), time cost: 80.548330
rpn_cls: 0.2374, rpn_box: 0.0779, rcnn_cls: 0.3044, rcnn_box 0.3277
[session 1][epoch 1][iter 2500/2505] loss: 0.7910, lr: 4.00e-03
fg/bg=(112/400), time cost: 80.743351
rpn_cls: 0.1638, rpn_box: 0.0516, rcnn_cls: 0.3541, rcnn_box 0.3596
save model: models/res101/pascal_voc/faster_rcnn_1_1_2504.pth
[session 1][epoch 2][iter 0/2505] loss: 0.8945, lr: 4.00e-03
fg/bg=(100/412), time cost: 0.732405
rpn_cls: 0.1068, rpn_box: 0.0797, rcnn_cls: 0.3975, rcnn_box 0.3106
[session 1][epoch 2][iter 100/2505] loss: 0.7174, lr: 4.00e-03
fg/bg=(115/397), time cost: 80.344738
rpn_cls: 0.2014, rpn_box: 0.0987, rcnn_cls: 0.3569, rcnn_box 0.3227
[session 1][epoch 2][iter 200/2505] loss: 0.7247, lr: 4.00e-03
fg/bg=(106/406), time cost: 80.984611
rpn_cls: 0.0716, rpn_box: 0.0760, rcnn_cls: 0.1755, rcnn_box 0.2736
[session 1][epoch 2][iter 300/2505] loss: 0.7214, lr: 4.00e-03
fg/bg=(124/388), time cost: 80.552769
rpn_cls: 0.0700, rpn_box: 0.0674, rcnn_cls: 0.2651, rcnn_box 0.3382
[session 1][epoch 2][iter 400/2505] loss: 0.7070, lr: 4.00e-03
fg/bg=(124/388), time cost: 80.854144
rpn_cls: 0.0780, rpn_box: 0.0328, rcnn_cls: 0.1831, rcnn_box 0.3122
[session 1][epoch 2][iter 500/2505] loss: 0.7517, lr: 4.00e-03
fg/bg=(108/404), time cost: 80.362625
rpn_cls: 0.2125, rpn_box: 0.1714, rcnn_cls: 0.3632, rcnn_box 0.3760
[session 1][epoch 2][iter 600/2505] loss: 0.6642, lr: 4.00e-03
fg/bg=(92/420), time cost: 79.861190
rpn_cls: 0.0827, rpn_box: 0.0543, rcnn_cls: 0.1484, rcnn_box 0.1583
[session 1][epoch 2][iter 700/2505] loss: 0.6570, lr: 4.00e-03
fg/bg=(101/411), time cost: 81.287454
rpn_cls: 0.0549, rpn_box: 0.0389, rcnn_cls: 0.2253, rcnn_box 0.2616