飞桨深度学习7日入门CV - Day05 - 初识PaddleHub

1.PaddleHub概述

2.PaddleHub使用指南

3.安装PaddleHub

# 安装PaddleHub
!pip install --upgrade paddlehub -i https://pypi.tuna.tsinghua.edu.cn/simple

'''
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting paddlehub
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/7f/9f/6617c2b8e9c5d847803ae89924b58bccd1b8fb2c98aa00e16531540591f2/paddlehub-1.6.0-py3-none-any.whl (206kB)
     |████████████████████████████████| 215kB 18.5MB/s eta 0:00:01
Requirement already satisfied,...
Requirement already satisfied,...
...
Installing collected packages: paddlehub
  Found existing installation: paddlehub 1.5.0
    Uninstalling paddlehub-1.5.0:
      Successfully uninstalled paddlehub-1.5.0
Successfully installed paddlehub-1.6.0
'''

4.案例分析

4.1 使用PaddleHub进行图像分类

图像分类是计算机视觉的重要领域，它的目标是将图像分类到预定义的标签。近期，许多研究者提出很多不同种类的神经网络，并且极大的提升了分类算法的性能。本文以Kaggle的猫狗分类数据集为例子，介绍如何使用PaddleHub进行图像分类。

# 通过关键字在服务端检索匹配的模型
!hub search resnet

下面是输出结果,有5个resnet模型

'''
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
2020-04-04 18:01:30,310-INFO: font search path ['/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/ttf', '/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/afm', '/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/pdfcorefonts']
2020-04-04 18:01:30,646-INFO: generated new fontManager

+--------------------------------+----------+----------+---------------------------+
|          ResourceName          |   Type   | Version  |          Summary          |
+--------------------------------+----------+----------+---------------------------+
|     resnet_v2_50_imagenet      |  Module  |  1.0.1   |ResNet50 is a image class  |
|                                |          |          |fication model trained wi  |
|                                |          |          |th ImageNet-2012 dataset.  |
|                                |          |          |                           |
+--------------------------------+----------+----------+---------------------------+
|     resnet_v2_34_imagenet      |  Module  |  1.0.0   |ResNet34 is a image class  |
|                                |          |          |fication model trained wi  |
|                                |          |          |th ImageNet-2012 dataset.  |
|                                |          |          |                           |
+--------------------------------+----------+----------+---------------------------+
|     resnet_v2_18_imagenet      |  Module  |  1.0.0   |ResNet18 is a image class  |
|                                |          |          |fication model trained wi  |
|                                |          |          |th ImageNet-2012 dataset.  |
|                                |          |          |                           |
+--------------------------------+----------+----------+---------------------------+
|     resnet_v2_152_imagenet     |  Module  |  1.0.1   |ResNet152 is a image clas  |
|                                |          |          |sfication model trained w  |
|                                |          |          |ith ImageNet-2012 dataset  |
|                                |          |          |.                          |
+--------------------------------+----------+----------+---------------------------+
|     resnet_v2_101_imagenet     |  Module  |  1.0.1   |ResNet101 is a image clas  |
|                                |          |          |sfication model trained w  |
|                                |          |          |ith ImageNet-2012 dataset  |
|                                |          |          |.                          |
+--------------------------------+----------+----------+---------------------------+
'''

Step1. 加载预训练模型

由于猫狗分类是一个图像分类任务，因此我们使用经典的ResNet-50作为预训练模型。PaddleHub提供了丰富的图像分类预训练模型，包括了最新的神经网络架构搜索类的PNASNet，我们推荐您尝试不同的预训练模型来获得更好的性能。

PaddleHub 还有着许多的图像分类预训练模型，更多信息参见PaddleHub官方网站

加载module，并获取module的上下文环境，包括输入和输出的变量，以及Paddle Program；

import paddlehub as hub

module = hub.Module(name="resnet_v2_50_imagenet")

input_dict, output_dict, program = module.context(trainable=True)

'''
[2020-04-04 18:01:32,986] [    INFO] - Installing resnet_v2_50_imagenet module
Downloading resnet_v2_50_imagenet
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmppobhgwyu/resnet_v2_50_imagenet
[==================================================] 100.00%
[2020-04-04 18:01:44,252] [    INFO] - Successfully installed resnet_v2_50_imagenet-1.0.1
[2020-04-04 18:01:44,580] [    INFO] - 267 pretrained paramaters loaded by PaddleHub
'''

Step2. 数据准备

接着需要加载图片数据集。为了快速体验，我们直接加载paddlehub提供的猫狗分类数据集，如果想要使用自定义的数据进行体验，请查看适配自定义数据

如果想加载自定义数据集完成迁移学习，详细参见自定义数据集

接着生成一个图像分类的reader，reader负责将dataset的数据进行预处理，接着以特定格式组织并输入给模型进行训练。

当我们生成一个图像分类的reader时，需要指定输入图片的大小

# 直接用PaddleHub提供的数据集
dataset = hub.dataset.DogCat()

data_reader = hub.reader.ImageClassificationReader(
    image_width=module.get_expected_image_width(),
    image_height=module.get_expected_image_height(),
    images_mean=module.get_pretrained_images_mean(),
    images_std=module.get_pretrained_images_std(),
    dataset=dataset)
'''
Downloading dog-cat.tar.gz
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/dataset/dog-cat.tar.gz
[==================================================] 100.00%
[2020-04-04 18:02:34,783] [    INFO] - Dataset label map = {'cat': 0, 'dog': 1}
'''

Step3. 配置策略

在进行Finetune前，我们可以设置一些运行时的配置，例如如下代码中的配置，表示：

use_cuda：设置为False表示使用CPU进行训练。如果您本机支持GPU，且安装的是GPU版本的PaddlePaddle，我们建议您将这个选项设置为True；

epoch：要求Finetune的任务只遍历1次训练集；

batch_size：每次训练的时候，给模型输入的每批数据大小为32，模型训练时能够并行处理批数据，因此batch_size越大，训练的效率越高，但是同时带来了内存的负荷，过大的batch_size可能导致内存不足而无法训练，因此选择一个合适的batch_size是很重要的一步；

log_interval：每隔10 step打印一次训练日志；

eval_interval：每隔50 step在验证集上进行一次性能评估；

checkpoint_dir：将训练的参数和数据保存到cv_finetune_turtorial_demo目录中；

strategy：使用DefaultFinetuneStrategy策略进行finetune；

更多运行配置，请查看RunConfig

同时PaddleHub提供了许多优化策略，如AdamWeightDecayStrategy、ULMFiTStrategy、DefaultFinetuneStrategy等，详细信息参见策略

config = hub.RunConfig(
    use_cuda=False,                              #是否使用GPU训练，默认为False；
    num_epoch=1,                                #Fine-tune的轮数；
    checkpoint_dir="cv_finetune_turtorial_demo",#模型checkpoint保存路径, 若用户没有指定，程序会自动生成；
    batch_size=32,                              #训练的批大小，如果使用GPU，请根据实际情况调整batch_size；
    eval_interval=50,                           #模型评估的间隔，默认每100个step评估一次验证集；
    strategy=hub.finetune.strategy.DefaultFinetuneStrategy())  #Fine-tune优化策略；
'''
[2020-04-10 00:42:16,151] [    INFO] - Checkpoint dir: cv_finetune_turtorial_demo
'''

Step4. 迁移组网

有了合适的预训练模型和准备要迁移的数据集后，我们开始组建一个Task。

由于猫狗分类是一个二分类的任务，而我们下载的分类module是在ImageNet数据集上训练的千分类模型，所以我们需要对模型进行简单的微调，把模型改造为一个二分类模型：

1. 从输出变量中找到特征图提取层feature_map；
2. 在feature_map后面接入一个全连接层，生成Task；

feature_map = output_dict["feature_map"]

feed_list = [input_dict["image"].name]

#ImageClassifierTask：通用的分类任务Task，该Task基于输入的特征，添加一个或多个全连接层来创建一个分类任务用于Fine-tune
task = hub.ImageClassifierTask(
    data_reader=data_reader, #提供数据的reader
    feed_list=feed_list,     #待feed变量的名字列表
    feature=feature_map,     #输入的特征矩阵
    num_classes=dataset.num_labels, #分类任务的类别数量
    config=config)                  #运行配置

Step5. Fine-tune

run_states = task.finetune_and_eval()

'''
2020-04-10 00:42:30,202-WARNING: paddle.fluid.layers.py_reader() may be deprecated in the near future. Please use paddle.fluid.io.DataLoader.from_generator() instead.
[2020-04-10 00:42:30,390] [    INFO] - Strategy with scheduler: {'warmup': 0.0, 'linear_decay': {'start_point': 1.0, 'end_learning_rate': 0.0}, 'noam_decay': False, 'discriminative': {'blocks': 0, 'factor': 2.6}, 'gradual_unfreeze': 0, 'slanted_triangle': {'cut_fraction': 0.0, 'ratio': 32}}, regularization: {'L2': 0.001, 'L2SP': 0.0, 'weight_decay': 0.0} and clip: {'GlobalNorm': 0.0, 'Norm': 0.0}
[2020-04-10 00:42:36,038] [    INFO] - Try loading checkpoint from cv_finetune_turtorial_demo/ckpt.meta
[2020-04-10 00:42:36,039] [    INFO] - PaddleHub model checkpoint not found, start from scratch...
[2020-04-10 00:42:36,211] [    INFO] - PaddleHub finetune start
[2020-04-10 00:45:19,782] [   TRAIN] - step 10 / 281: loss=0.24262 acc=0.89687 [step/sec: 0.06]
[2020-04-10 00:47:48,724] [   TRAIN] - step 20 / 281: loss=0.05443 acc=0.97813 [step/sec: 0.07]
......
[2020-04-10 02:02:29,740] [   TRAIN] - step 270 / 281: loss=0.05036 acc=0.97500 [step/sec: 0.07]
[2020-04-10 02:04:55,704] [   TRAIN] - step 280 / 281: loss=0.03833 acc=0.98438 [step/sec: 0.07]
[2020-04-10 02:05:23,759] [    INFO] - Evaluation on dev dataset start
[2020-04-10 02:08:00,578] [    EVAL] - [dev dataset evaluation result] loss=0.02530 acc=0.98994 [step/sec: 0.22]
[2020-04-10 02:08:00,580] [    INFO] - Load the best model from cv_finetune_turtorial_demo/best_model
[2020-04-10 02:08:01,574] [    INFO] - Evaluation on test dataset start
[2020-04-10 02:10:38,572] [    EVAL] - [test dataset evaluation result] loss=0.03652 acc=0.98973 [step/sec: 0.22]
[2020-04-10 02:10:38,573] [    INFO] - Saving model checkpoint to cv_finetune_turtorial_demo/step_282
[2020-04-10 02:10:39,486] [    INFO] - PaddleHub finetune finished.
'''

Step6. 预测

当Finetune完成后，我们使用模型来进行预测，先通过以下命令来获取测试的图片

注意：以下示例仍然以猫狗分类为例子，其他数据集所用的测试图片请自行准备

下载图片

!wget --no-check-certificate https://paddlehub.bj.bcebos.com/resources/test_img_cat.jpg
!wget --no-check-certificate https://paddlehub.bj.bcebos.com/resources/test_img_dog.jpg

import numpy as np

data = ["test_img_dog.jpg"]
label_map = dataset.label_dict()

index = 0
run_states = task.predict(data=data)
results = [run_state.run_results for run_state in run_states]

for batch_result in results:
    print(batch_result)
    batch_result = np.argmax(batch_result, axis=2)[0]
    print(batch_result)
    for result in batch_result:
        index += 1
        result = label_map[result]
        print("input %i is %s, and the predict result is %s" %
              (index, data[index - 1], result))
'''
[2020-04-10 02:13:39,698] [    INFO] - The best model has been loaded
[2020-04-10 02:13:39,700] [    INFO] - PaddleHub predict start
share_vars_from is set, scope is ignored.
[2020-04-10 02:13:40,042] [    INFO] - PaddleHub predict finished.
[array([[4.5764602e-05, 9.9995422e-01]], dtype=float32)]
[1]
input 1 is test_img_dog.jpg, and the predict result is dog
'''

4.2 使用PaddleHub进行口罩检测

Step1. 定义待预测数据

解压数据

#!unzip /home/aistudio/data/data25505/detection.zip

以数据集文件夹下test_mask_detection.png为待预测图片

import matplotlib.pyplot as plt 
import matplotlib.image as mpimg 

# 待预测图片
test_img_path = ["/home/aistudio/detection/test_mask_detection.jpg"]
img = mpimg.imread(test_img_path[0]) 

# 展示待预测图片
plt.figure(figsize=(10,10))
plt.imshow(img) 
plt.axis('off') 
plt.show()

若是待预测图片存放在一个文件中，如test.txt。每一行是待预测图片的存放路径。

!cat /home/aistudio/detection/test.txt
'''
/home/aistudio/detection/test_mask_detection.jpg
'''

with open('detection/test.txt', 'r') as f:
    test_img_path=[]
    for line in f:
        test_img_path.append(line.strip())
print(test_img_path)
'''
['/home/aistudio/detection/test_mask_detection.jpg']
'''

用户想要利用完成对该文件的口罩检测，只需读入该文件，将文件内容存成list，list中每个元素是待预测图片的存放路径。

Step2. 加载预训练模型

PaddleHub口罩检测提供了两种预训练模型，pyramidbox_lite_mobile_mask和pyramidbox_lite_server_mask。不同点在于，pyramidbox_lite_mobile_mask是针对于移动端优化过的模型，适合部署于移动端或者边缘检测等算力受限的设备上。

import paddlehub as hub

module = hub.Module(name="pyramidbox_lite_mobile_mask")
# module = hub.Module(name="pyramidbox_lite_server_mask")
'''
[2020-03-20 19:17:18,815] [    INFO] - Installing pyramidbox_lite_mobile_mask module
Downloading pyramidbox_lite_mobile_mask
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmpejfqd2s1/pyramidbox_lite_mobile_mask
[==================================================] 100.00%
[2020-03-20 19:17:20,947] [    INFO] - Successfully installed pyramidbox_lite_mobile_mask-1.2.0
[2020-03-20 19:17:21,054] [    INFO] - Installing pyramidbox_lite_mobile module
Downloading pyramidbox_lite_mobile
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmpw76_shxb/pyramidbox_lite_mobile
[==================================================] 100.00%
[2020-03-20 19:17:21,431] [    INFO] - Successfully installed pyramidbox_lite_mobile-1.1.1
'''

Step3. 预测

PaddleHub对于支持一键预测的module，可以调用module的相应预测API，完成预测功能。

input_dict = {"image": test_img_path}

# 口罩检测预测
results = module.face_detection(data=input_dict)
for result in results:
    print(result)

# 预测结果展示
img = mpimg.imread("detection_result/test_mask_detection.jpg")
plt.figure(figsize=(10,10))
plt.imshow(img) 
plt.axis('off') 
plt.show()

'''
[2020-03-20 19:17:25,402] [    INFO] - 133 pretrained paramaters loaded by PaddleHub
[2020-03-20 19:17:25,469] [    INFO] - 149 pretrained paramaters loaded by PaddleHub
image with bbox drawed saved as /home/aistudio/detection_result/test_mask_detection.jpg
{'data': {'label': 'MASK', 'left': 459.0687048435211, 'right': 657.8806104660034, 'top': 191.39037430286407, 'bottom': 444.0621703863144, 'confidence': 0.99913305}, 'id': 1, 'path': '/home/aistudio/detection/test_mask_detection.jpg'}
{'data': {'label': 'NO MASK', 'left': 1166.9745614528656, 'right': 1323.218990802765, 'top': 294.5009618997574, 'bottom': 500.64271688461304, 'confidence': 0.99955636}, 'id': 1, 'path': '/home/aistudio/detection/test_mask_detection.jpg'}
{'data': {'label': 'MASK', 'left': 942.4126571416855, 'right': 1127.8870369195938, 'top': 340.60501903295517, 'bottom': 585.9683722257614, 'confidence': 0.94331}, 'id': 1, 'path': '/home/aistudio/detection/test_mask_detection.jpg'}
{'data': {'label': 'NO MASK', 'left': 1237.7968076467514, 'right': 1443.2840930223465, 'top': 815.3819382190704, 'bottom': 1055.2424848079681, 'confidence': 0.54464865}, 'id': 1, 'path': '/home/aistudio/detection/test_mask_detection.jpg'}
'''

其中:
label有’MASK’和’NO MASK’两种选择：'MASK’表示戴了口罩，'NO MASK表示没有佩戴口罩。

’left’/‘rigth’/‘top’/'bottom’表示口罩在图片当中的位置。

‘confidence’表示预测为佩戴口罩’MASK’ 或者 不佩戴口罩’NO MASK’的概率大小。

5.结语 - Pytorch的迁移学习

虽然我还是习惯Pytorch

使用Pytorch中torchvision.models.resnet50。由于ImageNet是识别1000个物体，猫狗分类的类别数为2，所以需要对模型的最后一层全连接层进行微调，将输出从1000改为2。

import torch
import torchvision
from torchvision import models
#准备模型
model = models.resnet50(pretrained=True) #可用dir(model)查看属性及方法

#将所有参数冻结
for param in model.parameters(): 
    param.requires_grad = False
print(model.fc)

#修改fc层。可用model.named_parameters()迭代查看具体名称和参数
num_feature = model.fc.in_features  #获取fc层的输入个数
model.fc = nn.Linear(num_feature, 2)  #重新定义fc层
print(model.fc)
#print(model)

#将model移至GPU
model.to(DEVICE)

代码来自原博客
这篇博客也不错