基于 Inception V3 迁移学习与微调的猫狗识别分类

——Tesla M40 + singularity + keras2 + jupyter notebook

---

• 硬件：Nvidia Tesla M40 显存24GB
• 软件环境：CentOS 7 + singularity + jupyter notebook
• 深度学习框架：keras2 ( 底层 tensorflow-gpu 1.2.0 )
• 语言环境：python 3.5

---

本案例也可以用 Ubuntu + keras2 或 Ubuntu + docker + keras2 实现，当然最好使用 GPU 跑， CPU 太慢。因此对环境的要求就是只要能跑起来 keras 调用 GPU 就 OK 了。

迁移学习的定义：在 ImageNet 已经得到一个预训练好的 ConvNet 网络，删除网络的最后一个全连接层，然后将 ConvNet 网络的剩余部分作为新数据集的特征提取层。一旦你提取了所有图像的特征，就可以开始训练新数据集分类器。
微调：更换并重新训练 ConvNet 的网络层，还可以通过反向传播算法对预训练网络的权重进行微调。

在 jupyter notebook 命令行下，首先引入

%matplotlib inline
import os
import sys
import glob
# import argparse  #这个模块是命令行参数传入，在nb中不需要
import matplotlib.pyplot as plt

可以使用 jupyter notebook，也可以直接在编辑器(sublime text , Atom , pycharm 等)中编辑然后在终端用命令行运行。
本文用 notebook，如果想在终端来运行，就调用 argparse 模块，运行时给主函数传入参数。

引入一些必要的模块

from keras import __version__
from keras.applications.inception_v3 import InceptionV3, preprocess_input
from keras.models import Model
from keras.layers import Dense, GlobalAveragePooling2D
from keras.preprocessing.image import ImageDataGenerator
from keras.optimizers import SGD

引入 Inception V3 的 模型，第一次使用时，首先下载，大约88MB，会保存在 ~/.keras/models 下，以后再用就不用下载。

定义一些全局变量，这些全局变量是可以通过 argparse 模块从命令行获取，传递给主函数。在notebook中，调用主函数的时候直接传递给主函数。注意 keras2 中已经将 nb_epochs 修改为 epochs 了。

定义全连接层数为 FC_SIZE 变量（迁移学习需要传递的参数），定义冻结层数为 NB_IV3_LAYERS_TO_FREEZE 变量（微调需要传递的参数）。

IM_WIDTH, IM_HEIGHT = 299, 299    #修正 InceptionV3 的尺寸参数
EPOCHS = 10
BAT_SIZE = 40
FC_SIZE = 1024
NB_IV3_LAYERS_TO_FREEZE = 172

# 定义一个方法——获取训练集和验证集中的样本数量，即nb_train_samples,nb_val_samples

def get_nb_files(directory):
    """Get number of files by searching directory recursively"""
    if not os.path.exists(directory):
        return 0
    cnt = 0
    for r, dirs, files in os.walk(directory):
        for dr in dirs:
            cnt += len(glob.glob(os.path.join(r, dr + "/*")))       # glob模块是用来查找匹配文件的，后面接匹配规则。
    return cnt

定义迁移学习函数，冻结所有的 base_model 层，不训练。

# 定义迁移学习的函数，不需要训练的部分。

def setup_to_transfer_learn(model, base_model):
    """Freeze all layers and compile the model"""
    for layer in base_model.layers:
        layer.trainable = False
    model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])

定义增加最后一个全连接层的函数，1024层

# 定义增加最后一个全连接层的函数

def add_new_last_layer(base_model, nb_classes):
    """Add last layer to the convnet

    Args:
        base_model: keras model excluding top
        nb_classes: # of classes

    Returns:
        new keras model with last layer
    """
    x = base_model.output
    x = GlobalAveragePooling2D()(x)
    x = Dense(FC_SIZE, activation='relu')(x) #new FC layer, random init
    predictions = Dense(nb_classes, activation='softmax')(x) #new softmax layer
    model = Model(inputs=base_model.input, outputs=predictions)
    return model

定义微调函数，冻结172层之前的层

# 定义微调函数

def setup_to_finetune(model):
    """Freeze the bottom NB_IV3_LAYERS and retrain the remaining top layers.

        note: NB_IV3_LAYERS corresponds to the top 2 inception blocks in the inceptionv3 arch

    Args:
        model: keras model
    """
    for layer in model.layers[:NB_IV3_LAYERS_TO_FREEZE]:
        layer.trainable = False
    for layer in model.layers[NB_IV3_LAYERS_TO_FREEZE:]:
        layer.trainable = True
    model.compile(optimizer=SGD(lr=0.0001, momentum=0.9), loss='categorical_crossentropy', metrics=['accuracy'])

训练结束后画 acc-loss 图，查看训练效果。

def plot_training(history):
    acc = history.history['acc']
    val_acc = history.history['val_acc']
    loss = history.history['loss']
    val_loss = history.history['val_loss']
    epochs = range(len(acc))

    plt.plot(epochs, acc, 'r.')
    plt.plot(epochs, val_acc, 'r')
    plt.title('Training and validation accuracy')

    plt.figure()
    plt.plot(epochs, loss, 'r.')
    plt.plot(epochs, val_loss, 'r-')
    plt.title('Training and validation loss')
    plt.show()

主函数，传入一些参数。将图片处理的代码也放在主函数中。

def train(train_dir, val_dir, epochs=EPOCHS, batch_size=BAT_SIZE, output_model_file="inceptionv3_25000.model"):
    """Use transfer learning and fine-tuning to train a network on a new dataset"""
    nb_train_samples = get_nb_files(train_dir)
    nb_classes = len(glob.glob(train_dir + "/*"))
    nb_val_samples = get_nb_files(val_dir)
    epochs = int(epochs)
    batch_size = int(batch_size)

    # data prep
    train_datagen =  ImageDataGenerator(
        preprocessing_function=preprocess_input,
        rotation_range=30,
        width_shift_range=0.2,
        height_shift_range=0.2,
        shear_range=0.2,
        zoom_range=0.2,
        horizontal_flip=True
    )
    test_datagen = ImageDataGenerator(
        preprocessing_function=preprocess_input,
        rotation_range=30,
        width_shift_range=0.2,
        height_shift_range=0.2,
        shear_range=0.2,
        zoom_range=0.2,
        horizontal_flip=True
    )

    train_generator = train_datagen.flow_from_directory(
        train_dir,
        target_size=(IM_WIDTH, IM_HEIGHT),
        batch_size=batch_size,
    )

    validation_generator = test_datagen.flow_from_directory(
        val_dir,
        target_size=(IM_WIDTH, IM_HEIGHT),
        batch_size=batch_size,
    )

    # 准备跑起来，首先给 base_model 和 model 赋值，迁移学习和微调都是使用 InceptionV3 的 notop 模型（看 inception_v3.py 源码，此模型是打开了最后一个全连接层），利用 add_new_last_layer 函数增加最后一个全连接层。

    base_model = InceptionV3(weights='imagenet', include_top=False) #include_top=False excludes final FC layer
    model = add_new_last_layer(base_model, nb_classes)

    print "开始迁移学习:\n"

    # transfer learning
    setup_to_transfer_learn(model, base_model)

    history_tl = model.fit_generator(
        train_generator,
        steps_per_epoch=nb_train_samples // batch_size,
        epochs=epochs,
        validation_data=validation_generator,
        validation_steps=nb_val_samples // batch_size,
        class_weight='auto')
    
    print "开始微调:\n"

    # fine-tuning
    setup_to_finetune(model)

    history_ft = model.fit_generator(
        train_generator,
        steps_per_epoch=nb_train_samples // batch_size,
        epochs=epochs,
        validation_data=validation_generator,
        validation_steps=nb_val_samples // batch_size,
        class_weight='auto')

    model.save(output_model_file)

    plot_training(history_ft)

# train(train_dir, val_dir, epochs=EPOCHS, batch_size=BAT_SIZE, output_model_file="inceptionv3_nbs.model")

train("./data/train", "./data/validation")

Found 20000 images belonging to 2 classes.Found 5000 images belonging to 2 classes.Epoch 1/10
500/500 [==============================] - 364s - loss: 0.8835 - acc: 0.8754 - val_loss: 0.0983 - val_acc: 0.9596
Epoch 2/10
500/500 [==============================] - 350s - loss: 0.1555 - acc: 0.9423 - val_loss: 0.1182 - val_acc: 0.9570
Epoch 3/10
500/500 [==============================] - 348s - loss: 0.1257 - acc: 0.9497 - val_loss: 0.0827 - val_acc: 0.9686
Epoch 4/10
500/500 [==============================] - 349s - loss: 0.1244 - acc: 0.9531 - val_loss: 0.0774 - val_acc: 0.9656
Epoch 5/10
500/500 [==============================] - 348s - loss: 0.1112 - acc: 0.9589 - val_loss: 0.1371 - val_acc: 0.9506
Epoch 6/10
500/500 [==============================] - 347s - loss: 0.1082 - acc: 0.9590 - val_loss: 0.0708 - val_acc: 0.9732
Epoch 7/10
500/500 [==============================] - 350s - loss: 0.1078 - acc: 0.9601 - val_loss: 0.0730 - val_acc: 0.9712
Epoch 8/10
500/500 [==============================] - 351s - loss: 0.1055 - acc: 0.9617 - val_loss: 0.1071 - val_acc: 0.9650
Epoch 9/10
500/500 [==============================] - 351s - loss: 0.1028 - acc: 0.9638 - val_loss: 0.1173 - val_acc: 0.9580
Epoch 10/10
500/500 [==============================] - 353s - loss: 0.1036 - acc: 0.9611 - val_loss: 0.0654 - val_acc: 0.9748
Epoch 1/10
500/500 [==============================] - 363s - loss: 0.0712 - acc: 0.9741 - val_loss: 0.0720 - val_acc: 0.9770
Epoch 2/10
500/500 [==============================] - 357s - loss: 0.0587 - acc: 0.9779 - val_loss: 0.0566 - val_acc: 0.9756
Epoch 3/10
500/500 [==============================] - 360s - loss: 0.0555 - acc: 0.9781 - val_loss: 0.0561 - val_acc: 0.9798
Epoch 4/10
500/500 [==============================] - 360s - loss: 0.0518 - acc: 0.9795 - val_loss: 0.0580 - val_acc: 0.9796
Epoch 5/10
500/500 [==============================] - 360s - loss: 0.0458 - acc: 0.9815 - val_loss: 0.0509 - val_acc: 0.9826
Epoch 6/10
500/500 [==============================] - 361s - loss: 0.0458 - acc: 0.9827 - val_loss: 0.0491 - val_acc: 0.9792
Epoch 7/10
500/500 [==============================] - 363s - loss: 0.0457 - acc: 0.9810 - val_loss: 0.0538 - val_acc: 0.9816
Epoch 8/10
500/500 [==============================] - 362s - loss: 0.0490 - acc: 0.9809 - val_loss: 0.0489 - val_acc: 0.9820
Epoch 9/10
500/500 [==============================] - 363s - loss: 0.0388 - acc: 0.9847 - val_loss: 0.0530 - val_acc: 0.9830
Epoch 10/10
500/500 [==============================] - 365s - loss: 0.0390 - acc: 0.9849 - val_loss: 0.0419 - val_acc: 0.9842