利用CIFAR-10数据集，基于keras，练习构建简单的CNN网络和选VGG16做为baseline的迁移学习

---

前言

利用CIFAR-10数据集，基于keras，练习构建简单的CNN网络和选VGG16做为baseline的迁移学习
（又名：放假了了把做过的project整理一下，复习一遍课程

---

一、介绍Keras的中文版的文档，原文链接：

https://keras-cn.readthedocs.io/en/latest/getting_started/sequential_model/

二、keras 构建模型的流程

先指定模型 Sequential( ) ---->堆叠模块 .add( ) ---->编译模型 .compile( ) ---->在训练数据上迭代 .fit( ) ---->评估 .evaluate( ) ---->对新数据的预测 .predict( )

三、常用层

常用层对应于core模块，core内部定义了一系列常用的网络层，包括全连接层、激活层等。

Dense层（全连接层）：

所实现的运算是output = activation(dot(input, kernel)+bias)。其中activation是逐元素计算的激活函数，kernel是本层的权值矩阵，bias为偏置向量，只有当use_bias=True才会添加。

如果本层的输入数据的维度大于2，则会先被压为与kernel相匹配的大小。

Dropout层：

Dropout将在训练过程中每次更新参数时按一定概率（rate）随机断开输入神经元，Dropout层用于防止过拟合。

Flatten层：

用来将输入“压平”，即把多维的输入一维化，常用在从卷积层到全连接层的过渡。Flatten不影响和batch的大小。

reshape层

Permute层：当需要将RNN和CNN网络连接的时候，可能会用到该层。

RepeatVector层

repeatvector层将输入重复n次

四、实验

Task:
(1) train and test a CNN model on GPU without transfer learning;
(2) train and test a CNN model on GPU with transfer learning.
Dataset：
CIFAR-10 dataset, which is preinstalled with Tensorflow.
https://keras.io/api/datasets/
The CIFAR-10 dataset consists of 60000 32x32 color images in 10 classes, with 6000 images per
class. There are 50000 training images and 10000 test images.

1.Helpful Functions for Tensorflow (Little Gems)

The following functions will be used with TensorFlow to help preprocess the data. They allow you to build the feature vector for a neural network.

Predictors/Inputs
Fill any missing inputs with the median for that column. Use missing_median.
Encode textual/categorical values with encode_text_dummy.
Encode numeric values with encode_numeric_zscore.
Output
Discard rows with missing outputs.
Encode textual/categorical values with encode_text_index.
Do not encode output numeric values.
Produce final feature vectors (x) and expected output (y) with to_xy.

from collections.abc import Sequence
from sklearn import preprocessing
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import shutil
import os


# Encode text values to dummy variables(i.e. [1,0,0],[0,1,0],[0,0,1] for red,green,blue)
def encode_text_dummy(df, name):
    dummies = pd.get_dummies(df[name])
    for x in dummies.columns:
        dummy_name = "{}-{}".format(name, x)
        df[dummy_name] = dummies[x]
    df.drop(name, axis=1, inplace=True)


# Encode text values to indexes(i.e. [1],[2],[3] for red,green,blue).
def encode_text_index(df, name):
    le = preprocessing.LabelEncoder()
    df[name] = le.fit_transform(df[name])
    return le.classes_


# Encode a numeric column as zscores
def encode_numeric_zscore(df, name, mean=None, sd=None):
    if mean is None:
        mean = df[name].mean()

    if sd is None:
        sd = df[name].std()

    df[name] = (df[name] - mean) / sd


# Convert all missing values in the specified column to the median
def missing_median(df, name):
    med = df[name].median()
    df[name] = df[name].fillna(med)


# Convert all missing values in the specified column to the default
def missing_default(df, name, default_value):
    df[name] = df[name].fillna(default_value)


# Convert a Pandas dataframe to the x,y inputs that TensorFlow needs
def to_xy(df, target):
    result = []
    for x in df.columns:
        if x != target:
            result.append(x)
    # find out the type of the target column. 
    target_type = df[target].dtypes
    target_type = target_type[0] if isinstance(target_type, Sequence) else target_type
    # Encode to int for classification, float otherwise. TensorFlow likes 32 bits.
    if target_type in (np.int64, np.int32):
        # Classification
        dummies = pd.get_dummies(df[target])
        return df[result].values.astype(np.float32), dummies.values.astype(np.float32)
    else:
        # Regression
        return df[result].values.astype(np.float32), df[target].values.astype(np.float32)

# Nicely formatted time string
def hms_string(sec_elapsed):
    h = int(sec_elapsed / (60 * 60))
    m = int((sec_elapsed % (60 * 60)) / 60)
    s = sec_elapsed % 60
    return "{}:{:>02}:{:>05.2f}".format(h, m, s)


# Regression chart.
def chart_regression(pred,y,sort=True):
    t = pd.DataFrame({'pred' : pred, 'y' : y.flatten()})
    if sort:
        t.sort_values(by=['y'],inplace=True)
    a = plt.plot(t['y'].tolist(),label='expected')
    b = plt.plot(t['pred'].tolist(),label='prediction')
    plt.ylabel('output')
    plt.legend()
    plt.show()

# Remove all rows where the specified column is +/- sd standard deviations
def remove_outliers(df, name, sd):
    drop_rows = df.index[(np.abs(df[name] - df[name].mean()) >= (sd * df[name].std()))]
    df.drop(drop_rows, axis=0, inplace=True)


# Encode a column to a range between normalized_low and normalized_high.
def encode_numeric_range(df, name, normalized_low=-1, normalized_high=1,
                         data_low=None, data_high=None):
    if data_low is None:
        data_low = min(df[name])
        data_high = max(df[name])

    df[name] = ((df[name] - data_low) / (data_high - data_low)) \
               * (normalized_high - normalized_low) + normalized_low
    
# Return classifier based on number. Referenced from cifar10 documentation 
def image(i):
  switcher={
    0:'Airplane',
    1:'Automobile',
    2:'Bird',
    3:'Cat',
    4:'Deer',
    5:'Dog',
    6:'Frog',
    7:'Horse',
    8:'Ship',
    9:'Truck'
  }
  return switcher.get(i,"Invalid")

2.colab中选择GPU环境并查看

Runtime->Change runtime type->Hardware Accelerator->GPU.

#查看GPU
import tensorflow as tf
tf.test.gpu_device_name()
#查看内存
!nvidia-smi
#查看版本
import torch
torch.__version__
#路径
import os
from google.colab import drive
drive.mount('/content/drive')
path = "/content/drive/My Drive"
os.chdir(path)

3. 将数据集划分为测试集和训练集

#  Load cifar-10 data and split it to training and test


from tensorflow.keras.datasets import cifar10

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten
from tensorflow.keras.layers import Conv2D, MaxPooling2D

num_classes = 10

# The data split between train and test sets:
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
# print out data shape

print('x_train shape:', x_train.shape)
print('y_train shape:', y_train.shape)

print('x_test shape:', x_test.shape)
print('y_test shape:', y_test.shape)


print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')

# Convert y_train and y_test from 2D to 1D 
y_train = y_train.reshape(50000)
y_test = y_test.reshape(10000)

# Convert class vectors to one hot format
y_train = tf.keras.utils.to_categorical(y_train, num_classes)
y_test = tf.keras.utils.to_categorical(y_test, num_classes)

# Convert data from int to float and normalize it
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255

4.搭建自己的CNN模型

model = Sequential()
model.add(Conv2D(filters = 32, kernel_size = (3,3),
                 input_shape = (32,32,3),
                 activation = 'relu',
                 padding = 'same'))
model.add(Dropout(0.5))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(filters = 64, kernel_size = (3,3),
                 activation = 'relu',
                 padding = 'same'))
model.add(Dropout(0.5))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Flatten())
model.add(Dropout(0.5))
model.add(Dense(1024, activation = 'relu'))
model.add(Dropout(0.5))
model.add(Dense(10, activation = 'softmax'))
# Print model summary

model.summary()

5.compile, earlystopping and fit

from tensorflow.keras.optimizers import Adam
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint
callbacks = [EarlyStopping(monitor='val_loss', patience=8),
             ModelCheckpoint(filepath='best_model.h5', monitor='val_loss', save_best_only=True)]
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
history=model.fit(x_train, y_train,epochs=40,callbacks=callbacks, batch_size=32)

model.save('CVTLmodel.h5')

6.预测查看结果

from sklearn import metrics
predicted = model.predict(x_test,batch_size=32, verbose=0)
y_pred = np.argmax(predicted, axis=1)
y_pred = tf.keras.utils.to_categorical(y_pred, 10)
print(metrics.classification_report(y_test, y_pred))
#print(predicted.shape)
#print(y_test.shape)
#print(type(predicted),type(y_test))
#_, accuracy = model.evaluate(x_test, y_test)
#print('Accuracy: %.2f' % (accuracy*100))

7.预测结果与实际图片标签对比

y_pred_cat = np.argmax(y_pred, axis=1)
y_test_cat = np.argmax(y_test, axis=1)
# use function image to print out images
%matplotlib inline

#print("Image (#{}): Predicted as '{}', Actual Image is '{}'".format(5, y_pred_cat[5], y_test_cat[5]))
#plt.imshow(x_test[5], cmap='gray')
#plt.show()
_, axes = plt.subplots(nrows=1, ncols=5, figsize=(15, 3))
for ax, image, prediction,act in zip(axes, x_test, y_pred_cat,y_test_cat):
    ax.set_axis_off()
    image = image.reshape(32, 32, 3)
    ax.imshow(image, cmap=plt.cm.gray_r, interpolation="nearest")
    ax.set_title(f"Prediction: {prediction}，Actual{act}")  

8.迁移学习

pre-trained model： VGG16
https://neurohive.io/en/popular-networks/vgg16/
VGG16 支持低至 48x48 的图像作为输入。 但是，我们图像的分辨率太低，即 (32, 32)，所以我们需要提高分辨率。 这称为上采样。
找到一种方法对每个图像进行上采样，以将其分辨率从 32x32 提高到 64x64。 一种选择是使用 scikit-image 库 (https://scikit-image.org/) 提供的函数 resize()
注意事项：
(1) 如果您在 scikit-image 中使用 resize()，该函数也会对输入图像进行归一化，因此您可能不想进行两次归一化。
(2) 分别对 x_train 和 x_test 应用上采样。

数据处理

import skimage.transform

#new_x_train = []
from skimage.transform import rescale, resize, downscale_local_mean
new_x_train = np.zeros((50000, 64, 64, 3))

for x in range(len(x_train)):
  image = resize(x_train[x], (64, 64))
  new_x_train[x] = image  
    
#new_x_test = []
new_x_test = np.zeros((10000,64,64,3))

for i in range(len(x_test)):
    image_test = resize(x_test[i],(64,64))
    new_x_test[i] = image_test       
# this process may take about a few minutes ....
# covert y_train and y_test from 2D to 1D
y_train = y_train.reshape(50000)
y_test  = y_test.reshape(10000)
# Convert class vectors to one hot format

y_train = tf.keras.utils.to_categorical(y_train, num_classes)
y_test = tf.keras.utils.to_categorical(y_test, num_classes)

导入pre-trained model： VGG16

from tensorflow.keras.applications.vgg16 import VGG16

vgg_model = VGG16(weights='imagenet', include_top=False, input_shape=(64, 64, 3))   #  first hidden layer


model = Sequential()

  
# write your code here

model.add(Dropout(0.5))

model = tf.keras.models.Model(inputs=vgg_model.input, outputs=model(vgg_model.output)) 
# print out the model summary
model.summary()
#vgg_model.summary()

冻结权重

for layers in model.layers:
    layers.trainable = False

自己添加一些层

model = Sequential()
model.add(Flatten())

# Add some "Dense" layers here, including output layer
model.add(Dense(4096, activation = 'relu'))
model.add(Dropout(0.5))
model.add(Dense(4096, activation = 'relu'))
model.add(Dropout(0.5))
model.add(Dense(10, activation = 'softmax'))
model = tf.keras.models.Model(inputs=vgg_model.input, outputs=model(vgg_model.output)) 
# print out the model summary again to check number of parameters
model.summary()

训练

from tensorflow.keras.optimizers import Adam
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint

from tensorflow.keras.optimizers import Adam
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint
callbacks = [EarlyStopping(monitor='val_loss', patience=8),
             ModelCheckpoint(filepath='best_VGGmodel.h5', monitor='val_loss', save_best_only=True)]
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
history=model.fit(new_x_train, y_train,epochs=40,callbacks=callbacks, batch_size=32)

model.save('VGGCVTLmodel.h5')

预测

from sklearn import metrics
from sklearn import metrics
predicted = model.predict(new_x_test,batch_size=32, verbose=0)
y_pred = np.argmax(predicted, axis=1)
y_pred = tf.keras.utils.to_categorical(y_pred, 10)
print(metrics.classification_report(y_test, y_pred))

# use function image to print out images
%matplotlib inline
y_pred_cat = np.argmax(y_pred, axis=1)
y_test_cat = np.argmax(y_test, axis=1)
print("Image (#{}): Predicted as '{}', Actual Image is '{}'".format(5, y_pred_cat[5], y_test_cat[5]))
plt.imshow(x_test[5], cmap='gray')
plt.show()
_, axes = plt.subplots(nrows=1, ncols=5, figsize=(15, 3))
for ax, image, prediction,act in zip(axes, new_x_test, y_pred_cat,y_test_cat):
    ax.set_axis_off()
    image = image.reshape(64, 64, 3)
    ax.imshow(image, cmap=plt.cm.gray_r, interpolation="nearest")
    ax.set_title(f"Prediction: {prediction}，Actual{act}") 

五、容易遇到的报错

keras中one-hot和数组相互转换

from keras.utils import to_categorical
import numpy as np

input_data = [1, 2, 3, 4, 5, 6, 7, 8, 9, 0]
input_data = np.array(input_data)
input_data = to_categorical(input_data)
print(input_data)

input_data = np.argmax(input_data, axis=1)
print(input_data)