u013250861

CV-CNN-2015：ResNet【残差网络，改善深层网络难训练问题：梯度消失或爆炸导致性能退化】【Bottleneck：用1×1核卷积（减小通道数来降维）、3×3核卷积（不变维）、用1×1核升维】

《原始论文：Deep Residual Learning for Image Recognition》

一说起“深度学习”，自然就联想到它非常显著的特点“深、深、深”，通过很深层次的网络实现准确率非常高的图像识别、语音识别等能力。
因此，我们自然很容易就想到：深的网络一般会比浅的网络效果好，如果要进一步地提升模型的准确率，最直接的方法就是把网络设计得越深越好，这样模型的准确率也就会越来越准确。
那现实是这样吗？
先看几个经典的图像识别深度学习模型：
这几个模型都是在世界顶级比赛中获奖的著名模型，然而，一看这些模型的网络层次数量，似乎让人很失望，少则5层，多的也就22层而已，这些世界级模型的网络层级也没有那么深啊，这种也算深度学习吗？为什么不把网络层次加到成百上千层呢？
带着这个问题，我们先来看一个实验，对常规的网络（plain network，也称平原网络）直接堆叠很多层次，经对图像识别结果进行检验，训练集、测试集的误差结果如下图：
从上面两个图可以看出，在网络很深的时候（56层相比20层），模型效果却越来越差了（误差率越高），并不是网络越深越好。
通过实验可以发现：随着网络层级的不断增加，模型精度不断得到提升，而当网络层级增加到一定的数目以后，训练精度和测试精度迅速下降，这说明当网络变得很深以后，深度网络就变得更加难以训练了。

1、为什么随着网络层级越深，模型效果却变差了

下图是一个简单神经网络图，由输入层、隐含层、输出层构成：
回想一下神经网络反向传播的原理，先通过正向传播计算出结果output，然后与样本比较得出误差值 $E_{total}$
根据误差结果，利用著名的“链式法则”求偏导，使结果误差反向传播从而得出权重 $w$ 调整的梯度。下图是输出结果到隐含层的反向传播过程（隐含层到输入层的反向传播过程也是类似）：
通过不断迭代，对参数矩阵进行不断调整后，使得输出结果的误差值更小，使输出结果与事实更加接近。
从上面的过程可以看出，神经网络在反向传播过程中要不断地传播梯度，而当网络层数加深时，梯度在传播过程中会逐渐消失（假如采用Sigmoid函数，对于幅度为1的信号，每向后传递一层，梯度就衰减为原来的0.25，层数越多，衰减越厉害），导致无法对前面网络层的权重进行有效的调整。
那么，如何又能加深网络层数、又能解决梯度消失问题、又能提升模型精度呢？残差网络由此产生。

2、深度残差网络（Deep Residual Network，简称DRN）

前面描述了一个实验结果现象，在不断加神经网络的深度时，模型准确率会先上升然后达到饱和，再持续增加深度时则会导致准确率下降，示意图如下：
那么我们作这样一个假设：假设现有一个比较浅的网络（Shallow Net）已达到了饱和的准确率，这时在它后面再加上几个恒等映射层（identity mapping，也即 $y = x$ ，输出=输入），这样就增加了网络的深度，并且起码误差不会增加，也即更深的网络不应该带来训练集上误差的上升。而这里提到的使用恒等映射直接将前一层输出传到后面的思想，便是著名深度残差网络ResNet的灵感来源。
ResNet引入了残差网络结构（residual network），通过这种残差网络结构，可以把网络层弄的很深（据说目前可以达到1000多层），并且最终的分类效果也非常好，残差网络的基本结构如下图所示，该图表示一个 Basic Block：
残差网络借鉴了高速网络（Highway Network）的跨层链接思想，但对其进行改进（残差项原本是带权值的，但ResNet用恒等映射代替之）。
假定某段神经网络的输入是 $x$ ，期望输出是 $H (x)$ ，即 $H (x)$ 是期望的复杂潜在映射，如果是要学习这样的模型，则训练难度会比较大；
回想前面的假设，如果已经学习到较饱和的准确率（或者当发现下层的误差变大时），那么接下来的学习目标就转变为恒等映射的学习，也就是使输入 $x$ 近似于输出 $H (x)$ ，以保持在后面的层次中不会造成精度下降。
在上图的残差网络结构图中，通过 “shortcut connections（短路连接）” 的方式，直接把输入 $x$ 传到输出作为 “初始结果”，输出结果为 $H (x) = x + F (x)$
其中的 “+” 号表示输出 $H (x)$ 是矩阵 $x$ 与 $F (x)$ 的加和。所以 $x$ 与 $F (x)$ 维度要一致。
当 $F (x) = 0$ 时，那么 $H (x) = x$ ，也就是上面所提到的恒等映射。于是，ResNet相当于将学习目标改变了，不再是学习一个完整的输出，而是目标值 $H (X)$ 和 $x$ 的差值，也就是所谓的残差 $F (x) = H (x) - x$
因此，后面的训练目标就是要将残差结果逼近于0，使到随着网络加深，准确率不下降。
这种残差跳跃式的结构，打破了传统的神经网络 $n - 1$ 层的输出只能给 $n$ 层作为输入的惯例，使某一层的输出可以直接跨过几层作为后面某一层的输入，其意义在于为叠加多层网络而使得整个学习模型的错误率不降反升的难题提供了新的方向。
至此，神经网络的层数可以超越之前的约束，达到几十层、上百层甚至千层，为高级语义特征提取和分类提供了可行性。
下图最右侧是一个34层的深度残差网络的结构图，
- 每一条箭头曲线包裹的部分是一个 Basic Block
- 每一组不同颜色的部分代表一个 Residual Block，是由多个Basic Block组成(一般2~3个)，
- 每个Residual Network 是由多个Residual Block组成：
从图可以看出，怎么有一些 “shortcut connections（短路连接）” 是实线，有一些是虚线，有什么区别呢？
因为经过“shortcut connections（短路连接）” 后， $H (x) = F (x) + x$ ，如果 $F (x)$ 和 $x$ 的通道相同，则可直接相加，那么通道不同怎么相加呢。上图中的实线、虚线就是为了区分这两种情况的：
- 实线的Connection部分，表示通道相同，如上图的第一个粉色矩形和第三个粉色矩形，都是 $3 \times 3 \times 64$ 的特征图，由于通道相同，所以采用计算方式为 $H (x) = F (x) + x$
- 虚线的的Connection部分，表示通道不同，如上图的第一个绿色矩形和第三个绿色矩形，分别是 $3 \times 3 \times 64$ 和 $3 \times 3 \times 128$ 的特征图，通道不同，采用的计算方式为 $H (x) = F (x) + W x$ ，其中 $W$ 是卷积操作，用来调整 $x$ 维度的。
除了上面提到的 两层的 残差学习单元 Basic Block，还有 三层的 残差学习单元 Basic Block，如下图所示：
两种结构分别针对ResNet34（左图）和ResNet50/101/152（右图），其目的主要就是为了降低参数的数目。官方设计了18、34、50、101、152层。
- 左图是两个 $3 \times 3 \times 256$ 的卷积，参数数目: $3 \times 3 \times 256 \times 256 \times 2 = 1179648$ ，
- 右图是第一个 $1 \times 1$ 的卷积把 $256$ 维通道降到 $64$ 维，然后在最后通过 $1 \times 1$ 卷积恢复，整体上用的参数数目： $1 \times 1 \times 256 \times 64 + 3 \times 3 \times 64 \times 64 + 1 \times 1 \times 64 \times 256 = 69632$ ，右图的参数数量比左图减少了16.94倍，因此，右图的主要目的就是为了减少参数量，从而减少计算量。
- 对于常规的ResNet，比如34层或者更少的网络，使用左图结构；对于更深的网络（如101层），则使用右图，其目的是减少计算和参数量。
经检验，深度残差网络的确解决了退化问题，如下图所示，左图为平原网络（plain network）网络层次越深（34层）比网络层次浅的（18层）的误差率更高；右图为残差网络ResNet的网络层次越深（34层）比网络层次浅的（18层）的误差率更低。
ResNet在ILSVRC2015竞赛中惊艳亮相，一下子将网络深度提升到152层，将错误率降到了3.57，在图像识别错误率和网络深度方面，比往届比赛有了非常大的提升，ResNet毫无悬念地夺得了ILSVRC2015的第一名。如下图所示：
在ResNet的作者的第二篇相关论文《Identity Mappings in Deep Residual Networks》中，提出了ResNet V2。ResNet V2 和 ResNet V1 的主要区别在于，作者通过研究 ResNet 残差学习单元的传播公式，发现前馈和反馈信号可以直接传输，因此“shortcut connection”（捷径连接）的非线性激活函数（如ReLU）替换为 Identity Mappings。同时，ResNet V2 在每一层中都使用了 Batch Normalization。这样处理后，新的残差学习单元比以前更容易训练且泛化性更强。

3、ResNet案例-cifar100分类数据集

3.1 自定义ResNet神经网络【Tensorflow2】

import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'  # 放在 import tensorflow as tf 之前才有效

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers, optimizers, datasets, Sequential
#==========================================自定义ResNet神经网络：开始==========================================
# 两层的残差学习单元 BasicBlock [(3×3)-->(3×3)]形状，如果是三层的BasicBlock，形状则为：[(1×1)-->(3×3)-->(1×1)]
# stride>1时(比如stride=2)，则通过改层Layer后的FeatureMap的大小减半。strides: An integer or tuple/list of 2 integers
class BasicBlock(layers.Layer):
    def __init__(self, filter_count, stride=1):
        super(BasicBlock, self).__init__()
        # ================================F(x) 部分================================
        # Layer01
        self.conv1 = layers.Conv2D(filters=filter_count, kernel_size=[3, 3], strides=stride, padding='same')  # 如果padding='same'&stride!=1：输出维度是输入维度的stride分之一。
        self.bn1 = layers.BatchNormalization()
        self.relu = layers.Activation('relu')
        # Layer02
        self.conv2 = layers.Conv2D(filters=filter_count, kernel_size=[3, 3], strides=1, padding='same')  # padding='same'&stride==1：输出维度与输入维度一致。
        self.bn2 = layers.BatchNormalization()
        # ================================identity(x)部分================================
        if stride != 1:  # 如果 stride != 1，F(x)部分的输入维度减小stride倍。所以利用一层大小为[1×1×filter_count]的卷积层identity_layer设置strides与F(x)部分的stride一致，将输入值x的维度调整为和F(x)的维度一致，即进行SubSampling。然后再进行加和计算 H(x)=x+F(X)
            self.identity_layer = layers.Conv2D(filters=filter_count, kernel_size=[1, 1], strides=stride)
        else:  # 如果 stride = 1，则F(x)输出值与输入值x的维度保持不变(必须保证F(x)部分的padding='same'才能维度不变)。所以identity_layer部分直接可以和F(x)部分进行加和计算，不需要经过卷积层对x进行维度调整。也可减少参数的使用。
            self.identity_layer = lambda x: x  # lambda匿名函数：输入为x，return x

    def call(self, inputs, training=None):
        # 前向传播  # [b, h, w, c]
        # ================================F(x) 部分================================
        # Layer01
        F_out = self.conv1(inputs)
        F_out = self.bn1(F_out)
        F_out = self.relu(F_out)
        # Layer02
        F_out = self.conv2(F_out)
        F_out = self.bn2(F_out)
        # ================================identity部分================================
        # x=identity(x)
        identity_out = self.identity_layer(inputs)
        # ================================H(x)=F(x)+x================================
        basic_block_output = layers.add([F_out, identity_out])  # layers.add(): A tensor as the sum of the inputs. It has the same shape as the inputs.
        basic_block_output = tf.nn.relu(basic_block_output)

        return basic_block_output


# 由多个BasicBlock组成的ResidualBlock
class ResidualBlock:
    def __init__(self, filter_count, residualBlock_size, stride=1):
        self.filter_count = filter_count
        self.residualBlock_size = residualBlock_size
        self.stride = stride

    def __call__(self):
        basic_block_stride_not_1 = BasicBlock(self.filter_count, stride=self.stride)  # stride != 1 时的BasicBlock H(x)=x+F(X)，identity_layer进行SubSampling
        basic_block_stride_1 = BasicBlock(self.filter_count, stride=1)  # stride = 1 时的BasicBlock H(x)=x+F(X)，identity_layer层的输出为直接返回输入
        residualBlock = Sequential()
        residualBlock.add(basic_block_stride_not_1)  # 有一个BasicBlock必须是 stride != 1 时的BasicBlock
        for _ in range(1, self.residualBlock_size):  # 其余的BasicBlock都是 stride == 1 时的BasicBlock
            residualBlock.add(basic_block_stride_1)
        return residualBlock

# 由多个ResidualBlock组成的ResidualNet
# residualBlock_size_list：[2, 2, 2, 2]  表示该ResidualNet含有4个ResidualBlock，每个ResidualBlock包含2个BasicBlock
# residualBlock_size_list：[3, 4, 6, 3]  表示该ResidualNet含有4个ResidualBlock，第1个ResidualBlock包含3个BasicBlock，第2个ResidualBlock包含4个BasicBlock，第3个ResidualBlock包含6个BasicBlock，第4个ResidualBlock包含3个BasicBlock
class ResidualNet(keras.Model):
    def __init__(self, residualBlock_size_list, class_count=100):   # class_count：表示全连接层的输出维度，取决于数据集分类的类别总数量(cifar100为100类)
        super(ResidualNet, self).__init__()
        # ================================预处理Block================================
        self.preprocessBlock = Sequential([layers.Conv2D(filters=50, kernel_size=[3, 3], strides=(1, 1)),
                                            layers.BatchNormalization(),
                                            layers.Activation('relu'),
                                            layers.MaxPool2D(pool_size=(2, 2), strides=(1, 1), padding='same')
                                            ])
        # ================================所有ResidualBlock================================
        residualBlock01_size = residualBlock_size_list[0]
        residualBlock02_size = residualBlock_size_list[1]
        residualBlock03_size = residualBlock_size_list[2]
        residualBlock04_size = residualBlock_size_list[3]
        self.residualBlock1 = ResidualBlock(50, residualBlock01_size, stride=1)()  # 第01个ResidualBlock，包含residualBlock01_size个BasicBlock，residualBlock1设置为64通道
        self.residualBlock2 = ResidualBlock(150, residualBlock02_size, stride=2)()  # 第02个ResidualBlock，包含residualBlock02_size个BasicBlock，residualBlock2设置为128通道
        self.residualBlock3 = ResidualBlock(300, residualBlock03_size, stride=2)()  # 第03个ResidualBlock，包含residualBlock03_size个BasicBlock，residualBlock3设置为256通道
        self.residualBlock4 = ResidualBlock(500, residualBlock04_size, stride=2)()  # 第04个ResidualBlock，包含residualBlock04_size个BasicBlock，residualBlock4设置为512通道
        # ================================输出层================================
        # output: [b, h, w, 500] 以上步骤输出的FeatureMap的大小[h,w]不太方便计算
        self.avgpool_Layer = layers.GlobalAveragePooling2D()    # 不管输入的每一个FeatureMap的大小[h,w]是多少，取每一个FeatureMap上的所有元素的平均值作为输出。所以该步骤输出的数据维度为[1,500]
        # 将上一层的维度为[1,500]的输出传给全连接层进行分类，输出维度为[1,class_count]
        self.fullcon_Layer = layers.Dense(class_count)

    def call(self, inputs, training=None):
        # ================================预处理Block================================
        x = self.preprocessBlock(inputs)   # 输出维度：[b, h, w, 50]
        # ================================所有ResidualBlock================================
        x = self.residualBlock1(x)   # 输出维度：[b, h, w, 50]
        x = self.residualBlock2(x)   # 输出维度：[b, h, w, 150]
        x = self.residualBlock3(x)   # 输出维度：[b, h, w, 300]
        x = self.residualBlock4(x)   # 输出维度：[b, h, w, 500]
        # ================================输出层================================
        x = self.avgpool_Layer(x)   # 输出维度：[b, 500]
        x = self.fullcon_Layer(x)   # 输出维度：[b, 100]

        return x

def resnet18():
    return ResidualNet([2, 2, 2, 2])

def resnet34():
    return ResidualNet([3, 4, 6, 3])
    
#==========================================自定义ResNet神经网络：结束==========================================

# 一、获取数据集
(X_train, Y_train), (X_val, Y_val) = datasets.cifar100.load_data()
print('X_train.shpae = {0}，Y_train.shpae = {1}------------type(X_train) = {2}，type(Y_train) = {3}'.format(X_train.shape, Y_train.shape, type(X_train), type(Y_train)))
Y_train = tf.squeeze(Y_train)
Y_val = tf.squeeze(Y_val)
print('X_train.shpae = {0}，Y_train.shpae = {1}------------type(X_train) = {2}，type(Y_train) = {3}'.format(X_train.shape, Y_train.shape, type(X_train), type(Y_train)))


# 二、数据处理
# 预处理函数：将numpy数据转为tensor
def preprocess(x, y):
    x = tf.cast(x, dtype=tf.float32) / 255.
    y = tf.cast(y, dtype=tf.int32)
    return x, y


# 2.1 处理训练集
# print('X_train.shpae = {0}，Y_train.shpae = {1}------------type(X_train) = {2}，type(Y_train) = {3}'.format(X_train.shape, Y_train.shape, type(X_train), type(Y_train)))
db_train = tf.data.Dataset.from_tensor_slices((X_train, Y_train))  # 此步骤自动将numpy类型的数据转为tensor
db_train = db_train.map(preprocess)  # 调用map()函数批量修改每一个元素数据的数据类型
# 从data数据集中按顺序抽取buffer_size个样本放在buffer中，然后打乱buffer中的样本。buffer中样本个数不足buffer_size，继续从data数据集中安顺序填充至buffer_size，此时会再次打乱。
db_train = db_train.shuffle(buffer_size=1000)  # 打散db_train中的样本顺序，防止图片的原始顺序对神经网络性能的干扰。
print('db_train = {0}，type(db_train) = {1}'.format(db_train, type(db_train)))
batch_size_train = 500  # 每个batch里的样本数量设置100-200之间合适。
db_batch_train = db_train.batch(batch_size_train)  # 将db_batch_train中每sample_num_of_each_batch_train张图片分为一个batch，读取一个batch相当于一次性并行读取sample_num_of_each_batch_train张图片
print('db_batch_train = {0}，type(db_batch_train) = {1}'.format(db_batch_train, type(db_batch_train)))
# 2.2 处理测试集：测试数据集不需要打乱顺序
db_val = tf.data.Dataset.from_tensor_slices((X_val, Y_val))  # 此步骤自动将numpy类型的数据转为tensor
db_val = db_val.map(preprocess)  # 调用map()函数批量修改每一个元素数据的数据类型
batch_size_val = 500  # 每个batch里的样本数量设置100-200之间合适。
db_batch_val = db_val.batch(batch_size_val)  # 将db_val中每sample_num_of_each_batch_val张图片分为一个batch，读取一个batch相当于一次性并行读取sample_num_of_each_batch_val张图片

# 三、构建ResNet神经网络
# 1、构建ResNet神经网络
resnet18_network = resnet18()
resnet18_network.build(input_shape=[None, 32, 32, 3])  # 原始图片维度为：[32, 32, 3]，None表示样本数量，是不确定的值。
# 2、打印神经网络信息
resnet18_network.summary()  # 打印卷积神经网络network的简要信息

# 四、梯度下降优化器设置
optimizer = optimizers.Adam(lr=1e-3)


# 五、整体数据集进行一次梯度下降来更新模型参数，整体数据集迭代一次，一般用epoch。每个epoch中含有batch_step_no个step，每个step中就是设置的每个batch所含有的样本数量。
def train_epoch(epoch_no):
    print('++++++++++++++++++++++++++++++++++++++++++++第{0}轮Epoch-->Training 阶段：开始++++++++++++++++++++++++++++++++++++++++++++'.format(epoch_no))
    for batch_step_no, (X_batch, Y_batch) in enumerate(db_batch_train):  # 每次计算一个batch的数据，循环结束则计算完毕整体数据的一次梯度下降；每个batch的序号一般用step表示(batch_step_no)
        print('epoch_no = {0}, batch_step_no = {1}，X_batch.shpae = {2}，Y_batch.shpae = {3}------------type(X_batch) = {4}，type(Y_batch) = {5}'.format(epoch_no, batch_step_no + 1, X_batch.shape, Y_batch.shape, type(X_batch), type(Y_batch)))
        Y_batch_one_hot = tf.one_hot(Y_batch, depth=100)  # One-Hot编码，共有100类  [] => [b,100]
        print('\tY_train_one_hot.shpae = {0}'.format(Y_batch_one_hot.shape))
        # 梯度带tf.GradientTape：连接需要计算梯度的”函数“和”变量“的上下文管理器（context manager）。将“函数”(即Loss的定义式)与“变量”(即神经网络的所有参数)都包裹在tf.GradientTape中进行追踪管理
        with tf.GradientTape() as tape:
            # Step1. 前向传播/前向运算-->计算当前参数下模型的预测值
            out_logits = resnet18_network(X_batch)  # [b, 32, 32, 3] => [b, 100]
            print('\tout_logits.shape = {0}'.format(out_logits.shape))
            # Step2. 计算预测值与真实值之间的损失Loss：交叉熵损失
            MSE_Loss = tf.losses.categorical_crossentropy(Y_batch_one_hot, out_logits, from_logits=True)    # categorical_crossentropy()第一个参数是真实值，第二个参数是预测值，顺序不能颠倒
            print('\tMSE_Loss.shape = {0}'.format(MSE_Loss.shape))
            MSE_Loss = tf.reduce_mean(MSE_Loss)
            print('\t求均值后：MSE_Loss.shape = {0}'.format(MSE_Loss.shape))
            print('\t第{0}个epoch-->第{1}个batch step的初始时的：MSE_Loss = {2}'.format(epoch_no, batch_step_no + 1, MSE_Loss))
        # Step3. 反向传播-->损失值Loss下降一个学习率的梯度之后所对应的更新后的各个Layer的参数：W1, W2, W3, B1, B2, B3...
        # grads为整个全连接神经网络模型中所有Layer的待优化参数trainable_variables [W1, W2, W3, B1, B2, B3...]分别对目标函数MSE_Loss 在 X_batch 处的梯度值，
        grads = tape.gradient(MSE_Loss, resnet18_network.trainable_variables)  # grads为梯度值。MSE_Loss为目标函数，variables为卷积神经网络、全连接神经网络所有待优化参数，
        # grads, _ = tf.clip_by_global_norm(grads, 15)  # 限幅：解决gradient explosion或者gradients vanishing的问题。
        # print('\t第{0}个epoch-->第{1}个batch step的初始时的参数：'.format(epoch_no, batch_step_no + 1))
        if batch_step_no == 0:
            index_variable = 1
            for grad in grads:
                print('\t\tgrad{0}：grad.shape = {1}，grad.ndim = {2}'.format(index_variable, grad.shape, grad.ndim))
                index_variable = index_variable + 1
        # 进行一次梯度下降
        print('\t梯度下降步骤-->optimizer.apply_gradients(zip(grads, resnet18_network.trainable_variables))：开始')
        optimizer.apply_gradients(zip(grads, resnet18_network.trainable_variables))  # network的所有参数 trainable_variables [W1, W2, W3, B1, B2, B3...]下降一个梯度  w' = w - lr * grad，zip的作用是让梯度值与所属参数前后一一对应
        print('\t梯度下降步骤-->optimizer.apply_gradients(zip(grads, resnet18_network.trainable_variables))：结束\n')
    print('++++++++++++++++++++++++++++++++++++++++++++第{0}轮Epoch-->Training 阶段：结束++++++++++++++++++++++++++++++++++++++++++++'.format(epoch_no))


# 六、模型评估 test/evluation
def evluation(epoch_no):
    print('++++++++++++++++++++++++++++++++++++++++++++第{0}轮Epoch-->Evluation 阶段：开始++++++++++++++++++++++++++++++++++++++++++++'.format(epoch_no))
    total_correct, total_num = 0, 0
    for batch_step_no, (X_batch, Y_batch) in enumerate(db_batch_val):
        print('epoch_no = {0}, batch_step_no = {1}，X_batch.shpae = {2}，Y_batch.shpae = {3}'.format(epoch_no, batch_step_no + 1, X_batch.shape, Y_batch.shape))
        # 根据训练模型计算测试数据的输出值out
        out_logits = resnet18_network(X_batch)   # [b, 32, 32, 3] => [b, 100]
        print('\tout_logits.shape = {0}'.format(out_logits.shape))
        # print('\tout_logits_fullcon[:1,:] = {0}'.format(out_logits_fullcon[:1, :]))
        # 利用softmax()函数将network的输出值转为0~1范围的值，并且使得所有类别预测概率总和为1
        out_logits_prob = tf.nn.softmax(out_logits, axis=1)  # out_logits_prob: [b, 100] ~ [0, 1]
        # print('\tout_logits_prob[:1,:] = {0}'.format(out_logits_prob[:1, :]))
        out_logits_prob_max_index = tf.cast(tf.argmax(out_logits_prob, axis=1), dtype=tf.int32)  # [b, 100] => [b] 查找最大值所在的索引位置 int64 转为 int32
        # print('\t预测值：out_logits_prob_max_index = {0},\t真实值：Y_train_one_hot = {1}'.format(out_logits_prob_max_index, Y_batch))
        is_correct_boolean = tf.equal(out_logits_prob_max_index, Y_batch.numpy())
        # print('\tis_correct_boolean = {0}'.format(is_correct_boolean))
        is_correct_int = tf.cast(is_correct_boolean, dtype=tf.float32)
        # print('\tis_correct_int = {0}'.format(is_correct_int))
        is_correct_count = tf.reduce_sum(is_correct_int)
        print('\tis_correct_count = {0}\n'.format(is_correct_count))
        total_correct += int(is_correct_count)
        total_num += X_batch.shape[0]
    print('total_correct = {0}---total_num = {1}'.format(total_correct, total_num))
    acc = total_correct / total_num
    print('第{0}轮Epoch迭代的准确度： acc = {1}'.format(epoch_no, acc))
    print('++++++++++++++++++++++++++++++++++++++++++++第{0}轮Epoch-->Evluation 阶段：结束++++++++++++++++++++++++++++++++++++++++++++'.format(epoch_no))


# 七、整体数据迭代多次梯度下降来更新模型参数
def train():
    epoch_count = 1  # epoch_count为整体数据集迭代梯度下降次数
    for epoch_no in range(1, epoch_count + 1):
        print('\n\n利用整体数据集进行模型的第{0}轮Epoch迭代开始:**********************************************************************************************************************************'.format(epoch_no))
        train_epoch(epoch_no)
        evluation(epoch_no)
        print('利用整体数据集进行模型的第{0}轮Epoch迭代结束:**********************************************************************************************************************************'.format(epoch_no))


if __name__ == '__main__':
    train()

打印结果：

X_train.shpae = (50000, 32, 32, 3)，Y_train.shpae = (50000, 1)------------type(X_train) = <class 'numpy.ndarray'>，type(Y_train) = <class 'numpy.ndarray'>
X_train.shpae = (50000, 32, 32, 3)，Y_train.shpae = (50000,)------------type(X_train) = <class 'numpy.ndarray'>，type(Y_train) = <class 'tensorflow.python.framework.ops.EagerTensor'>
db_train = <ShuffleDataset shapes: ((32, 32, 3), ()), types: (tf.float32, tf.int32)>，type(db_train) = <class 'tensorflow.python.data.ops.dataset_ops.ShuffleDataset'>
db_batch_train = <BatchDataset shapes: ((None, 32, 32, 3), (None,)), types: (tf.float32, tf.int32)>，type(db_batch_train) = <class 'tensorflow.python.data.ops.dataset_ops.BatchDataset'>
Model: "residual_net"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
sequential (Sequential)      (None, 30, 30, 50)        1600      
_________________________________________________________________
sequential_1 (Sequential)    (None, 30, 30, 50)        91000     
_________________________________________________________________
sequential_2 (Sequential)    (None, 15, 15, 150)       685650    
_________________________________________________________________
sequential_3 (Sequential)    (None, 8, 8, 300)         2886300   
_________________________________________________________________
sequential_4 (Sequential)    (None, 4, 4, 500)         8260500   
_________________________________________________________________
global_average_pooling2d (Gl multiple                  0         
_________________________________________________________________
dense (Dense)                multiple                  50100     
=================================================================
Total params: 11,975,150
Trainable params: 11,967,050
Non-trainable params: 8,100
_________________________________________________________________


利用整体数据集进行模型的第1轮Epoch迭代开始:**********************************************************************************************************************************
++++++++++++++++++++++++++++++++++++++++++++第1轮Epoch-->Training 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 1, batch_step_no = 1，X_batch.shpae = (500, 32, 32, 3)，Y_batch.shpae = (500,)------------type(X_batch) = <class 'tensorflow.python.framework.ops.EagerTensor'>，type(Y_batch) = <class 'tensorflow.python.framework.ops.EagerTensor'>
	Y_train_one_hot.shpae = (500, 100)
	out_logits.shape = (500, 100)
	MSE_Loss.shape = (500,)
	求均值后：MSE_Loss.shape = ()
	第1个epoch-->第1个batch step的初始时的：MSE_Loss = 4.608854293823242
		grad1：grad.shape = (3, 3, 3, 50)，grad.ndim = 4
		grad2：grad.shape = (50,)，grad.ndim = 1
		grad3：grad.shape = (50,)，grad.ndim = 1
		grad4：grad.shape = (50,)，grad.ndim = 1
		grad5：grad.shape = (3, 3, 50, 50)，grad.ndim = 4
		grad6：grad.shape = (50,)，grad.ndim = 1
		grad7：grad.shape = (50,)，grad.ndim = 1
		grad8：grad.shape = (50,)，grad.ndim = 1
		grad9：grad.shape = (3, 3, 50, 50)，grad.ndim = 4
		grad10：grad.shape = (50,)，grad.ndim = 1
		grad11：grad.shape = (50,)，grad.ndim = 1
		grad12：grad.shape = (50,)，grad.ndim = 1
		grad13：grad.shape = (3, 3, 50, 50)，grad.ndim = 4
		grad14：grad.shape = (50,)，grad.ndim = 1
		grad15：grad.shape = (50,)，grad.ndim = 1
		grad16：grad.shape = (50,)，grad.ndim = 1
		grad17：grad.shape = (3, 3, 50, 50)，grad.ndim = 4
		grad18：grad.shape = (50,)，grad.ndim = 1
		grad19：grad.shape = (50,)，grad.ndim = 1
		grad20：grad.shape = (50,)，grad.ndim = 1
		grad21：grad.shape = (3, 3, 50, 150)，grad.ndim = 4
		grad22：grad.shape = (150,)，grad.ndim = 1
		grad23：grad.shape = (150,)，grad.ndim = 1
		grad24：grad.shape = (150,)，grad.ndim = 1
		grad25：grad.shape = (3, 3, 150, 150)，grad.ndim = 4
		grad26：grad.shape = (150,)，grad.ndim = 1
		grad27：grad.shape = (150,)，grad.ndim = 1
		grad28：grad.shape = (150,)，grad.ndim = 1
		grad29：grad.shape = (1, 1, 50, 150)，grad.ndim = 4
		grad30：grad.shape = (150,)，grad.ndim = 1
		grad31：grad.shape = (3, 3, 150, 150)，grad.ndim = 4
		grad32：grad.shape = (150,)，grad.ndim = 1
		grad33：grad.shape = (150,)，grad.ndim = 1
		grad34：grad.shape = (150,)，grad.ndim = 1
		grad35：grad.shape = (3, 3, 150, 150)，grad.ndim = 4
		grad36：grad.shape = (150,)，grad.ndim = 1
		grad37：grad.shape = (150,)，grad.ndim = 1
		grad38：grad.shape = (150,)，grad.ndim = 1
		grad39：grad.shape = (3, 3, 150, 300)，grad.ndim = 4
		grad40：grad.shape = (300,)，grad.ndim = 1
		grad41：grad.shape = (300,)，grad.ndim = 1
		grad42：grad.shape = (300,)，grad.ndim = 1
		grad43：grad.shape = (3, 3, 300, 300)，grad.ndim = 4
		grad44：grad.shape = (300,)，grad.ndim = 1
		grad45：grad.shape = (300,)，grad.ndim = 1
		grad46：grad.shape = (300,)，grad.ndim = 1
		grad47：grad.shape = (1, 1, 150, 300)，grad.ndim = 4
		grad48：grad.shape = (300,)，grad.ndim = 1
		grad49：grad.shape = (3, 3, 300, 300)，grad.ndim = 4
		grad50：grad.shape = (300,)，grad.ndim = 1
		grad51：grad.shape = (300,)，grad.ndim = 1
		grad52：grad.shape = (300,)，grad.ndim = 1
		grad53：grad.shape = (3, 3, 300, 300)，grad.ndim = 4
		grad54：grad.shape = (300,)，grad.ndim = 1
		grad55：grad.shape = (300,)，grad.ndim = 1
		grad56：grad.shape = (300,)，grad.ndim = 1
		grad57：grad.shape = (3, 3, 300, 500)，grad.ndim = 4
		grad58：grad.shape = (500,)，grad.ndim = 1
		grad59：grad.shape = (500,)，grad.ndim = 1
		grad60：grad.shape = (500,)，grad.ndim = 1
		grad61：grad.shape = (3, 3, 500, 500)，grad.ndim = 4
		grad62：grad.shape = (500,)，grad.ndim = 1
		grad63：grad.shape = (500,)，grad.ndim = 1
		grad64：grad.shape = (500,)，grad.ndim = 1
		grad65：grad.shape = (1, 1, 300, 500)，grad.ndim = 4
		grad66：grad.shape = (500,)，grad.ndim = 1
		grad67：grad.shape = (3, 3, 500, 500)，grad.ndim = 4
		grad68：grad.shape = (500,)，grad.ndim = 1
		grad69：grad.shape = (500,)，grad.ndim = 1
		grad70：grad.shape = (500,)，grad.ndim = 1
		grad71：grad.shape = (3, 3, 500, 500)，grad.ndim = 4
		grad72：grad.shape = (500,)，grad.ndim = 1
		grad73：grad.shape = (500,)，grad.ndim = 1
		grad74：grad.shape = (500,)，grad.ndim = 1
		grad75：grad.shape = (500, 100)，grad.ndim = 2
		grad76：grad.shape = (100,)，grad.ndim = 1
	梯度下降步骤-->optimizer.apply_gradients(zip(grads, resnet18_network.trainable_variables))：开始
	梯度下降步骤-->optimizer.apply_gradients(zip(grads, resnet18_network.trainable_variables))：结束

epoch_no = 1, batch_step_no = 2，X_batch.shpae = (500, 32, 32, 3)，Y_batch.shpae = (500,)------------type(X_batch) = <class 'tensorflow.python.framework.ops.EagerTensor'>，type(Y_batch) = <class 'tensorflow.python.framework.ops.EagerTensor'>
	Y_train_one_hot.shpae = (500, 100)
	out_logits.shape = (500, 100)
	MSE_Loss.shape = (500,)
	求均值后：MSE_Loss.shape = ()
	第1个epoch-->第2个batch step的初始时的：MSE_Loss = 5.222436428070068
	梯度下降步骤-->optimizer.apply_gradients(zip(grads, resnet18_network.trainable_variables))：开始
	梯度下降步骤-->optimizer.apply_gradients(zip(grads, resnet18_network.trainable_variables))：结束
...
...
...
...
...
epoch_no = 1, batch_step_no = 100，X_batch.shpae = (500, 32, 32, 3)，Y_batch.shpae = (500,)------------type(X_batch) = <class 'tensorflow.python.framework.ops.EagerTensor'>，type(Y_batch) = <class 'tensorflow.python.framework.ops.EagerTensor'>
	Y_train_one_hot.shpae = (500, 100)
	out_logits.shape = (500, 100)
	MSE_Loss.shape = (500,)
	求均值后：MSE_Loss.shape = ()
	第1个epoch-->第100个batch step的初始时的：MSE_Loss = 4.207188129425049
	梯度下降步骤-->optimizer.apply_gradients(zip(grads, resnet18_network.trainable_variables))：开始
	梯度下降步骤-->optimizer.apply_gradients(zip(grads, resnet18_network.trainable_variables))：结束

++++++++++++++++++++++++++++++++++++++++++++第1轮Epoch-->Training 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++第1轮Epoch-->Evluation 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 1, batch_step_no = 1，X_batch.shpae = (500, 32, 32, 3)，Y_batch.shpae = (500,)
	out_logits.shape = (500, 100)
	is_correct_count = 18.0

epoch_no = 1, batch_step_no = 2，X_batch.shpae = (500, 32, 32, 3)，Y_batch.shpae = (500,)
	out_logits.shape = (500, 100)
	is_correct_count = 27.0
...
...
...

epoch_no = 1, batch_step_no = 20，X_batch.shpae = (500, 32, 32, 3)，Y_batch.shpae = (500,)
	out_logits.shape = (500, 100)
	is_correct_count = 26.0

total_correct = 454---total_num = 10000
第1轮Epoch迭代的准确度： acc = 0.0454
++++++++++++++++++++++++++++++++++++++++++++第1轮Epoch-->Evluation 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
利用整体数据集进行模型的第1轮Epoch迭代结束:**********************************************************************************************************************************

Process finished with exit code 0

3.2 自定义ResNet神经网络【Pytorch】

import torch
from torch.utils.data import DataLoader
from torchvision import datasets
from torchvision import transforms
from torch import nn, optim
from torch.nn import functional as F


# 两层的残差学习单元 BasicBlock [(3×3)-->(3×3)]形状，如果是三层的BasicBlock，形状则为：[(1×1)-->(3×3)-->(1×1)]
# filter_count_in≠filter_count_out时，则通过该层Layer后的FeatureMap的大小改变，identity层也需要reshape
class BasicBlock(nn.Module):
    def __init__(self, filter_count_in, filter_count_out, stride=1):
        super(BasicBlock, self).__init__()
        # we add stride support for resbok, which is distinct from tutorials.
        self.conv1 = nn.Conv2d(in_channels=filter_count_in, out_channels=filter_count_out, kernel_size=3, stride=stride, padding=1)
        self.bn1 = nn.BatchNorm2d(filter_count_out)
        self.conv2 = nn.Conv2d(filter_count_out, filter_count_out, kernel_size=3, stride=1, padding=1)
        self.bn2 = nn.BatchNorm2d(filter_count_out)
        self.identity = nn.Sequential()
        if filter_count_in != filter_count_out:  # 将输入值x的维度调整为和F(x)的输出维度保持一致  [b, filter_count_in, h, w] => [b, filter_count_out, h, w]
            self.identity = nn.Sequential(
                nn.Conv2d(filter_count_in, filter_count_out, kernel_size=1, stride=stride),
                nn.BatchNorm2d(filter_count_out)
            )

    def forward(self, input):
        x = self.conv1(input)
        x = self.bn1(x)
        x = F.relu(x)
        x = self.conv2(x)
        F_out = self.bn2(x)
        # short cut
        identity_out = self.identity(input)  # 调整input的维度与F_out保持一致，然后才能和F_out相加：[b, ch_in, h, w] => [b, ch_out, h, w]
        H_out = identity_out + F_out
        H_out = F.relu(H_out)

        return H_out


# 由多个BasicBlock组成的ResidualBlock
class ResidualBlock:
    def __init__(self, filter_count_in, filter_count_out, residualBlock_size=1, stride=1):
        self.filter_count_in = filter_count_in
        self.filter_count_out = filter_count_out
        self.residualBlock_size = residualBlock_size
        self.stride = stride

    def __call__(self):
        basic_block_stride_eq = BasicBlock(self.filter_count_in, self.filter_count_in, stride=1)  # stride = 1 时的BasicBlock H(x)=x+F(X)，identity_layer层的输出为直接返回输入
        basic_block_stride_not_eq = BasicBlock(self.filter_count_in, self.filter_count_out, stride=self.stride)  # stride != 1 时的BasicBlock H(x)=x+F(X)，identity_layer进行SubSampling
        residualBlock = nn.Sequential()
        for _ in range(0, self.residualBlock_size - 1):  # 其余的BasicBlock都是 filter_count_in == filter_count_out 时的BasicBlock
            residualBlock.add_module('basic_block_stride_eq', basic_block_stride_eq)
        residualBlock.add_module('basic_block_stride_not_eq', basic_block_stride_not_eq)  # 有一个BasicBlock必须是 filter_count_in != filter_count_out 时的BasicBlock
        return residualBlock


# 由多个ResidualBlock组成的ResidualNet
class ResNet18(nn.Module):
    def __init__(self):
        super(ResNet18, self).__init__()
        self.conv1 = nn.Sequential(
            nn.Conv2d(3, 64, kernel_size=3, stride=3, padding=0),
            nn.BatchNorm2d(64)
        )
        # followed 4 ResidualBlock
        self.residualBlock1 = ResidualBlock(filter_count_in=64, filter_count_out=128, residualBlock_size=2, stride=2)()  # [b, 64, h, w] => [b, 128, h ,w]
        self.residualBlock2 = ResidualBlock(filter_count_in=128, filter_count_out=256, residualBlock_size=2, stride=2)()  # [b, 128, h, w] => [b, 256, h, w]
        self.residualBlock3 = ResidualBlock(filter_count_in=256, filter_count_out=512, residualBlock_size=2, stride=2)()  # [b, 256, h, w] => [b, 512, h, w]
        self.residualBlock4 = ResidualBlock(filter_count_in=512, filter_count_out=512, residualBlock_size=2, stride=2)()  # [b, 512, h, w] => [b, 1024, h, w]
        self.outlayer = nn.Linear(512 * 1 * 1, 10)

    def forward(self, X):
        X = F.relu(self.conv1(X))
        # [b, 64, h, w] => [b, 1024, h, w]
        X = self.residualBlock1(X)
        X = self.residualBlock2(X)
        X = self.residualBlock3(X)
        X = self.residualBlock4(X)  # [b, 512, 2, 2]
        X = F.adaptive_avg_pool2d(X, [1, 1])  # [b, 512, 2, 2] => [b, 512, 1, 1]
        X = X.view(X.size(0), -1)  # [b, 512, 1, 1] => [b, 512]
        X = self.outlayer(X)  # [b, 512] => [b, 10]

        return X


def main():
    batch_size = 200
    # 一、获取cifar10训练数据集
    cifar_train = datasets.CIFAR10('cifar', True, transform=transforms.Compose([
        transforms.Resize((32, 32)),
        transforms.ToTensor(),
        transforms.Normalize(mean=[0.485, 0.456, 0.406],
                             std=[0.229, 0.224, 0.225])
    ]), download=True)
    cifar_train = DataLoader(cifar_train, batch_size=batch_size, shuffle=True)
    cifar_test = datasets.CIFAR10('cifar', False, transform=transforms.Compose([
        transforms.Resize((32, 32)),
        transforms.ToTensor(),
        transforms.Normalize(mean=[0.485, 0.456, 0.406],
                             std=[0.229, 0.224, 0.225])
    ]), download=True)
    cifar_test = DataLoader(cifar_test, batch_size=batch_size, shuffle=True)

    # 二、设置GPU
    device = torch.device('cuda')

    # 三、实例化ResNet18神经网络模型
    model = ResNet18().to(device)
    # Find total parameters and trainable parameters
    total_params = sum(p.numel() for p in model.parameters())
    print(f'{total_params:,} total parameters.')
    total_trainable_params = sum(
        p.numel() for p in model.parameters() if p.requires_grad)
    print(f'{total_trainable_params:,} training parameters.')
    print('model = {0}\n'.format(model))

    # 四、实例化损失函数
    criteon = nn.CrossEntropyLoss().to(device)

    # 五、梯度下降优化器设置
    optimizer = optim.Adam(model.parameters(), lr=1e-3)

    # 六、训练
    for epoch in range(3):
        # **********************************************************训练**********************************************************
        print('**************************训练模式：开始**************************')
        model.train()  # 切换至训练模式
        for batch_index, (X_batch, Y_batch) in enumerate(cifar_train):
            X_batch, Y_batch = X_batch.to(device), Y_batch.to(device)
            out_logits = model(X_batch)
            loss = criteon(out_logits, Y_batch)
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
            if batch_index % 100 == 0:
                print('epoch = {0}, batch_index = {1}, loss.item() = {2}'.format(epoch, batch_index, loss.item()))
        print('**************************训练模式：结束**************************')
        # **********************************************************模型评估**********************************************************
        print('**************************验证模式：开始**************************')
        model.eval()  # 切换至验证模式
        with torch.no_grad():  # torch.no_grad()所包裹的部分不需要参与反向传播
            # test
            total_correct = 0
            total_num = 0
            for batch_index, (X_batch, Y_batch) in enumerate(cifar_test):
                X_batch, Y_batch = X_batch.to(device), Y_batch.to(device)
                out_logits = model(X_batch)
                out_pred = out_logits.argmax(dim=1)
                correct = torch.eq(out_pred, Y_batch).float().sum().item()
                total_correct += correct
                total_num += X_batch.size(0)
                acc = total_correct / total_num
                if batch_index % 100 == 0:
                    print('epoch = {0}, batch_index = {1}, test acc = {2}'.format(epoch, batch_index, acc))
        print('**************************验证模式：结束**************************')


if __name__ == '__main__':
    main()

打印结果：

Files already downloaded and verified
Files already downloaded and verified
15,826,314 total parameters.
15,826,314 training parameters.
model = ResNet18(
  (conv1): Sequential(
    (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(3, 3))
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  )
  (residualBlock1): Sequential(
    (basic_block_stride_eq): BasicBlock(
      (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
    (basic_block_stride_not_eq): BasicBlock(
      (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (identity): Sequential(
        (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2))
        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
  )
  (residualBlock2): Sequential(
    (basic_block_stride_eq): BasicBlock(
      (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
    (basic_block_stride_not_eq): BasicBlock(
      (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (identity): Sequential(
        (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2))
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
  )
  (residualBlock3): Sequential(
    (basic_block_stride_eq): BasicBlock(
      (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
    (basic_block_stride_not_eq): BasicBlock(
      (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (identity): Sequential(
        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2))
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
  )
  (residualBlock4): Sequential(
    (basic_block_stride_eq): BasicBlock(
      (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
    (basic_block_stride_not_eq): BasicBlock(
      (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (outlayer): Linear(in_features=512, out_features=10, bias=True)
)

**************************训练模式：开始**************************
epoch = 0, batch_index = 0, loss.item() = 2.784912109375
epoch = 0, batch_index = 100, loss.item() = 1.2591865062713623
epoch = 0, batch_index = 200, loss.item() = 1.2418736219406128
**************************训练模式：结束**************************
**************************验证模式：开始**************************
epoch = 0, batch_index = 0, test acc = 0.515
**************************验证模式：结束**************************
**************************训练模式：开始**************************
epoch = 1, batch_index = 0, loss.item() = 1.0537413358688354
epoch = 1, batch_index = 100, loss.item() = 1.088006615638733
epoch = 1, batch_index = 200, loss.item() = 1.0332653522491455
**************************训练模式：结束**************************
**************************验证模式：开始**************************
epoch = 1, batch_index = 0, test acc = 0.635
**************************验证模式：结束**************************
**************************训练模式：开始**************************
epoch = 2, batch_index = 0, loss.item() = 0.9080470204353333
epoch = 2, batch_index = 100, loss.item() = 0.7950635552406311
epoch = 2, batch_index = 200, loss.item() = 0.7487978339195251
**************************训练模式：结束**************************
**************************验证模式：开始**************************
epoch = 2, batch_index = 0, test acc = 0.64
**************************验证模式：结束**************************

Process finished with exit code 0

3.3 自定义ResNet18&自定义数据集【Pytorch】

import torch
from torch.utils.data import DataLoader
from torch import nn, optim
from torch.nn import functional as F
import visdom
import csv
import glob
import os
import random
from PIL import Image
from torch.utils.data import Dataset  # 自定义数据集的父类
from torchvision import transforms

torch.manual_seed(1234)  # 随机种子
device = torch.device('cuda')  # 设置GPU


# =============================================================================Pokemon自定义数据集：开始=============================================================================
class Pokemon(Dataset):
    # root表示数据位置；resize表示数据输出的size；mode表示训练模式/测试模式
    def __init__(self, root, resize, mode):
        super(Pokemon, self).__init__()
        self.root = root
        self.resize = resize

        # 给各个类型进行编号
        self.name2label = {}  # {'bulbasaur': 0, 'charmander': 1, 'mewtwo': 2, 'pikachu': 3, 'squirtle': 4}
        for name in sorted(os.listdir(os.path.join(root))):
            if not os.path.isdir(os.path.join(root, name)):  # 过滤掉不是文件夹的文件
                continue
            self.name2label[name] = len(self.name2label.keys())
        print('self.name2label = {0}'.format(self.name2label))  # {'bulbasaur': 0, 'charmander': 1, 'mewtwo': 2, 'pikachu': 3, 'squirtle': 4}

        # 读取已保存的图片+标签数据集
        self.img_paths, self.labels = self.load_csv('img_paths.csv')  # 数据对(img_path + image_label)：img_paths, labels
        # 对数据集根据当前模式进行裁剪
        if mode == 'train':  # 60%
            self.img_paths = self.img_paths[:int(0.6 * len(self.img_paths))]
            self.labels = self.labels[:int(0.6 * len(self.labels))]
        elif mode == 'val':  # 20% = 60%->80%
            self.img_paths = self.img_paths[int(0.6 * len(self.img_paths)):int(0.8 * len(self.img_paths))]
            self.labels = self.labels[int(0.6 * len(self.labels)):int(0.8 * len(self.labels))]
        else:  # 20% = 80%->100%
            self.img_paths = self.img_paths[int(0.8 * len(self.img_paths)):]
            self.labels = self.labels[int(0.8 * len(self.labels)):]

    def load_csv(self, filename):
        # 1、如果没有csv文件，则创建该csv文件
        if not os.path.exists(os.path.join(self.root, filename)):
            img_paths = []  # 把所有图片的path都保存在该list中，各个图片的label可以从path推断出来，所有没有单独保存。
            for name in self.name2label.keys():
                img_paths += glob.glob(os.path.join(self.root, name, '*.png'))  # 'pokemon\\mewtwo\\00001.png
                img_paths += glob.glob(os.path.join(self.root, name, '*.jpg'))
                img_paths += glob.glob(os.path.join(self.root, name, '*.jpeg'))
                img_paths += glob.glob(os.path.join(self.root, name, '*.gif'))
            print('len(img_paths) = {0}, img_paths = {1}'.format(len(img_paths), img_paths))  # len(img_paths) = 1168, img_paths = ['pokemon\\bulbasaur\\00000000.png','pokemon\\bulbasaur\\00000001.png',...]
            random.shuffle(img_paths)  # 打乱list中的图片顺序
            # 向csv文件保存图片的path+label
            with open(os.path.join(self.root, filename), mode='w', newline='') as f:
                writer = csv.writer(f)
                for img_path in img_paths:  # 'pokemon\\bulbasaur\\00000000.png'
                    name = img_path.split(os.sep)[-2]
                    label = self.name2label[name]
                    writer.writerow([img_path, label])  # 'pokemon\\bulbasaur\\00000000.png', 0
                print('writen into csv file:', filename)
        # 2、如果已经有csv文件，则读取该csv文件
        img_paths, labels = [], []
        with open(os.path.join(self.root, filename)) as f:
            reader = csv.reader(f)
            for row in reader:
                img_path, label = row  # 'pokemon\\bulbasaur\\00000000.png', 0
                label = int(label)
                img_paths.append(img_path)
                labels.append(label)
        assert len(img_paths) == len(labels)
        return img_paths, labels

    def __len__(self):
        return len(self.img_paths)

    def denormalize(self, x_hat):
        mean = [0.485, 0.456, 0.406]
        std = [0.229, 0.224, 0.225]
        # x_hat = (x-mean)/std
        # x = x_hat*std = mean
        # x: [c, h, w]
        # mean: [3] => [3, 1, 1]
        mean = torch.tensor(mean).unsqueeze(1).unsqueeze(1)
        std = torch.tensor(std).unsqueeze(1).unsqueeze(1)
        print('denormalize-->mean.shape = {0}, std.shape = {1}'.format(mean.shape, std.shape))
        x = x_hat * std + mean

        return x

    def __getitem__(self, img_idx):  # img_idx~[0~len(img_paths)]
        img_path, label = self.img_paths[img_idx], self.labels[img_idx]  # img_path: 'pokemon\\bulbasaur\\00000000.png'；label: 0
        transform = transforms.Compose([
            lambda x: Image.open(x).convert('RGB'),  # string path --> image data
            transforms.Resize((int(self.resize * 1.25), int(self.resize * 1.25))),
            transforms.RandomRotation(15),  # rotate如果比较大的话，可能会造成网络不收敛
            transforms.CenterCrop(self.resize),
            transforms.ToTensor(),
            transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])  # 该数值是实践中统计的效果比较好的值
        ])
        img = transform(img_path)
        label = torch.tensor(label)

        return img, label


# =============================================================================Pokemon自定义数据集：结束=============================================================================


# =============================================================================ResNet18神经网络：开始=============================================================================
# 两层的残差学习单元 BasicBlock [(3×3)-->(3×3)]形状，如果是三层的BasicBlock，形状则为：[(1×1)-->(3×3)-->(1×1)]
# filter_count_in≠filter_count_out时，则通过该层Layer后的FeatureMap的大小改变，identity层也需要reshape
class BasicBlock(nn.Module):
    def __init__(self, filter_count_in, filter_count_out, stride=1):
        super(BasicBlock, self).__init__()
        self.filter_count_in = filter_count_in
        self.filter_count_out = filter_count_out
        self.stride = stride
        # we add stride support for resbok, which is distinct from tutorials.
        self.conv1 = nn.Conv2d(in_channels=filter_count_in, out_channels=filter_count_out, kernel_size=3, stride=stride, padding=1)
        self.bn1 = nn.BatchNorm2d(filter_count_out)
        self.conv2 = nn.Conv2d(filter_count_out, filter_count_out, kernel_size=3, stride=1, padding=1)
        self.bn2 = nn.BatchNorm2d(filter_count_out)
        self.identity = nn.Sequential()
        if filter_count_in != filter_count_out:  # 将输入值x的维度调整为和F(x)的输出维度保持一致  [b, filter_count_in, h, w] => [b, filter_count_out, h, w]
            self.identity = nn.Sequential(
                nn.Conv2d(filter_count_in, filter_count_out, kernel_size=1, stride=stride),
                nn.BatchNorm2d(filter_count_out)
            )

    def forward(self, input):
        x = self.conv1(input)
        x = self.bn1(x)
        x = F.relu(x)
        x = self.conv2(x)
        F_out = self.bn2(x)
        # short cut
        identity_out = self.identity(input)  # 调整input的维度与F_out保持一致，然后才能和F_out相加：[b, ch_in, h, w] => [b, ch_out, h, w]
        # print('stride = {0}，filter_count_in = {1}，filter_count_out = {2}，F_out.shape = {3}，identity_out.shape = {4}'.format(self.stride, self.filter_count_in, self.filter_count_out, F_out.shape, identity_out.shape))
        H_out = identity_out + F_out
        H_out = F.relu(H_out)

        return H_out


# 由多个BasicBlock组成的ResidualBlock
class ResidualBlock:
    def __init__(self, filter_count_in, filter_count_out, residualBlock_size=1, stride=1):
        self.filter_count_in = filter_count_in
        self.filter_count_out = filter_count_out
        self.residualBlock_size = residualBlock_size
        self.stride = stride

    def __call__(self):
        basic_block_stride_eq = BasicBlock(self.filter_count_in, self.filter_count_in, stride=1)  # stride = 1 时的BasicBlock H(x)=x+F(X)，identity_layer层的输出为直接返回输入
        basic_block_stride_not_eq = BasicBlock(self.filter_count_in, self.filter_count_out, stride=self.stride)  # stride != 1 时的BasicBlock H(x)=x+F(X)，identity_layer进行SubSampling
        residualBlock = nn.Sequential()
        for _ in range(0, self.residualBlock_size - 1):  # 其余的BasicBlock都是 filter_count_in == filter_count_out 时的BasicBlock
            residualBlock.add_module('basic_block_stride_eq', basic_block_stride_eq)
        residualBlock.add_module('basic_block_stride_not_eq', basic_block_stride_not_eq)  # 有一个BasicBlock必须是 filter_count_in != filter_count_out 时的BasicBlock
        return residualBlock


# 由多个ResidualBlock组成的ResidualNet
class ResNet18(nn.Module):
    def __init__(self, num_class):  # num_class 表示最终所有分类数量
        super(ResNet18, self).__init__()
        self.conv1 = nn.Sequential(
            nn.Conv2d(3, 64, kernel_size=3, stride=3, padding=0),
            nn.BatchNorm2d(64)
        )
        # followed 4 ResidualBlock
        self.residualBlock1 = ResidualBlock(filter_count_in=64, filter_count_out=128, residualBlock_size=2, stride=2)()  # [b, 64, h, w] => [b, 128, h ,w]
        self.residualBlock2 = ResidualBlock(filter_count_in=128, filter_count_out=256, residualBlock_size=2, stride=2)()  # [b, 128, h, w] => [b, 256, h, w]
        self.residualBlock3 = ResidualBlock(filter_count_in=256, filter_count_out=512, residualBlock_size=2, stride=2)()  # [b, 256, h, w] => [b, 512, h, w]
        self.residualBlock4 = ResidualBlock(filter_count_in=512, filter_count_out=512, residualBlock_size=2, stride=1)()  # [b, 512, h, w] => [b, 1024, h, w]
        self.outlayer = nn.Linear(512 * 1 * 1, num_class)

    def forward(self, X):
        X = F.relu(self.conv1(X))
        # [b, 64, h, w] => [b, 1024, h, w]
        X = self.residualBlock1(X)
        X = self.residualBlock2(X)
        X = self.residualBlock3(X)
        X = self.residualBlock4(X)  # [b, 512, 2, 2]
        X = F.adaptive_avg_pool2d(X, [1, 1])  # [b, 512, 2, 2] => [b, 512, 1, 1]
        X = X.view(X.size(0), -1)  # [b, 512, 1, 1] => [b, 512]
        X = self.outlayer(X)  # [b, 512] => [b, 5]

        return X


# =============================================================================ResNet18神经网络：结束=============================================================================

# =============================================================================训练主体：开始=============================================================================
batch_size = 32
viz = visdom.Visdom()  # 在控制台开启Visdom：python -m visdom.server
global_step = 0

# 一、获取Pokemon训练数据集
train_db = Pokemon('pokemon', 224, mode='train')
val_db = Pokemon('pokemon', 224, mode='val')
test_db = Pokemon('pokemon', 224, mode='test')
train_loader = DataLoader(train_db, batch_size=batch_size, shuffle=True, num_workers=0)  # num_workers表示开启的线程数量
val_loader = DataLoader(val_db, batch_size=batch_size, num_workers=0)
test_loader = DataLoader(test_db, batch_size=batch_size, num_workers=0)

# 三、实例化ResNet18神经网络模型
model = ResNet18(5).to(device)
# Find total parameters and trainable parameters
total_params = sum(p.numel() for p in model.parameters())
print(f'{total_params:,} total parameters.')
total_trainable_params = sum(
    p.numel() for p in model.parameters() if p.requires_grad)
print(f'{total_trainable_params:,} training parameters.')
print('model = {0}\n'.format(model))

# 四、实例化损失函数
criteon = nn.CrossEntropyLoss().to(device)

# 五、梯度下降优化器设置
optimizer = optim.Adam(model.parameters(), lr=1e-3)


def train_epoch(epoch_no):
    global global_step
    print('++++++++++++++++++++++++++++++++++++++++++++第{0}轮Epoch-->Training 阶段：开始++++++++++++++++++++++++++++++++++++++++++++'.format(epoch_no))
    model.train()  # 切换至训练模式
    for batch_index, (X_batch, Y_batch) in enumerate(train_loader):
        X_batch, Y_batch = X_batch.to(device), Y_batch.to(device)
        out_logits = model(X_batch)
        loss = criteon(out_logits, Y_batch)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        viz.line([loss.item()], [global_step], win='loss', update='append')
        global_step += 1
        if batch_index % 5 == 0:
            print('epoch_no = {0}, batch_index = {1}, loss.item() = {2}'.format(epoch_no, batch_index, loss.item()))
    print('++++++++++++++++++++++++++++++++++++++++++++第{0}轮Epoch-->Training 阶段：结束++++++++++++++++++++++++++++++++++++++++++++'.format(epoch_no))


def evalute(epoch_no, loader):
    print('++++++++++++++++++++++++++++++++++++++++++++第{0}轮Epoch-->Evluation 阶段：开始++++++++++++++++++++++++++++++++++++++++++++'.format(epoch_no))
    model.eval()
    with torch.no_grad():
        total_correct = 0
        total_num = 0
        for batch_index, (X_batch, Y_batch) in enumerate(loader):
            X_batch, Y_batch = X_batch.to(device), Y_batch.to(device)
            out_logits = model(X_batch)
            out_pred = out_logits.argmax(dim=1)
            correct = torch.eq(out_pred, Y_batch).float().sum().item()
            total_correct += correct
            total_num += X_batch.size(0)
            val_acc = total_correct / total_num
            viz.line([val_acc], [global_step], win='val_acc', update='append')
            if batch_index % 5 == 0:
                print('epoch_no = {0}, batch_index = {1}, val_acc = {2}'.format(epoch_no, batch_index, val_acc))
    print('++++++++++++++++++++++++++++++++++++++++++++第{0}轮Epoch-->Evluation 阶段：结束++++++++++++++++++++++++++++++++++++++++++++'.format(epoch_no))
    return val_acc


def main():
    epoch_count = 4  # epoch_count为整体数据集迭代梯度下降次数
    best_acc, best_epoch = 0, 0
    viz.line([0], [-1], win='loss', opts=dict(title='loss'))
    viz.line([0], [-1], win='val_acc', opts=dict(title='val_acc'))
    for epoch_no in range(1, epoch_count + 1):
        print('\n\n利用整体数据集进行模型的第{0}轮Epoch迭代开始:**********************************************************************************************************************************'.format(epoch_no))
        train_epoch(epoch_no)  # 训练
        val_acc = evalute(epoch_no, val_loader)  # 验证
        if val_acc > best_acc:
            best_epoch = epoch_no
            best_acc = val_acc
            torch.save(model.state_dict(), 'best.mdl')
        print('epoch = {0}, best_epoch = {1}, best_acc = {2}'.format(epoch_no, best_epoch, best_acc))
        print('**************************验证模式：结束**************************')
        print('利用整体数据集进行模型的第{0}轮Epoch迭代结束:**********************************************************************************************************************************'.format(epoch_no))
    print('best acc:', best_acc, 'best epoch:', best_epoch)
    model.load_state_dict(torch.load('best.mdl'))
    print('loaded from ckpt!')
    test_acc = evalute(best_epoch, test_loader)  # 测试
    print('test acc:', test_acc)


if __name__ == '__main__':
    main()

# =============================================================================训练主体：结束=============================================================================

打印结果：

Setting up a new session...
self.name2label = {'bulbasaur': 0, 'charmander': 1, 'mewtwo': 2, 'pikachu': 3, 'squirtle': 4}
self.name2label = {'bulbasaur': 0, 'charmander': 1, 'mewtwo': 2, 'pikachu': 3, 'squirtle': 4}
self.name2label = {'bulbasaur': 0, 'charmander': 1, 'mewtwo': 2, 'pikachu': 3, 'squirtle': 4}
15,823,749 total parameters.
15,823,749 training parameters.
model = ResNet18(
  (conv1): Sequential(
    (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(3, 3))
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  )
  (residualBlock1): Sequential(
    (basic_block_stride_eq): BasicBlock(
      (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (identity): Sequential()
    )
    (basic_block_stride_not_eq): BasicBlock(
      (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (identity): Sequential(
        (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2))
        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
  )
  (residualBlock2): Sequential(
    (basic_block_stride_eq): BasicBlock(
      (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (identity): Sequential()
    )
    (basic_block_stride_not_eq): BasicBlock(
      (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (identity): Sequential(
        (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2))
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
  )
  (residualBlock3): Sequential(
    (basic_block_stride_eq): BasicBlock(
      (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (identity): Sequential()
    )
    (basic_block_stride_not_eq): BasicBlock(
      (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (identity): Sequential(
        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2))
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
  )
  (residualBlock4): Sequential(
    (basic_block_stride_eq): BasicBlock(
      (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (identity): Sequential()
    )
    (basic_block_stride_not_eq): BasicBlock(
      (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (identity): Sequential()
    )
  )
  (outlayer): Linear(in_features=512, out_features=5, bias=True)
)



利用整体数据集进行模型的第1轮Epoch迭代开始:**********************************************************************************************************************************
++++++++++++++++++++++++++++++++++++++++++++第1轮Epoch-->Training 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 1, batch_index = 0, loss.item() = 1.939097285270691
epoch_no = 1, batch_index = 5, loss.item() = 1.332801342010498
epoch_no = 1, batch_index = 10, loss.item() = 1.3339236974716187
epoch_no = 1, batch_index = 15, loss.item() = 0.44973278045654297
epoch_no = 1, batch_index = 20, loss.item() = 0.4216762185096741
++++++++++++++++++++++++++++++++++++++++++++第1轮Epoch-->Training 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++第1轮Epoch-->Evluation 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 1, batch_index = 0, val_acc = 0.6875
epoch_no = 1, batch_index = 5, val_acc = 0.7395833333333334
++++++++++++++++++++++++++++++++++++++++++++第1轮Epoch-->Evluation 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
epoch = 1, best_epoch = 1, best_acc = 0.7478632478632479
**************************验证模式：结束**************************
利用整体数据集进行模型的第1轮Epoch迭代结束:**********************************************************************************************************************************


利用整体数据集进行模型的第2轮Epoch迭代开始:**********************************************************************************************************************************
++++++++++++++++++++++++++++++++++++++++++++第2轮Epoch-->Training 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 2, batch_index = 0, loss.item() = 0.5493289232254028
epoch_no = 2, batch_index = 5, loss.item() = 0.6154159307479858
epoch_no = 2, batch_index = 10, loss.item() = 0.6554363965988159
epoch_no = 2, batch_index = 15, loss.item() = 0.4766008257865906
epoch_no = 2, batch_index = 20, loss.item() = 0.45220986008644104
++++++++++++++++++++++++++++++++++++++++++++第2轮Epoch-->Training 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++第2轮Epoch-->Evluation 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 2, batch_index = 0, val_acc = 0.71875
epoch_no = 2, batch_index = 5, val_acc = 0.8020833333333334
++++++++++++++++++++++++++++++++++++++++++++第2轮Epoch-->Evluation 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
epoch = 2, best_epoch = 2, best_acc = 0.8076923076923077
**************************验证模式：结束**************************
利用整体数据集进行模型的第2轮Epoch迭代结束:**********************************************************************************************************************************


利用整体数据集进行模型的第3轮Epoch迭代开始:**********************************************************************************************************************************
++++++++++++++++++++++++++++++++++++++++++++第3轮Epoch-->Training 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 3, batch_index = 0, loss.item() = 0.6022523641586304
epoch_no = 3, batch_index = 5, loss.item() = 0.5406889319419861
epoch_no = 3, batch_index = 10, loss.item() = 0.22856442630290985
epoch_no = 3, batch_index = 15, loss.item() = 0.5484329462051392
epoch_no = 3, batch_index = 20, loss.item() = 0.36236143112182617
++++++++++++++++++++++++++++++++++++++++++++第3轮Epoch-->Training 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++第3轮Epoch-->Evluation 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 3, batch_index = 0, val_acc = 0.84375
epoch_no = 3, batch_index = 5, val_acc = 0.859375
++++++++++++++++++++++++++++++++++++++++++++第3轮Epoch-->Evluation 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
epoch = 3, best_epoch = 3, best_acc = 0.8589743589743589
**************************验证模式：结束**************************
利用整体数据集进行模型的第3轮Epoch迭代结束:**********************************************************************************************************************************


利用整体数据集进行模型的第4轮Epoch迭代开始:**********************************************************************************************************************************
++++++++++++++++++++++++++++++++++++++++++++第4轮Epoch-->Training 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 4, batch_index = 0, loss.item() = 0.47427237033843994
epoch_no = 4, batch_index = 5, loss.item() = 0.30755600333213806
epoch_no = 4, batch_index = 10, loss.item() = 0.7977475523948669
epoch_no = 4, batch_index = 15, loss.item() = 0.3868430554866791
epoch_no = 4, batch_index = 20, loss.item() = 0.46423253417015076
++++++++++++++++++++++++++++++++++++++++++++第4轮Epoch-->Training 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++第4轮Epoch-->Evluation 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 4, batch_index = 0, val_acc = 0.90625
epoch_no = 4, batch_index = 5, val_acc = 0.8958333333333334
++++++++++++++++++++++++++++++++++++++++++++第4轮Epoch-->Evluation 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
epoch = 4, best_epoch = 4, best_acc = 0.8931623931623932
**************************验证模式：结束**************************
利用整体数据集进行模型的第4轮Epoch迭代结束:**********************************************************************************************************************************
best acc: 0.8931623931623932 best epoch: 4
loaded from ckpt!
++++++++++++++++++++++++++++++++++++++++++++第4轮Epoch-->Evluation 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 4, batch_index = 0, val_acc = 0.84375
epoch_no = 4, batch_index = 5, val_acc = 0.828125
++++++++++++++++++++++++++++++++++++++++++++第4轮Epoch-->Evluation 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
test acc: 0.8290598290598291

Process finished with exit code 0

3.4 迁移学习 & 预训练ResNet18 & 自定义数据集【Pytorch】

import torch
from torch.utils.data import DataLoader
from torch import nn, optim
from torch.nn import functional as F
import visdom
import csv
import glob
import os
import random
from PIL import Image
from torch.utils.data import Dataset  # 自定义数据集的父类
from torchvision import transforms
from torchvision.models import resnet18

torch.manual_seed(1234)  # 随机种子
device = torch.device('cuda')  # 设置GPU


# =============================================================================Pokemon自定义数据集：开始=============================================================================
class Pokemon(Dataset):
    # root表示数据位置；resize表示数据输出的size；mode表示训练模式/测试模式
    def __init__(self, root, resize, mode):
        super(Pokemon, self).__init__()
        self.root = root
        self.resize = resize

        # 给各个类型进行编号
        self.name2label = {}  # {'bulbasaur': 0, 'charmander': 1, 'mewtwo': 2, 'pikachu': 3, 'squirtle': 4}
        for name in sorted(os.listdir(os.path.join(root))):
            if not os.path.isdir(os.path.join(root, name)):  # 过滤掉不是文件夹的文件
                continue
            self.name2label[name] = len(self.name2label.keys())
        print('self.name2label = {0}'.format(self.name2label))  # {'bulbasaur': 0, 'charmander': 1, 'mewtwo': 2, 'pikachu': 3, 'squirtle': 4}

        # 读取已保存的图片+标签数据集
        self.img_paths, self.labels = self.load_csv('img_paths.csv')  # 数据对(img_path + image_label)：img_paths, labels
        # 对数据集根据当前模式进行裁剪
        if mode == 'train':  # 60%
            self.img_paths = self.img_paths[:int(0.6 * len(self.img_paths))]
            self.labels = self.labels[:int(0.6 * len(self.labels))]
        elif mode == 'val':  # 20% = 60%->80%
            self.img_paths = self.img_paths[int(0.6 * len(self.img_paths)):int(0.8 * len(self.img_paths))]
            self.labels = self.labels[int(0.6 * len(self.labels)):int(0.8 * len(self.labels))]
        else:  # 20% = 80%->100%
            self.img_paths = self.img_paths[int(0.8 * len(self.img_paths)):]
            self.labels = self.labels[int(0.8 * len(self.labels)):]

    def load_csv(self, filename):
        # 1、如果没有csv文件，则创建该csv文件
        if not os.path.exists(os.path.join(self.root, filename)):
            img_paths = []  # 把所有图片的path都保存在该list中，各个图片的label可以从path推断出来，所有没有单独保存。
            for name in self.name2label.keys():
                img_paths += glob.glob(os.path.join(self.root, name, '*.png'))  # 'pokemon\\mewtwo\\00001.png
                img_paths += glob.glob(os.path.join(self.root, name, '*.jpg'))
                img_paths += glob.glob(os.path.join(self.root, name, '*.jpeg'))
                img_paths += glob.glob(os.path.join(self.root, name, '*.gif'))
            print('len(img_paths) = {0}, img_paths = {1}'.format(len(img_paths), img_paths))  # len(img_paths) = 1168, img_paths = ['pokemon\\bulbasaur\\00000000.png','pokemon\\bulbasaur\\00000001.png',...]
            random.shuffle(img_paths)  # 打乱list中的图片顺序
            # 向csv文件保存图片的path+label
            with open(os.path.join(self.root, filename), mode='w', newline='') as f:
                writer = csv.writer(f)
                for img_path in img_paths:  # 'pokemon\\bulbasaur\\00000000.png'
                    name = img_path.split(os.sep)[-2]
                    label = self.name2label[name]
                    writer.writerow([img_path, label])  # 'pokemon\\bulbasaur\\00000000.png', 0
                print('writen into csv file:', filename)
        # 2、如果已经有csv文件，则读取该csv文件
        img_paths, labels = [], []
        with open(os.path.join(self.root, filename)) as f:
            reader = csv.reader(f)
            for row in reader:
                img_path, label = row  # 'pokemon\\bulbasaur\\00000000.png', 0
                label = int(label)
                img_paths.append(img_path)
                labels.append(label)
        assert len(img_paths) == len(labels)
        return img_paths, labels

    def __len__(self):
        return len(self.img_paths)

    def denormalize(self, x_hat):
        mean = [0.485, 0.456, 0.406]
        std = [0.229, 0.224, 0.225]
        # x_hat = (x-mean)/std
        # x = x_hat*std = mean
        # x: [c, h, w]
        # mean: [3] => [3, 1, 1]
        mean = torch.tensor(mean).unsqueeze(1).unsqueeze(1)
        std = torch.tensor(std).unsqueeze(1).unsqueeze(1)
        print('denormalize-->mean.shape = {0}, std.shape = {1}'.format(mean.shape, std.shape))
        x = x_hat * std + mean

        return x

    def __getitem__(self, img_idx):  # img_idx~[0~len(img_paths)]
        img_path, label = self.img_paths[img_idx], self.labels[img_idx]  # img_path: 'pokemon\\bulbasaur\\00000000.png'；label: 0
        transform = transforms.Compose([
            lambda x: Image.open(x).convert('RGB'),  # string path --> image data
            transforms.Resize((int(self.resize * 1.25), int(self.resize * 1.25))),
            transforms.RandomRotation(15),  # rotate如果比较大的话，可能会造成网络不收敛
            transforms.CenterCrop(self.resize),
            transforms.ToTensor(),
            transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])  # 该数值是实践中统计的效果比较好的值
        ])
        img = transform(img_path)
        label = torch.tensor(label)

        return img, label


# =============================================================================Pokemon自定义数据集：结束=============================================================================

class Flatten(nn.Module):
    def __init__(self):
        super(Flatten, self).__init__()

    def forward(self, x):
        shape = torch.prod(torch.tensor(x.shape[1:])).item()
        return x.view(-1, shape)

# =============================================================================训练主体：开始=============================================================================
batch_size = 32
viz = visdom.Visdom()  # 在控制台开启Visdom：python -m visdom.server
global_step = 0

# 一、获取Pokemon训练数据集
train_db = Pokemon('pokemon', 224, mode='train')
val_db = Pokemon('pokemon', 224, mode='val')
test_db = Pokemon('pokemon', 224, mode='test')
train_loader = DataLoader(train_db, batch_size=batch_size, shuffle=True, num_workers=0)  # num_workers表示开启的线程数量
val_loader = DataLoader(val_db, batch_size=batch_size, num_workers=0)
test_loader = DataLoader(test_db, batch_size=batch_size, num_workers=0)

# 三、实例化预训练ResNet18神经网络模型
trained_model = resnet18(pretrained=True)
model = nn.Sequential(*list(trained_model.children())[:-1],  # 提取已经训练好的resnet18模型的前17层，打散。[b, 512, 1, 1]
                      Flatten(),  # [b, 512, 1, 1] => [b, 512]
                      nn.Linear(512, 5)
                      ).to(device)
# Find total parameters and trainable parameters
total_params = sum(p.numel() for p in model.parameters())
print(f'{total_params:,} total parameters.')
total_trainable_params = sum(
    p.numel() for p in model.parameters() if p.requires_grad)
print(f'{total_trainable_params:,} training parameters.')
print('model = {0}\n'.format(model))

# 四、实例化损失函数
criteon = nn.CrossEntropyLoss().to(device)

# 五、梯度下降优化器设置
optimizer = optim.Adam(model.parameters(), lr=1e-3)


def train_epoch(epoch_no):
    global global_step
    print('++++++++++++++++++++++++++++++++++++++++++++第{0}轮Epoch-->Training 阶段：开始++++++++++++++++++++++++++++++++++++++++++++'.format(epoch_no))
    model.train()  # 切换至训练模式
    for batch_index, (X_batch, Y_batch) in enumerate(train_loader):
        X_batch, Y_batch = X_batch.to(device), Y_batch.to(device)
        out_logits = model(X_batch)
        loss = criteon(out_logits, Y_batch)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        viz.line([loss.item()], [global_step], win='loss', update='append')
        global_step += 1
        if batch_index % 5 == 0:
            print('epoch_no = {0}, batch_index = {1}, loss.item() = {2}'.format(epoch_no, batch_index, loss.item()))
    print('++++++++++++++++++++++++++++++++++++++++++++第{0}轮Epoch-->Training 阶段：结束++++++++++++++++++++++++++++++++++++++++++++'.format(epoch_no))


def evalute(epoch_no, loader):
    print('++++++++++++++++++++++++++++++++++++++++++++第{0}轮Epoch-->Evluation 阶段：开始++++++++++++++++++++++++++++++++++++++++++++'.format(epoch_no))
    model.eval()
    with torch.no_grad():
        total_correct = 0
        total_num = 0
        for batch_index, (X_batch, Y_batch) in enumerate(loader):
            X_batch, Y_batch = X_batch.to(device), Y_batch.to(device)
            out_logits = model(X_batch)
            out_pred = out_logits.argmax(dim=1)
            correct = torch.eq(out_pred, Y_batch).float().sum().item()
            total_correct += correct
            total_num += X_batch.size(0)
            val_acc = total_correct / total_num
            viz.line([val_acc], [global_step], win='val_acc', update='append')
            if batch_index % 5 == 0:
                print('epoch_no = {0}, batch_index = {1}, val_acc = {2}'.format(epoch_no, batch_index, val_acc))
    print('++++++++++++++++++++++++++++++++++++++++++++第{0}轮Epoch-->Evluation 阶段：结束++++++++++++++++++++++++++++++++++++++++++++'.format(epoch_no))
    return val_acc


def main():
    epoch_count = 4  # epoch_count为整体数据集迭代梯度下降次数
    best_acc, best_epoch = 0, 0
    viz.line([0], [-1], win='loss', opts=dict(title='loss'))
    viz.line([0], [-1], win='val_acc', opts=dict(title='val_acc'))
    for epoch_no in range(1, epoch_count + 1):
        print('\n\n利用整体数据集进行模型的第{0}轮Epoch迭代开始:**********************************************************************************************************************************'.format(epoch_no))
        train_epoch(epoch_no)  # 训练
        val_acc = evalute(epoch_no, val_loader)  # 验证
        if val_acc > best_acc:
            best_epoch = epoch_no
            best_acc = val_acc
            torch.save(model.state_dict(), 'best.mdl')
        print('epoch = {0}, best_epoch = {1}, best_acc = {2}'.format(epoch_no, best_epoch, best_acc))
        print('**************************验证模式：结束**************************')
        print('利用整体数据集进行模型的第{0}轮Epoch迭代结束:**********************************************************************************************************************************'.format(epoch_no))
    print('best acc:', best_acc, 'best epoch:', best_epoch)
    model.load_state_dict(torch.load('best.mdl'))
    print('loaded from ckpt!')
    test_acc = evalute(best_epoch, test_loader)  # 测试
    print('test acc:', test_acc)


if __name__ == '__main__':
    main()

# =============================================================================训练主体：结束=============================================================================

打印结果：

Setting up a new session...
self.name2label = {'bulbasaur': 0, 'charmander': 1, 'mewtwo': 2, 'pikachu': 3, 'squirtle': 4}
self.name2label = {'bulbasaur': 0, 'charmander': 1, 'mewtwo': 2, 'pikachu': 3, 'squirtle': 4}
self.name2label = {'bulbasaur': 0, 'charmander': 1, 'mewtwo': 2, 'pikachu': 3, 'squirtle': 4}
11,179,077 total parameters.
11,179,077 training parameters.
model = Sequential(
  (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
  (4): Sequential(
    (0): BasicBlock(
      (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
    (1): BasicBlock(
      (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (5): Sequential(
    (0): BasicBlock(
      (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (downsample): Sequential(
        (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (1): BasicBlock(
      (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (6): Sequential(
    (0): BasicBlock(
      (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (downsample): Sequential(
        (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (1): BasicBlock(
      (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (7): Sequential(
    (0): BasicBlock(
      (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (downsample): Sequential(
        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (1): BasicBlock(
      (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (8): AdaptiveAvgPool2d(output_size=(1, 1))
  (9): Flatten()
  (10): Linear(in_features=512, out_features=5, bias=True)
)



利用整体数据集进行模型的第1轮Epoch迭代开始:**********************************************************************************************************************************
++++++++++++++++++++++++++++++++++++++++++++第1轮Epoch-->Training 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 1, batch_index = 0, loss.item() = 1.664962887763977
epoch_no = 1, batch_index = 5, loss.item() = 0.4224851131439209
epoch_no = 1, batch_index = 10, loss.item() = 0.3056411147117615
epoch_no = 1, batch_index = 15, loss.item() = 0.6770390868186951
epoch_no = 1, batch_index = 20, loss.item() = 0.778434157371521
++++++++++++++++++++++++++++++++++++++++++++第1轮Epoch-->Training 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++第1轮Epoch-->Evluation 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 1, batch_index = 0, val_acc = 0.875
epoch_no = 1, batch_index = 5, val_acc = 0.7239583333333334
++++++++++++++++++++++++++++++++++++++++++++第1轮Epoch-->Evluation 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
epoch = 1, best_epoch = 1, best_acc = 0.7136752136752137
**************************验证模式：结束**************************
利用整体数据集进行模型的第1轮Epoch迭代结束:**********************************************************************************************************************************


利用整体数据集进行模型的第2轮Epoch迭代开始:**********************************************************************************************************************************
++++++++++++++++++++++++++++++++++++++++++++第2轮Epoch-->Training 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 2, batch_index = 0, loss.item() = 0.5391928553581238
epoch_no = 2, batch_index = 5, loss.item() = 0.641627848148346
epoch_no = 2, batch_index = 10, loss.item() = 0.28850072622299194
epoch_no = 2, batch_index = 15, loss.item() = 0.44357800483703613
epoch_no = 2, batch_index = 20, loss.item() = 0.15881212055683136
++++++++++++++++++++++++++++++++++++++++++++第2轮Epoch-->Training 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++第2轮Epoch-->Evluation 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 2, batch_index = 0, val_acc = 0.65625
epoch_no = 2, batch_index = 5, val_acc = 0.7447916666666666
++++++++++++++++++++++++++++++++++++++++++++第2轮Epoch-->Evluation 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
epoch = 2, best_epoch = 2, best_acc = 0.7478632478632479
**************************验证模式：结束**************************
利用整体数据集进行模型的第2轮Epoch迭代结束:**********************************************************************************************************************************


利用整体数据集进行模型的第3轮Epoch迭代开始:**********************************************************************************************************************************
++++++++++++++++++++++++++++++++++++++++++++第3轮Epoch-->Training 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 3, batch_index = 0, loss.item() = 0.11576351523399353
epoch_no = 3, batch_index = 5, loss.item() = 0.10171618312597275
epoch_no = 3, batch_index = 10, loss.item() = 0.19451947510242462
epoch_no = 3, batch_index = 15, loss.item() = 0.06140638515353203
epoch_no = 3, batch_index = 20, loss.item() = 0.049921028316020966
++++++++++++++++++++++++++++++++++++++++++++第3轮Epoch-->Training 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++第3轮Epoch-->Evluation 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 3, batch_index = 0, val_acc = 0.96875
epoch_no = 3, batch_index = 5, val_acc = 0.953125
++++++++++++++++++++++++++++++++++++++++++++第3轮Epoch-->Evluation 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
epoch = 3, best_epoch = 3, best_acc = 0.9487179487179487
**************************验证模式：结束**************************
利用整体数据集进行模型的第3轮Epoch迭代结束:**********************************************************************************************************************************


利用整体数据集进行模型的第4轮Epoch迭代开始:**********************************************************************************************************************************
++++++++++++++++++++++++++++++++++++++++++++第4轮Epoch-->Training 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 4, batch_index = 0, loss.item() = 0.08163614571094513
epoch_no = 4, batch_index = 5, loss.item() = 0.1351318359375
epoch_no = 4, batch_index = 10, loss.item() = 0.06922706216573715
epoch_no = 4, batch_index = 15, loss.item() = 0.051600512117147446
epoch_no = 4, batch_index = 20, loss.item() = 0.05538956820964813
++++++++++++++++++++++++++++++++++++++++++++第4轮Epoch-->Training 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++第4轮Epoch-->Evluation 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 4, batch_index = 0, val_acc = 0.90625
epoch_no = 4, batch_index = 5, val_acc = 0.9479166666666666
++++++++++++++++++++++++++++++++++++++++++++第4轮Epoch-->Evluation 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
epoch = 4, best_epoch = 3, best_acc = 0.9487179487179487
**************************验证模式：结束**************************
利用整体数据集进行模型的第4轮Epoch迭代结束:**********************************************************************************************************************************
best acc: 0.9487179487179487 best epoch: 3
loaded from ckpt!
++++++++++++++++++++++++++++++++++++++++++++第3轮Epoch-->Evluation 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
epoch_no = 3, batch_index = 0, val_acc = 0.96875
epoch_no = 3, batch_index = 5, val_acc = 0.921875
++++++++++++++++++++++++++++++++++++++++++++第3轮Epoch-->Evluation 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
test acc: 0.9230769230769231

Process finished with exit code 0

参考资料：
深度学习：残差网络（ResNet），理论及代码结构
CNN模型合集 | 8 ResNet v1/v2
Residual Networks (ResNet)
Resnet要解决的是什么问题
残差网络ResNet笔记
ResNet详解与分析
ResNet论文详解

你可能感兴趣的:(#,CV经典模型,深度学习,ResNet)

机器学习与深度学习间关系与区别 ℒℴѵℯ心·动ꦿ໊ོ꫞ 人工智能学习深度学习 python
一、机器学习概述定义机器学习（MachineLearning,ML）是一种通过数据驱动的方法，利用统计学和计算算法来训练模型，使计算机能够从数据中学习并自动进行预测或决策。机器学习通过分析大量数据样本，识别其中的模式和规律，从而对新的数据进行判断。其核心在于通过训练过程，让模型不断优化和提升其预测准确性。主要类型1.监督学习（SupervisedLearning）监督学习是指在训练数据集中包含输入
【iOS】MVC设计模式 Magnetic_h ios mvc 设计模式 objective-c 学习 ui
MVC前言如何设计一个程序的结构，这是一门专门的学问，叫做"架构模式"（architecturalpattern），属于编程的方法论。MVC模式就是架构模式的一种。它是Apple官方推荐的App开发架构，也是一般开发者最先遇到、最经典的架构。MVC各层controller层Controller/ViewController/VC（控制器）负责协调Model和View，处理大部分逻辑它将数据从Mod
LocalDateTime 转 String igotyback java 开发语言
importjava.time.LocalDateTime;importjava.time.format.DateTimeFormatter;publicclassMain{publicstaticvoidmain(String[]args){//获取当前时间LocalDateTimenow=LocalDateTime.now();//定义日期格式化器DateTimeFormatterformat
Linux下QT开发的动态库界面弹出操作（SDL2） 13jjyao QT类 qt 开发语言 sdl2 linux
需求：操作系统为linux，开发框架为qt，做成需带界面的qt动态库，调用方为java等非qt程序难点：调用方为java等非qt程序，也就是说调用方肯定不带QApplication::exec()，缺少了这个，QTimer等事件和QT创建的窗口将不能弹出(包括opencv也是不能弹出)；这与qt调用本身qt库是有本质的区别的思路：1.调用方缺QApplication::exec()，那么我们在接口
消息中间件有哪些常见类型 xmh-sxh-1314 java
消息中间件根据其设计理念和用途，可以大致分为以下几种常见类型：点对点消息队列（Point-to-PointMessagingQueues）：在这种模型中，消息被发送到特定的队列中，消费者从队列中取出并处理消息。队列中的消息只能被一个消费者消费，消费后即被删除。常见的实现包括IBM的MQSeries、RabbitMQ的部分使用场景等。适用于任务分发、负载均衡等场景。发布/订阅消息模型（Pub/Sub
LLM 词汇表落难Coder LLMs NLP 大语言模型大模型 llama 人工智能
Contextwindow“上下文窗口”是指语言模型在生成新文本时能够回溯和参考的文本量。这不同于语言模型训练时所使用的大量数据集，而是代表了模型的“工作记忆”。较大的上下文窗口可以让模型理解和响应更复杂和更长的提示，而较小的上下文窗口可能会限制模型处理较长提示或在长时间对话中保持连贯性的能力。Fine-tuning微调是使用额外的数据进一步训练预训练语言模型的过程。这使得模型开始表示和模仿微调数
将cmd中命令输出保存为txt文本文件落难Coder Windows cmd window
最近深度学习本地的训练中我们常常要在命令行中运行自己的代码，无可厚非，我们有必要保存我们的炼丹结果，但是复制命令行输出到txt是非常麻烦的，其实Windows下的命令行为我们提供了相应的操作。其基本的调用格式就是：运行指令>输出到的文件名称或者具体保存路径测试下，我打开cmd并且ping一下百度：pingwww.baidu.com>./data.txt看下相同目录下data.txt的输出：如果你再
探索OpenAI和LangChain的适配器集成：轻松切换模型提供商 nseejrukjhad langchain easyui 前端 python
#探索OpenAI和LangChain的适配器集成：轻松切换模型提供商##引言在人工智能和自然语言处理的世界中，OpenAI的模型提供了强大的能力。然而，随着技术的发展，许多人开始探索其他模型以满足特定需求。LangChain作为一个强大的工具，集成了多种模型提供商，通过提供适配器，简化了不同模型之间的转换。本篇文章将介绍如何使用LangChain的适配器与OpenAI集成，以便轻松切换模型提供商
使用Apify加载Twitter消息以进行微调的完整指南 nseejrukjhad twitter easyui 前端 python
#使用Apify加载Twitter消息以进行微调的完整指南##引言在自然语言处理领域，微调模型以适应特定任务是提升模型性能的常见方法。本文将介绍如何使用Apify从Twitter导出聊天信息，以便进一步进行微调。##主要内容###使用Apify导出推文首先，我们需要从Twitter导出推文。Apify可以帮助我们做到这一点。通过Apify的强大功能，我们可以批量抓取和导出数据，适用于各类应用场景。
如何部分格式化提示模板:LangChain中的高级技巧 nseejrukjhad langchain java 服务器 python
标题:如何部分格式化提示模板:LangChain中的高级技巧内容:如何部分格式化提示模板:LangChain中的高级技巧引言在使用大型语言模型(LLM)时,提示工程是一个关键环节。LangChain提供了强大的提示模板功能,让我们能更灵活地构建和管理提示。本文将介绍LangChain中一个高级特性-部分格式化提示模板,这个技巧可以让你的提示管理更加高效和灵活。什么是部分格式化提示模板?部分格式化提
人工智能时代，程序员如何保持核心竞争力？ jmoych 人工智能
随着AIGC（如chatgpt、midjourney、claude等）大语言模型接二连三的涌现，AI辅助编程工具日益普及，程序员的工作方式正在发生深刻变革。有人担心AI可能取代部分编程工作，也有人认为AI是提高效率的得力助手。面对这一趋势,程序员应该如何应对?是专注于某个领域深耕细作，还是广泛学习以适应快速变化的技术环境?又或者，我们是否应该将重点转向AI无法轻易替代的软技能？让我们一起探讨程序员
数据仓库——维度表一致性墨染丶eye 背诵数据仓库
数据仓库基础笔记思维导图已经整理完毕，完整连接为：数据仓库基础知识笔记思维导图维度一致性问题从逻辑层面来看，当一系列星型模型共享一组公共维度时，所涉及的维度称为一致性维度。当维度表存在不一致时，短期的成功难以弥补长期的错误。维度时确保不同过程中信息集成起来实现横向钻取货活动的关键。造成横向钻取失败的原因维度结构的差别，因为维度的差别，分析工作涉及的领域从简单到复杂，但是都是通过复杂的报表来弥补设计
【华为OD技术面试真题 - 技术面】- python八股文真题题库（1）算法大师华为od 面试 python
华为OD面试真题精选专栏：华为OD面试真题精选目录:2024华为OD面试手撕代码真题目录以及八股文真题目录文章目录华为OD面试真题精选1.数据预处理流程数据预处理的主要步骤工具和库2.介绍线性回归、逻辑回归模型线性回归（LinearRegression）模型形式：关键点：逻辑回归（LogisticRegression）模型形式：关键点：参数估计与评估：3.python浅拷贝及深拷贝浅拷贝（Shal
2019-01-19 王小康KK
姓名:王康公司:扬州市方圆建筑工程有限公司2018年3月16日～3月18日上海361期《六项精进》感谢二组学员【日精进打卡第307天】【知～学习】《六项精进》大纲3遍共862遍《大学》通篇3遍共860遍《六项精进》全书40页【经典名句】思想决定行为，行为决定习惯，习惯决定性格，性格决定命运。【行～实践】一、修身：（对自己个人）1、践行六项精进的理念。二、齐家：（对家庭和家人）1、和女朋友视频聊天。
Xinference如何注册自定义模型玩人工智能的辣条哥人工智能 AI 大模型 Xinference
环境：Xinference问题描述：Xinference如何注册自定义模型解决方案：1.写个model_config.json，内容如下{"version":1,"context_length":2048,"model_name":"custom-llama-3","model_lang":["en","ch"],"model_ability":["generate","chat"],"model
网络编程基础记得开心一点啊网络
目录♫什么是网络编程♫Socket套接字♪什么是Socket套接字♪数据报套接字♪流套接字♫数据报套接字通信模型♪数据报套接字通讯模型♪DatagramSocket♪DatagramPacket♪实现UDP的服务端代码♪实现UDP的客户端代码♫流套接字通信模型♪流套接字通讯模型♪ServerSocket♪Socket♪实现TCP的服务端代码♪实现TCP的客户端代码♫什么是网络编程网络编程，指网络上
简单了解 JVM 记得开心一点啊 jvm
目录♫什么是JVM♫JVM的运行流程♫JVM运行时数据区♪虚拟机栈♪本地方法栈♪堆♪程序计数器♪方法区/元数据区♫类加载的过程♫双亲委派模型♫垃圾回收机制♫什么是JVMJVM是JavaVirtualMachine的简称，意为Java虚拟机。虚拟机是指通过软件模拟的具有完整硬件功能的、运行在一个完全隔离的环境中的完整计算机系统（如：JVM、VMwave、VirtualBox）。JVM和其他两个虚拟机
Python实现简单的机器学习算法 master_chenchengg python python 办公效率 python开发 IT
Python实现简单的机器学习算法开篇：初探机器学习的奇妙之旅搭建环境：一切从安装开始必备工具箱第一步：安装Anaconda和JupyterNotebook小贴士：如何配置Python环境变量算法初体验：从零开始的Python机器学习线性回归：让数据说话数据准备：从哪里找数据编码实战：Python实现线性回归模型评估：如何判断模型好坏逻辑回归：从分类开始理论入门：什么是逻辑回归代码实现：使用skl
推荐算法_隐语义-梯度下降 _feivirus_ 算法机器学习和数学推荐算法机器学习隐语义
importnumpyasnp1.模型实现"""inputrate_matrix:M行N列的评分矩阵，值为P*Q.P:初始化用户特征矩阵M*K.Q:初始化物品特征矩阵K*N.latent_feature_cnt:隐特征的向量个数max_iteration:最大迭代次数alpha:步长lamda:正则化系数output分解之后的P和Q"""defLFM_grad_desc(rate_matrix,l
多线程之——ExecutorCompletionService 阿福德
在我们开发中，经常会遇到这种情况，我们起多个线程来执行，等所有的线程都执行完成后，我们需要得到个线程的执行结果来进行聚合处理。我在内部代码评审时，发现了不少这种情况。看很多同学都使用正确，但比较啰嗦，效率也不高。本文介绍一个简单处理这种情况的方法：直接上代码：publicclassExecutorCompletionServiceTest{@TestpublicvoidtestExecutorCo
BART&BERT Ambition_LAO 深度学习
BART和BERT都是基于Transformer架构的预训练语言模型。模型架构：BERT(BidirectionalEncoderRepresentationsfromTransformers)主要是一个编码器（Encoder）模型，它使用了Transformer的编码器部分来处理输入的文本，并生成文本的表示。BERT特别擅长理解语言的上下文，因为它在预训练阶段使用了掩码语言模型（MLM）任务，即
现代汉语粗糙版文学史与经典学习搬运工
第十六章文学史与经典文学史的兴起在西方,虽然从亚里士多德开始,在人类的著述中已经可以找到文学史概念与写作方式的萌芽,但是,人们一般认为17世纪后期到18世纪是现代文学史写作真正开始的时期。长达百年波及整个欧洲的“古今之争”孕育出文学研究的历史意识,现代意义上的文学史观念在这场影响深远的论争中初见端倪。从18世纪晚期到19世纪初,由于席勒、弗·施莱格尔和赫尔德等人的介入,文学史研究逐渐变得复杂和成熟
系统架构设计师需求分析篇二 AmHardy 软件架构设计师系统架构需求分析面向对象分析分析模型 UML和SysML
面向对象分析方法1.用例模型构建用例模型一般需要经历4个阶段：识别参与者：识别与系统交互的所有事物。合并需求获得用例：将需求分配给予其相关的参与者。细化用例描述：详细描述每个用例的功能。调整用例模型：优化用例之间的关系和结构，前三个阶段是必需的。2.用例图的三元素参与者：使用系统的用户或其他外部系统和设备。用例：系统所提供的服务。通信关联：参与者和用例之间的关系，或用例与用例之间的关系。3.识别参
语文主题教学学习笔记之87 东哥杂谈
“语文主题教学”学习笔记之八十七（0125）今天继续学习小学语文主题教学的实践样态。板块三：教学中体现“书艺”味道。作为四大名著之一的《水浒传》，堪称我国文学宝库之经典。对从《水浒传》中摘选的单元，教师就要了解其原生态，即评书体特点。这也要求教师要了解一些常用的评书行话术语，然后在教学时适时地加入一些，让学生体味其文本中原有的特色。学生也要尽可能地通过朗读的方式，而不单是分析讲解的方式进行学习。细
2019-03-24 李飞720
姓名：李飞企业名称：临沂鑫道食品有限公司组别373期利他1组日精进打卡第338天】【知~学习】1、阿米巴经营一段2、活用人才1段3、活法、一段【行~实践】一、修身：读书、抽烟减量、俯卧撑个跑步3公里二、齐家、劝说老爸与姑姑和好三、建功、业务洽谈【经典名句分享】1、依据原理原则追求事物的本质，以“作为人，何谓正确”进行判断2、经营者必须为员工物质和精神两方面的幸福殚精竭虑，倾尽全力，必须超脱私心，让
如何用matlab灵活控制feko的求解 NingrLi matlab 开发语言
https://bbs.rfeda.cn/read.php?tid=3778Feko中的模型和求解设置等都可以通过editfeko进行设置，其文件存储为.pre文件，该文件可以用文本打开，因此，我们可以通过VB、VC、matlab等工具对.pre文件进行读写操作，以达到更灵活的使用feko。同样，对于.out文件，我们也可以进行读操作。熟练使用对.pre文件和.out文件的操作后，我们可以方便的计
Python算法L5：贪心算法小熊同学哦 Python算法算法 python 贪心算法
Python贪心算法简介目录Python贪心算法简介贪心算法的基本步骤贪心算法的适用场景经典贪心算法问题1.**零钱兑换问题**2.**区间调度问题**3.**背包问题**贪心算法的优缺点优点：缺点：结语贪心算法（GreedyAlgorithm）是一种在每一步选择中都采取当前最优或最优解的算法。它的核心思想是，在保证每一步局部最优的情况下，希望通过贪心选择达到全局最优解。虽然贪心算法并不总能得到全
tiff批量转png 诺有缸的高飞鸟 opencv 图像处理 python opencv 图像处理
目录写在前面代码完写在前面1、本文内容tiff批量转png2、平台/环境opencv,python3、转载请注明出处：https://blog.csdn.net/qq_41102371/article/details/132975023代码importnumpyasnpimportcv2importosdeffindAllFile(base):file_list=[]forroot,ds,fsin
遥感影像的切片处理 sand&wich 计算机视觉 python 图像处理
在遥感影像分析中，经常需要将大尺寸的影像切分成小片段，以便于进行详细的分析和处理。这种方法特别适用于机器学习和图像处理任务，如对象检测、图像分类等。以下是如何使用Python和OpenCV库来实现这一过程，同时确保每个影像片段保留正确的地理信息。准备环境首先，确保安装了必要的Python库，包括numpy、opencv-python和xml.etree.ElementTree。这些库将用于图像处理
计算机网络八股总结 Petrichorzncu 八股总结计算机网络笔记
这里写目录标题网络模型划分（五层和七层）及每一层的功能五层网络模型七层网络模型（OSI模型）==三次握手和四次挥手具体过程及原因==三次握手四次挥手TCP/IP协议组成==UDP协议与TCP/IP协议的区别==Http协议相关知识网络地址，子网掩码等相关计算网络模型划分（五层和七层）及每一层的功能五层网络模型应用层：负责处理网络应用程序，如电子邮件、文件传输和网页浏览。主要协议包括HTTP、FTP
redis学习笔记——不仅仅是存取数据 Everyday都不同 returnSource expire/del incr/lpush 数据库分区 redis
最近项目中用到比较多redis，感觉之前对它一直局限于get/set数据的层面。其实作为一个强大的NoSql数据库产品，如果好好利用它，会带来很多意想不到的效果。（因为我搞java，所以就从jedis的角度来补充一点东西吧。PS：不一定全，只是个人理解，不喜勿喷） 1、关于JedisPool.returnSource(Jedis jeids) 这个方法是从red
SQL性能优化-持续更新中。。。。。。 atongyeye oracle sql
1 通过ROWID访问表--索引你可以采用基于ROWID的访问方式情况,提高访问表的效率, , ROWID包含了表中记录的物理位置信息..ORACLE采用索引(INDEX)实现了数据和存放数据的物理位置(ROWID)之间的联系. 通常索引提供了快速访问ROWID的方法,因此那些基于索引列的查询就可以得到性能上的提高. 2 共享SQL语句--相同的sql放入缓存 3 选择最有效率的表
[JAVA语言]JAVA虚拟机对底层硬件的操控还不完善 comsci JAVA虚拟机
如果我们用汇编语言编写一个直接读写CPU寄存器的代码段，然后利用这个代码段去控制被操作系统屏蔽的硬件资源，这对于JVM虚拟机显然是不合法的，对操作系统来讲，这样也是不合法的，但是如果是一个工程项目的确需要这样做，合同已经签了，我们又不能够这样做，怎么办呢？那么一个精通汇编语言的那种X客，是否在这个时候就会发生某种至关重要的作用呢？ &n
lvs- real 男人50 LVS
#!/bin/bash # # Script to start LVS DR real server. # description: LVS DR real server # #. /etc/rc.d/init.d/functions VIP=10.10.6.252 host='/bin/hostname' case "$1" in sta
生成公钥和私钥 oloz DSA 安全加密
package com.msserver.core.util; import java.security.KeyPair; import java.security.PrivateKey; import java.security.PublicKey; import java.security.SecureRandom; public class SecurityUtil {
UIView 中加入的cocos2d，背景透明 374016526 cocos2d glClearColor
要点是首先pixelFormat:kEAGLColorFormatRGBA8，必须有alpha层才能透明。然后view设置为透明glView.opaque = NO;[director setOpenGLView:glView];[self.viewController.view setBackgroundColor:[UIColor clearColor]];[self.viewControll
mysql常用命令香水浓 mysql
连接数据库 mysql -u troy -ptroy 备份表 mysqldump -u troy -ptroy mm_database mm_user_tbl > user.sql 恢复表（与恢复数据库命令相同） mysql -u troy -ptroy mm_database < user.sql 备份数据库 mysqldump -u troy -ptroy
我的架构经验系列文章 - 后端架构 - 系统层面 agevs JavaScript jquery css html5
系统层面：高可用性所谓高可用性也就是通过避免单独故障加上快速故障转移实现一旦某台物理服务器出现故障能实现故障快速恢复。一般来说，可以采用两种方式，如果可以做业务可以做负载均衡则通过负载均衡实现集群，然后针对每一台服务器进行监控，一旦发生故障则从集群中移除；如果业务只能有单点入口那么可以通过实现Standby机加上虚拟IP机制，实现Active机在出现故障之后虚拟IP转移到Standby的快速
利用ant进行远程tomcat部署 aijuans tomcat
在javaEE项目中，需要将工程部署到远程服务器上，如果部署的频率比较高，手动部署的方式就比较麻烦，可以利用Ant工具实现快捷的部署。这篇博文详细介绍了ant配置的步骤（http://www.cnblogs.com/GloriousOnion/archive/2012/12/18/2822817.html），但是在tomcat7以上不适用，需要修改配置，具体如下： 1.配置tomcat的用户角色
获取复利总收入 baalwolf 获取
public static void main(String args[]){ int money=200; int year=1; double rate=0.1; &
eclipse.ini解释 BigBird2012 eclipse
大多数java开发者使用的都是eclipse，今天感兴趣去eclipse官网搜了一下eclipse.ini的配置，供大家参考，我会把关键的部分给大家用中文解释一下。还是推荐有问题不会直接搜谷歌，看官方文档，这样我们会知道问题的真面目是什么，对问题也有一个全面清晰的认识。 Overview 1、Eclipse.ini的作用 Eclipse startup is controlled by th
AngularJS实现分页功能 bijian1013 JavaScript AngularJS 分页
对于大多数web应用来说显示项目列表是一种很常见的任务。通常情况下，我们的数据会比较多，无法很好地显示在单个页面中。在这种情况下，我们需要把数据以页的方式来展示，同时带有转到上一页和下一页的功能。既然在整个应用中这是一种很常见的需求，那么把这一功能抽象成一个通用的、可复用的分页（Paginator）服务是很有意义的。 &nbs
[Maven学习笔记三]Maven archetype bit1129 ArcheType
archetype的英文意思是原型，Maven archetype表示创建Maven模块的模版，比如创建web项目，创建Spring项目等等. mvn archetype提供了一种命令行交互式创建Maven项目或者模块的方式， mvn archetype 1.在LearnMaven-ch03目录下，执行命令mvn archetype:gener
【Java命令三】jps bit1129 Java命令
jps很简单，用于显示当前运行的Java进程，也可以连接到远程服务器去查看 [hadoop@hadoop bin]$ jps -help usage: jps [-help] jps [-q] [-mlvV] [<hostid>] Definitions: <hostid>: <hostname>[:
ZABBIX2.2 2.4 等各版本之间的兼容性 ronin47
zabbix更新很快，从2009年到现在已经更新多个版本，为了使用更多zabbix的新特性，随之而来的便是升级版本，zabbix版本兼容性是必须优先考虑的一点客户端AGENT兼容 zabbix1.x到zabbix2.x的所有agent都兼容zabbix server2.4：如果你升级zabbix server，客户端是可以不做任何改变，除非你想使用agent的一些新特性。 Zabbix代理（p
unity 3d还是cocos2dx哪个适合游戏？ brotherlamp unity自学 unity教程 unity视频 unity资料 unity
unity 3d还是cocos2dx哪个适合游戏？问：unity 3d还是cocos2dx哪个适合游戏？答：首先目前来看unity视频教程因为是3d引擎，目前对2d支持并不完善，unity 3d 目前做2d普遍两种思路，一种是正交相机，3d画面2d视角，另一种是通过一些插件，动态创建mesh来绘制图形单元目前用的较多的是2d toolkit，ex2d，smooth moves，sm2，
百度笔试题：一个已经排序好的很大的数组，现在给它划分成m段，每段长度不定，段长最长为k，然后段内打乱顺序，请设计一个算法对其进行重新排序 bylijinnan java 算法面试百度招聘
import java.util.Arrays; /** * 最早是在陈利人老师的微博看到这道题： * #面试题#An array with n elements which is K most sorted，就是每个element的初始位置和它最终的排序后的位置的距离不超过常数K * 设计一个排序算法。It should be faster than O(n*lgn)。
获取checkbox复选框的值 chiangfai checkbox
<title>CheckBox</title> <script type = "text/javascript"> doGetVal: function doGetVal() { //var fruitName = document.getElementById("apple").value;//根据
MySQLdb用户指南 chenchao051 mysqldb
原网页被墙，放这里备用。 MySQLdb User's Guide Contents Introduction Installation _mysql MySQL C API translation MySQL C API function mapping Some _mysql examples MySQLdb
HIVE 窗口及分析函数 daizj hive 窗口函数分析函数
窗口函数应用场景：（1）用于分区排序（2）动态Group By （3）Top N （4）累计计算（5）层次查询一、分析函数用于等级、百分点、n分片等。函数说明 RANK() &nbs
PHP ZipArchive 实现压缩解压Zip文件 dcj3sjt126com PHP zip
PHP ZipArchive 是PHP自带的扩展类，可以轻松实现ZIP文件的压缩和解压，使用前首先要确保PHP ZIP 扩展已经开启，具体开启方法就不说了，不同的平台开启PHP扩增的方法网上都有，如有疑问欢迎交流。这里整理一下常用的示例供参考。一、解压缩zip文件 01 02 03 04 05 06 07 08 09 10 11
精彩英语贺词 dcj3sjt126com 英语
I'm always here 我会一直在这里支持你 &nb
基于Java注解的Spring的IoC功能 e200702084 java spring bean IOC Office
java模拟post请求 geeksun java
一般API接收客户端（比如网页、APP或其他应用服务）的请求，但在测试时需要模拟来自外界的请求，经探索，使用HttpComponentshttpClient可模拟Post提交请求。此处用HttpComponents的httpclient来完成使命。 import org.apache.http.HttpEntity ; import org.apache.http.HttpRespon
Swift语法之 ---- ?和!区别 hongtoushizi ?swift !
转载自： http://blog.sina.com.cn/s/blog_71715bf80102ux3v.html Swift语言使用var定义变量，但和别的语言不同，Swift里不会自动给变量赋初始值，也就是说变量不会有默认值，所以要求使用变量之前必须要对其初始化。如果在使用变量之前不进行初始化就会报错： var stringValue : String //
centos7安装jdk1.7 jisonami jdk centos
安装JDK1.7 步骤1、解压tar包在当前目录 [root@localhost usr]#tar -xzvf jdk-7u75-linux-x64.tar.gz 步骤2：配置环境变量在etc/profile文件下添加 export JAVA_HOME=/usr/java/jdk1.7.0_75 export CLASSPATH=/usr/java/jdk1.7.0_75/lib
数据源架构模式之数据映射器 home198979 PHP 架构数据映射器 datamapper
前面分别介绍了数据源架构模式之表数据入口、数据源架构模式之行和数据入口数据源架构模式之活动记录，相较于这三种数据源架构模式，数据映射器显得更加“高大上”。一、概念数据映射器（Data Mapper）：在保持对象和数据库（以及映射器本身）彼此独立的情况下，在二者之间移动数据的一个映射器层。概念永远都是抽象的，简单的说，数据映射器就是一个负责将数据映射到对象的类数据。 &nb
在Python中使用MYSQL pda158 mysql python
缘由　　近期在折腾一个小东西须要抓取网上的页面。然后进行解析。将结果放到数据库中。　　了解到 Python在这方面有优势，便选用之。　　由于我有台 server上面安装有 mysql，自然使用之。在进行数据库的这个操作过程中遇到了不少问题，这里记录一下，大家共勉。　　 python中mysql的调用　　百度之后能够通过MySQLdb进行数据库操作。
单例模式 hxl1988_0311 java 单例设计模式单件
package com.sosop.designpattern.singleton; /* * 单件模式：保证一个类必须只有一个实例，并提供全局的访问点 * * 所以单例模式必须有私有的构造器，没有私有构造器根本不用谈单件 * * 必须考虑到并发情况下创建了多个实例对象 * */ /** * 虽然有锁，但是只在第一次创建对象的时候加锁，并发时不会存在效率
27种迹象显示你应该辞掉程序员的工作 vipshichg 工作
1、你仍然在等待老板在2010年答应的要提拔你的暗示。 2、你的上级近10年没有开发过任何代码。 3、老板假装懂你说的这些技术，但实际上他完全不知道你在说什么。 4、你干完的项目6个月后才部署到现场服务器上。 5、时不时的，老板在检查你刚刚完成的工作时，要求按新想法重新开发。 6、而最终这个软件只有12个用户。 7、时间全浪费在办公室政治中，而不是用在开发好的软件上。 8、部署前5分钟才开始测试。