CV入门之模型集成

前言

集成学习在深度学习上为了取得更好的效果，往往通过使用集成学习的一些技巧来实现精度的进一步的提升

数据集的划分

深度学习离不开数据，好的数据往往能够让我们的效率事半功倍，最为常见
的数据划分的方法就是划分9:1的训练集和验证集，除此之外还可以划分如下图所示*的
交叉验证，在每个数据集上留出一部分作为验证集

可以参考这篇博客

加入Dropout

相信入门深度学习的人都听说过
来段英文的解释
Deep neural nets with a large number of parameters are very powerful machine learning systems. However, overfitting is a serious problem in such networks. Large networks are also slow to use, making it difficult to deal with overfitting by combining the predictions of many different large neural nets at test time. Dropout is a technique for addressing this problem. The key idea is to randomly drop units (along with their connections) from the neural network during training. This prevents units from co-adapting too much. During training, dropout samples from an exponential number of different “thinned” networks. At test time, it is easy to approximate the effect of averaging the predictions of all these thinned networks by simply using a single unthinned network that has smaller weights. This significantly reduces overfitting and gives major improvements over other regularization methods. We show that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
带有大量参数的深度神经网络是非常强大的机器学习系统。然而，在这样的网络中，过度拟合是一个严重的问题。大型神经网络的使用也很缓慢，因此很难通过在测试时结合许多不同大型神经网络的预测来处理过拟合问题。dropout是一种解决这个问题的技术。关键的想法是在训练过程中从神经网络中随机丢弃单元(以及它们的连接)。这可以防止单位之间的相互适应过度。在培训期间,辍学指数样本数量的不同€œthinneda€网络。在测试时，只需使用一个权重较小的网络，就可以很容易地估计出所有这些网络的预测的平均效果。这大大减少了过度拟合，并对其他正则化方法进行了重大改进。我们发现，在视觉、语音识别、文档分类和计算生物学等监督学习任务中，dropout提高了神经网络的性能，在许多基准数据集上获得了最先进的结果。

#具体的在pytorch上的用法简单
class SVHN_Model1(nn.Module):
    def __init__(self):
        super(SVHN_Model1, self).__init__()

        model_conv = models.resnet18(pretrained=True,nn.Dropout(0.25))#这里加dropout
        model_conv.avgpool = nn.AdaptiveAvgPool2d(1)
        model_conv = nn.Sequential(*list(model_conv.children())[:-1],nn.Dropout(0.25))
        self.cnn = model_conv

        self.fc1 = nn.Linear(512, 11)
        self.fc2 = nn.Linear(512, 11)
        self.fc3 = nn.Linear(512, 11)
        self.fc4 = nn.Linear(512, 11)
        self.fc5 = nn.Linear(512, 11)

TTA

TTA就是测试数据增广的意思，论文的参考

Snapshot

也就是所谓的跳出局部最优解，这个技术对于实验的设备的要求很高

---

后记

这一段的打卡学习暂时告一段落，感谢datawhile让像我们这些小白入门深度学习，体会到深度学习的乐趣，继续加油！！！