【笔记】训练神经网络时解决显存不足的办法：核常兼备，常：减少参数规模；核：减少参数大小

常规方法1：

 常规方法2：

GitHub - gpleiss/efficient_densenet_pytorch: A memory-efficient implementation of DenseNets
model2GitHub - mapillary/inplace_abn: In-Place Activated BatchNorm for Memory-Optimized Training of DNNs

 

 

 

删除loss： Tensor to Variable and memory freeing best practices - #2 by fmassa - PyTorch Forums

降低精度：

https://github.com/NVIDIA/apex

分清楚 eval 和 requires_grad = False

'model.eval()' vs 'with torch.no_grad()' - PyTorch Forums

使用torch.cuda.empty_cache()，在确定的地方释放显存

How can we release GPU memory cache? - PyTorch Forums

核子武器：

    参考资料：

    {
        [1] https://discuss.pytorch.org/t/cuda-memory-continuously-increases-when-net-images-called-in-every-iteration/501

        [2] https://forums.fast.ai/t/clearing-gpu-memory-pytorch/14637/3

        [3] https://pytorch.org/docs/stable/notes/cuda.html#cuda-memory-management

        [4] https://discuss.pytorch.org/t/how-to-debug-causes-of-gpu-memory-leaks/6741/3

    }   

}