torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 86.00 MiB (GPU 0； 31.74 GiB total

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 86.00 MiB (GPU 0; 31.74 GiB total capacity; 6.39 GiB already allocated; 78.38 MiB free; 6.47 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

解决办法
逐步调小PYTORCH_CUDA_ALLOC_CONF，但是会损失性能。
参考：https://blog.csdn.net/MirageTanker/article/details/127998036

set PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:16