python index out of bounds_Assertion `index ＞= -sizes[i] && index ＜ sizes[i] && “index out of bounds...

pytorch报了一堆错，

首先是类似这样的重复出现:

/opt/conda/conda-bld/pytorch_1573049304260/work/aten/src/ATen/native/cuda/IndexKernel.cu:60: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [32,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.

然后是traceback,

最后是：

RuntimeError: CUDA error: device-side assert triggered

terminate called after throwing an instance of 'c10::Error'

what(): CUDA error: device-side assert triggered (insert_events at /opt/conda/conda-bld/pytorch_1573049304260/work/c10/cuda/CUDACachingAllocator.cpp:569)

frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x47 (0x7f957bdd7687 in /data2/lingjian/anaconda3/lib/python3.6/site-packages/torch/lib/libc10.so)

frame #1: + 0x17044 (0x7f957c013044 in /data2/lingjian/anaconda3/lib/python3.6/site-packages/torch/lib/libc10_cuda.so)

frame #2: + 0x1cccb (0x7f957c018ccb in /data2/lingjian/anaconda3/lib/python3.6/site-packages/torch/lib/libc10_cuda.so)

frame #3: c10::TensorImpl::release_resources() + 0x4d (0x7f957bdc4e9d in /data2/lingjian/anaconda3/lib/python3.6/site-packages/torch/lib/libc10.so)

frame #4: + 0x1c86c9 (0x7f95ac80d6c9 in /data2/lingjian/anaconda3/lib/python3.6/site-packages/torch/lib/libtorch_python.so)

frame #5: + 0x4aff6b (0x7f95acaf4f6b in /data2/lingjian/anaconda3/lib/python3.6/site-packages/torch/lib/libtorch_python.so)

frame #6: + 0x4affa6 (0x7f95acaf4fa6 in /data2/lingjian/anaconda3/lib/python3.6/site-packages/torch/lib/libtorch_python.so)

frame #24: __libc_start_main + 0xf0 (0x7f95bbeaa840 in /lib/x86_64-linux-gnu/libc.so.6)

Aborted (core dumped)，

从“Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed”能够获取到的信息是某个地方的index越界了，接下来就是定位这个地方，然后根据traceback查找到最近的一个地方，竟然是一个print语句!!! 这不是胡扯吗，查了一些资料后发现traceback不能准确定位出错行的原因是这是在gpu上跑的程序。

那么要怎样才能准确定位出错的行数呢？其实很简单，运行代码的时候加上CUDA_LAUNCH_BLOCKING=1就行了，然后就能从traceback里找到准确的出错地点。

原文链接:https://blog.csdn.net/qq_36288181/article/details/108031011