目录
报错信息:
解决:
整理不易,欢迎一键三连!!!
介绍:mmseg工程在一台服务器上训练了几十个epoch,在新的服务器上接着resume,已经训练了500个iter,报错RuntimeError: CUDA error: an illegal memory access was encountered
File "/data/xx/project/mmsegmentation-master/mmseg/models/losses/accuracy.py", line 49, in accuracy
correct = correct[:, target != ignore_index]
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
terminate called after throwing an instance of 'c10::CUDAError'
what(): CUDA error: an illegal memory access was encountered
data = dict(
samples_per_gpu=16,
workers_per_gpu=16,
......
)
修改了batch_size 和worker就好了。
送你们一条美丽的--分割线--
⛵⛵⭐⭐