【报错】pytorch DataParallel - StopIteration: Caught StopIteration in replica 0 on device 0.

【报错】pytorch DataParallel  -  StopIteration: Caught StopIteration in replica 0 on device 0.

 

环境：pytorch 1.5

 

问题：

pytorch单机多卡用nn.DataParallel 的时候无法forward，会报错

 

原因：

pytorch1.5的bug

 

解决方案:

降级到pytorch1.4

 

参考文献：

https://github.com/huggingface/transformers/issues/3936

https://github.com/huggingface/transformers/issues/4189

https://github.com/huggingface/transformers/issues/3936