PYtorch运行报错 CUDNN_STATUS_EXECUTION_FAILED

PYtorch运行报错
折磨了两天三夜的问题
RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED

Use load_from_local loader
*********0 / 1*********
Traceback (most recent call last):
  File "run_server.py", line 15, in 
    from endback import create_app
  File "/cjhx/appsoft/table_ocr/table_ocr_merge/endback/__init__.py", line 13, in 
    from endback.utils import gen_response
  File "/cjhx/appsoft/table_ocr/table_ocr_merge/endback/utils.py", line 151, in 
    from ocr_lib.fanyang_code.interfere_old import fanyang_predict
  File "/cjhx/appsoft/table_ocr/table_ocr_merge/ocr_lib/fanyang_code/interfere_old.py", line 80, in 
    fanyang_predict('data_new',tab_data)
  File "/cjhx/appsoft/table_ocr/table_ocr_merge/ocr_lib/fanyang_code/interfere_old.py", line 60, in fanyang_predict
    logits = model(batch)
  File "/cjhx/anaconda3/envs/yfan_cu11_combine/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/cjhx/appsoft/table_ocr/table_ocr_merge/ocr_lib/fanyang_code/modeling_OLD.py", line 34, in forward
    x, _ = self.rnn(x)
  File "/cjhx/anaconda3/envs/yfan_cu11_combine/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/cjhx/anaconda3/envs/yfan_cu11_combine/lib/python3.8/site-packages/torch/nn/modules/rnn.py", line 739, in forward
    result = _VF.gru(input, hx, self._flat_weights, self.bias, self.num_layers,
RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED

原因pytorch 好像会自带一套cuda工具 A-100 这张卡，好像只支持cuda 11.0 以上的版本和pytouch 自带的cuda 有冲突

官网链接

https://pytorch.org/get-started/previous-versions/

卸载旧的pytorch
conda uninstall pytorch
conda uninstall libtorch
pip uninstall torch
安装新的pytorch
#安装 CUDA 11.0
conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=11.0 -c pytorch

#安装 pytorch_lightning
python -m  pip install pytorch_lightning

# 更新 transformers 到最新版本 
python -m pip install transformers

# 更新 torch-sparse 到最新版本 很慢
python -m pip install torch-sparse

如果运行失败

https://pytorch-geometric.com/whl/torch-1.7.1%2Bcu110.html 从这里下载 对应python cuda的 插件

检查命令

import torch

print(torch.__version__)
print(torch.version.cuda)
print(torch.cuda.is_available())

代码更改

vim /home/ocrproject/new_table_ocr/table_ocr_merge/ocr_lib/HTTP_SET/jie_kou/test.py