pytorch模型转caffe模型（pytorch-＞onnx-＞caffe）

---

前言

这里主要用ubuntu18.04对caffe环境进行搭建，之前用windows搭建过caffe的环境但是还需要安装onnx的库，windows内存不太够用了，我也就不敢再瞎捣鼓了，于是乎我就用windows原先安装好的pytorch环境进行pytorch转onnx，用ubuntu进行onnx转caffe。
这里的ubuntu系统是我在阿里云租的一个服务器，因为第一次租有优惠很便宜。。。。反正比某迅的会员便宜。
这里用到的训练代码是yolov5

一.pytorch转onnx

以下这些我是在windows下的pytorch环境实现的。

1.修改yolov5/models/export.py

将opset_version=12改为opset_version=10（大约在72行左右）

2.在yolov5的目录下运行

python models/export.py --weights weights/yolov5s.pt --img 640 --batch 1
（ --weights 后面跟你们自己训练好的模型）

3.简化onnx模型

很多时候，很多节点比如cast节点，Identity 这些节点可能都不需要，所以我们需要进行简化。
安装简化器
pip install onnx-simplifier
简化onnx模型
python -m onnxsim weights/yolov5s.onnx（简化前的模型名字与存放地址） weights/yolov5s_sim.onnx （简化后的模型名字与存放地址）
如果不做简化会报这个错
TypeError: ONNX node of type Clip is not supported.

二.onnx转caffe

1.yolov5网络层优化

在yolov5训练之前最好是改一下网络层，要不会报这个错。

Traceback (most recent call last):
  File "convertCaffe.py", line 159, in <module>
    convertToCaffe(graph, prototxt_path, caffemodel_path, exis_focus=True, focus_concat_name="Concat_40", focus_conv_name="Conv_41")
  File "convertCaffe.py", line 83, in convertToCaffe
    layer = converter_fn(node, graph, err)
  File "/home/admin/code/yolov5_onnx2caffe/onnx2caffe/_operators.py", line 505, in _convert_resize
    height_scale = scales[2]
IndexError: index 2 is out of bounds for axis 0 with size 0

因为我用的是yolov5s模型所以更改yolov5/models/yolov5s.yaml
将yolov5的focus层替换为conv层（stride为2），upsample层替换为deconv层
（反卷积层），因为caffe不支持focus层。

# parameters
nc: 4  # number of classes
depth_multiple: 0.33  # model depth multiple
width_multiple: 0.50  # layer channel multiple

# anchors
anchors:
  - [10,13, 16,30, 33,23]  # P3/8
  - [30,61, 62,45, 59,119]  # P4/16
  - [116,90, 156,198, 373,326]  # P5/32

# YOLOv5 backbone
backbone:
  # [from, number, module, args]
  #[[-1, 1, Focus, [64, 3]],  # 0-P1/2
  [[-1, 1, Conv, [64, 3, 2]],  # 0-P1/2
   [-1, 1, Conv, [128, 3, 2]],  # 1-P2/4
   [-1, 3, C3, [128]],
   [-1, 1, Conv, [256, 3, 2]],  # 3-P3/8
   [-1, 9, C3, [256]],
   [-1, 1, Conv, [512, 3, 2]],  # 5-P4/16
   [-1, 9, C3, [512]],
   [-1, 1, Conv, [1024, 3, 2]],  # 7-P5/32
   [-1, 1, SPP, [1024, [5, 9, 13]]],
   [-1, 3, C3, [1024, False]],  # 9
  ]

# YOLOv5 head
head:
  [[-1, 1, Conv, [512, 1, 1]],
   #[-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [-1, 1, nn.ConvTranspose2d, [256,256,2,2]],
   [[-1, 6], 1, Concat, [1]],  # cat backbone P4
   [-1, 3, C3, [512, False]],  # 13

   [-1, 1, Conv, [256, 1, 1]],
   #[-1, 1, nn.Upsample, [None, 2, 'nearest']],
  [-1, 1, nn.ConvTranspose2d, [128,128,2,2]],
   [[-1, 4], 1, Concat, [1]],  # cat backbone P3
   [-1, 3, C3, [256, False]],  # 17 (P3/8-small)

   [-1, 1, Conv, [256, 3, 2]],
   [[-1, 14], 1, Concat, [1]],  # cat head P4
   [-1, 3, C3, [512, False]],  # 20 (P4/16-medium)

   [-1, 1, Conv, [512, 3, 2]],
   [[-1, 10], 1, Concat, [1]],  # cat head P5
   [-1, 3, C3, [1024, False]],  # 23 (P5/32-large)

   [[17, 20, 23], 1, Detect, [nc, anchors]],  # Detect(P3, P4, P5)
  ]

2.ubuntu服务器环境搭建

参考博客：https://blog.csdn.net/sinat_38439143/article/details/97244296
Ubuntu18.04
python3.6（ubuntu18.04自带的python3）
cpu（带GPU的服务器太贵了。。。。。租不起）
这里使用的远程文件传输软件是filezilla

从windows拷贝文件到ubuntu服务器上后需要对文件夹进行权限修改否则没办法对文件进行修改。
sudo chmod a+rwx -R 需要加权限的文件夹
（权限不够报错：PermissionError: [Errno 13] Permission denied）

①安装库

sudo  apt-get update
sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler 
sudo apt-get install --no-install-recommends libboost-all-dev
sudo apt-get install python-dev
sudo apt-get install libatlas-base-dev
sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev
sudo apt-get install python-opencv

在命令窗输入python，假如打开的不是python3.6，
需要软连接python3.6到python
sudo ln -s /usr/bin/python3.6 /usr/bin/python

安装python的onnx库
pip install onnx

②安装caffe的源码

我曾经为了简单直接安装了caffe的python库（sudo apt install caffe-cpu）但是最后要改源码我找不到库在哪，最后卸了python的caffe库又安装了caffe的源码。
git clone git://github.com/BVLC/caffe.git

③修改caffe/Makefile.config这个文件

1.去掉CPU_ONLY :=1的注释 
2.注释掉CUDA有关的行： 
#CUDA_DIR := /usr/local/cuda 
#CUDA_DIR := /usr 
#CUDA_ARCH := .... 
#TEST_GPUID := 0 
3.去掉WITH_PYTHON_LAYER := 1的注释
4.修改这一行：
# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib  /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/serial
5.修改python的版本
将
PYTHON_INCLUDE := /usr/include/python2.7 \
		/usr/lib/python2.7/dist-packages/numpy/core/include
改成
PYTHON_INCLUDE := /usr/include/python3.6 \
		/usr/lib/python3.6/dist-packages/numpy/core/include

④修改caffe/Makefile文件

1.opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs     
后面加入 opencv_imgcodecs  
2. 找到LIBRARIES +=glog gflags protobuf boost_system boost_filesystem m hdf5_hl hdf5
更改最后两项为：
LIBRARIES +=glog gflags protobuf boost_system boost_filesystem m hdf5_serial_hl hdf5_serial
3.修改python版本
将
PYTHON_LIBRARIES := boost_python python2.7
改成
PYTHON_LIBRARIES := boost_python-py36  #py36需要改为你自己的python版本，如py35,py37(208行左右)

⑤添加Upsample层和permute层

caffe源码不支持这两个层所以需要添加。
如果不添加会报这个错
参考博客：https://blog.csdn.net/ynzzxc/article/details/116931806

Traceback (most recent call last):
  File "convertCaffe.py", line 122, in <module>
    convertToCaffe(graph, opset_version, prototxt_path, caffemodel_path)
  File "convertCaffe.py", line 79, in convertToCaffe
    layers[id] = layer._to_proto()
  File "/home/ubuntu/ONNXToCaffe-master/MyCaffe.py", line 100, in _to_proto
    assign_proto(layer, k, v)
  File "/home/ubuntu/ONNXToCaffe-master/MyCaffe.py", line 29, in assign_proto
    is_repeated_field = hasattr(getattr(proto, name), 'extend')
AttributeError: permute_param

下载caffe_plus源码
git clone https://github.com/jnulzl/caffe_plus.git

#将caffe_plus/include/caffe/layers/upsample_layer.hpp 
   caffe_plus/include/caffe/layers/permute_layer.hpp
#复制到caffe/include/caffe/layers/

#将caffe_plus/src/caffe/layers/upsample_layer.cpp 
   caffe_plus/src/caffe/layers/upsample_layer.cu 
   caffe_plus/src/caffe/layers/permute_layer.cpp 
   caffe_plus/src/caffe/layers/permute_layer.cu 
#复制到caffe/src/caffe/layers/

# 修改caffe.proto文件
vim caffe/src/caffe/proto/caffe.proto
在optional WindowDataParameter window_data_param = 129;（约第423行）后增加代码：
optional PermuteParameter permute_param = 150;
optional UpsampleParameter upsample_param = 151;

在末尾增加代码：
message PermuteParameter {
  // The new orders of the axes of data. Notice it should be with
  // in the same range as the input data, and it starts from 0.
  // Do not provide repeated order.
  repeated uint32 order = 1;
}
message UpsampleParameter {		
	optional int32 height = 1 [default = 32];
	optional int32 width = 2 [default = 32];
	optional int32 height_scale = 3 [default = 2];
	optional int32 width_scale = 4 [default = 2];
	enum UpsampleOp {
		NEAREST = 0;
		BILINEAR = 1;
	}
	optional UpsampleOp mode = 5 [default = BILINEAR];
}

⑥编译caffe源码

（服务器操作需要权限加sudo）

cd caffe
sudo make all
sudo make test
sudo make runtest

⑦pycaffe安装

安装依赖库

cd caffe/python
for req in $(cat requirements.txt); do pip install $req; done

添加 PYTHONPATH环境变量

sudo gedit /etc/profile
export PYTHONPATH=$PYTHONPATH:/home/XXX/caffe/python:$PYTHONPATH   # xxx为用户名
$ source /etc/profile

编译
在caffe目录下

sudo make pycaffe

⑧caffe环境测试

admin@iZbp1j0uggzced8qupgb5rZ:~$ python
Python 3.6.9 (default, Jan 26 2021, 15:33:00) 
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
>>> 

⑨onnx转caffe

这里我用到的onnx转caffe代码是： https://github.com/Wulingtian/yolov5_onnx2caffe
更改yolov5_onnx2caffe/convertCaffe.py
设置onnx_path（上面转换得到的onnx简化后的模型）
prototxt_path（caffe的prototxt保存路径）
caffemodel_path（caffe的caffemodel保存路径）

onnx_path = "/home/admin/code/yolov5_onnx2caffe/weights/yolov5s_sim.onnx"
    prototxt_path = "/home/admin/code/yolov5_onnx2caffe/weights/yolov5s_sim.pro
totxt"
    caffemodel_path = "/home/admin/code/yolov5_onnx2caffe/weights/yolov5s_sim.c
affemodel"

运行python convertCaffe.py完成转换

学习时间

2021.6.21

---