使用Libtorch在C++环境下的PyTorch模型部署

使用Libtorch在C++环境下的PyTorch模型部署

win11+VS2022

提前在下好和pytorch版本一致的libtorch。在pytorch官网只能下到最新版本的，之前版本可以在csdn上搜索。

1.在python对训练好的模型进行转换

import torch
import torchvision.models as models
import torch.nn.functional as F

model = models.vgg16()
example = torch.rand(1, 3, 224, 224)
model = model.eval()
traced_script_module = torch.jit.trace(model, example)
output = traced_script_module(torch.ones(1, 3, 224, 224))
# output_softmax = F.softmax(output, dim=1)
traced_script_module.save('vgg16_trace.pt')
print(output)

2.在c++中导入序列化后的TorchSricpt模型

先编写CMakeLists.txt文件

cmake_minimum_required(VERSION 3.0.0 FATAL_ERROR)
project(libtorch_test)
# set(CMAKE_PREFIX_PATH "D:\\cpplib\\libtorch\\share\\cmake\\Torch")
set(Torch_DIR D:\\cpplib\\libtorch\\share\\cmake\\Torch)
find_package(Torch REQUIRED)
message(STATUS "Pytorch status:")
message(STATUS "libraries: ${TORCH_LIBRARIES}")
add_executable(libtorch_test test.cpp)
target_link_libraries(libtorch_test "${TORCH_LIBRARIES}")
set_property(TARGET libtorch_test PROPERTY CXX_STANDARD 14)

编写调用模型的test.cpp文件

#include "torch/script.h"
#include "torch/torch.h"
#include 
#include 
using namespace std;


int main(int argc, const char* argv[]) {
    if (argc != 2) {
        std::cerr << "usage: libtorch_test \n";
        return -1;
    }


    // 读取TorchScript转化后的模型
    torch::jit::script::Module module;
    try {
        module = torch::jit::load(argv[1]);
    }


    catch (const c10::Error& e) {
        std::cerr << "error loading the model\n";
        return -1;
    }


    //module->to(at::kCUDA);
    assert(module != nullptr);
    std::cout << "ok\n";


    // 构建示例输入
    std::vector<torch::jit::IValue> inputs;
    inputs.push_back(torch::ones({ 1, 3, 224, 224 }));


    // 执行模型推理并输出tensor
    at::Tensor output = module.forward(inputs).toTensor();
    std::cout << output.slice(/*dim=*/1, /*start=*/0, /*end=*/5) << '\n';
}

两个文件写好后点击视图->终端，进行cmake编译

cd test //这里是因为创了一个项目，进入cmakelists和cpp文件所在的文件夹就行
mkdir example_test //建一个文件加用来放cmake产生的文件
cd example_test
cmake -DCMAKE_PREFIX_PATH=D:/cpplib/libtorch/share/cmake/Torch -DCMAKE_BUILD_TYPE=Release -G"Visual Studio 17 2022"  ..   //最后-G那个可以不写

这里记录一下碰到的问题

CMake Error at CMakeProject1/CMakeLists.txt:4 (find_package):
  By not providing "FindTorch.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "Torch", but
  CMake did not find one.

  Could not find a package configuration file provided by "Torch" with any of
  the following names:

    TorchConfig.cmake
    torch-config.cmake

  Add the installation prefix of "Torch" to CMAKE_PREFIX_PATH or set
  "Torch_DIR" to a directory containing one of the above files.  If "Torch"
  provides a separate development package or SDK, be sure it has been
  installed.

找不到Torch包

设置->系统信息->高级系统设置->环境变量->新建系统变量

变量名： Torch_DIR

变量值： D:\cpplib\libtorch\share\cmake\Torch

设置完之后一定要重启！

生成后进入example_test文件夹，可以看到生成了项目文件

打开libtorch_test，在解决方案管理器中选中libtorch_test右键设为启动项目

调到Release后运行

在example_test文件夹中打开Release文件夹，可以看到生成了.exe文件

在终端中运行这个文件

./Release/libtorch_test ******/vgg16_trace.pt