NCNN----Monodepthv2单目深度估计 小米手机部署

题目要求:学习了解单目深度估计模型MonoDepthv2,基于NCNN推理框架部署到小米手机
MonoDepthv2 论文:Digging Into Self-Supervised Monocular Depth Estimation
MonoDepthv2 源码:Monodepth2 GitHub

分析:
1)了解MonoDepthv2的基本原理和代码理解
2)将模型转化为更加方便高效的NCNN模型并在小米手机端完成推理过程

  • 结果展示:

  • 参考代码
    nihui/ncnn-android-nanodet

  • 模型转换
    一键转换 onnx2ncnn

  • 包依赖
    opencv-mobile-4.6.0-android (版本可选)
    ncnn-20230223-android-vulkan (版本可选)

  • 软件环境
    Android Studio, SDK Platforms: Android 11, 12, 13; SDK tools, NDK tools

  • 工程目录(重点修改部分已经标红)
    NCNN----Monodepthv2单目深度估计 小米手机部署_第1张图片

  • 具体修改项

    1. CMakeLists.txt
    project(nanodetncnn)
    
    cmake_minimum_required(VERSION 3.10)
    
    set(OpenCV_DIR ${CMAKE_SOURCE_DIR}/opencv-mobile-4.6.0-android/sdk/native/jni)
    find_package(OpenCV REQUIRED core imgproc)
    
    set(ncnn_DIR ${CMAKE_SOURCE_DIR}/ncnn-20230223-android-vulkan/${ANDROID_ABI}/lib/cmake/ncnn)
    find_package(ncnn REQUIRED)
    add_library(nanodetncnn SHARED nanodetncnn.cpp nanodet.cpp ndkcamera.cpp)
    
    target_link_libraries(nanodetncnn ncnn ${OpenCV_LIBS} camera2ndk mediandk)
    
    
    1. nanodet.cpp (#59)设置模型输入输出尺寸
    int NanoDet::detect(cv::Mat& rgb)
    {
        ncnn::Mat in = ncnn::Mat::from_pixels_resize(rgb.data, ncnn::Mat::PIXEL_RGB, rgb.cols, rgb.rows,640, 192);
        in.substract_mean_normalize(mean_vals, norm_vals);
    
        ncnn::Extractor ex = nanodet.create_extractor();
        ex.input("input", in);
        ncnn::Mat model_out;
        ex.extract("output", model_out);
    
        cv::Mat out(192, 640, CV_32FC1, model_out.data);
    
    1. string.xml
    
    <resources>
        <string name="app_name">nanodetncnnstring>
        <string-array name="model_array">
            <item>MonoDepthv2item>
    
    
    
        string-array>
        <string-array name="cpugpu_array">
            <item>CPUitem>
            <item>GPUitem>
        string-array>
    resources>
    
    1. nanodetncnn.cpp (#170) 设置模型路径和设备cpugpu选项
    JNIEXPORT jboolean JNICALL Java_com_tencent_nanodetncnn_NanoDetNcnn_loadModel(JNIEnv* env, jobject thiz, jobject assetManager, jint modelid, jint cpugpu)
    {
        // 检查一下选的模型和设备是不是在范围
        if (modelid < 0 || modelid > 2 || cpugpu < 0 || cpugpu > 1)
        {
            return JNI_FALSE;
        }
    
        AAssetManager* mgr = AAssetManager_fromJava(env, assetManager);
    
        __android_log_print(ANDROID_LOG_DEBUG, "ncnn", "loadModel %p", mgr);
    
        const char* modeltypes[] =
        {"mono-sim-opt",
        };
    
    1. gradle-wrapper.properties
    distributionBase=GRADLE_USER_HOME
    distributionPath=wrapper/dists
    zipStoreBase=GRADLE_USER_HOME
    zipStorePath=wrapper/dists
    #distributionUrl=https\://services.gradle.org/distributions/gradle-5.4.1-all.zip
    distributionUrl=https\://mirrors.cloud.tencent.com/gradle/gradle-5.4.1-all.zip
    
    1. local.properties
    ## This file must *NOT* be checked into Version Control Systems,
    # as it contains information specific to your local configuration.
    #
    # Location of the SDK. This is only used by Gradle.
    # For customization when using a Version Control System, please read the
    # header note.
    #Tue Apr 11 21:28:46 CST 2023
    sdk.dir=\\path\\to\\AppData\\Local
    ndk.dir=\\path\\to\\AppData\\Local\\ndk\\22.1.7171670
    
  • 小结
    1)Android Studio 配置过程会出现各种问题,需要耐心解决。如网络问题,SDK,NDK等路径配置问题;
    2)Android Studio检测手机设备时,需要检测相应的硬件环境,软件环境等一致;

你可能感兴趣的:(模型部署,android,NCNN,depth,monodepth)