Tengine在Ubuntu16.04上的部署及MobileNet_SSD的实时测试

Tengine在Ubuntu16.04上的部署及MobileNet_SSD的实时测试

  • 0.Tengine介绍
  • 1.Ubuntu系统烧录
  • 2. 安装Tengine
  • 3. 运行Tengine自带的MobileNet SSD并实时测试

0.Tengine介绍

OADI/Tengine | github

Tengine 是OPEN AI LAB 为嵌入式设备开发的一个轻量级、高性能并且模块化的引擎。
Tengine在嵌入式设备上支持CPU,GPU,DLA/NPU,DSP异构计算的计算框架,实现异构计算的调度器,基于ARM平台的高效的计算库实现,针对特定硬件平台的性能优化,动态规划计算图的内存使用,提供对于网络远端AI计算能力的访问支持,支持多级别并行,整个系统模块可拆卸,基于事件驱动的计算模型,吸取已有AI计算框架的优点,设计全新的计算图表示。

1.Ubuntu系统烧录

  1. 本文采用的是win7下安装Ubuntu16.04,形成单硬盘双系统。电脑原系统是win7,采用的是BIOS启动,这里要注意的是有一些电脑是UEFI启动的,启动方式决定了之后Ubuntu的安装的一些设置,这里提供了一种查询自己电脑启动方式的方法。
    Win10如何查看是否uefi启动 怎么知道BIOS或UEFI
  2. 怎么烧录进系统,网上提供了很多教程。在知道自己是传统的BIOS启动还是UEFI启动后,找对应的教程即可。
  3. Ubuntu分区,这里使用了两分法,第一个区 是 主分区,位置 是 空间起始位置,用于 Ext4日志文件系统,挂载点/,分配空间是82120MB(80GB);第二个区 是 逻辑分区,位置 是 空间起始位置,用于 交换空间,分配空间是剩下的20多个G。所以之前在windows系统下的磁盘管理要划出100G的未分配空间。
  4. 选择英文安装,之后再安装中文输入法。

2. 安装Tengine

参考博文链接: http://hey-yahei.cn/2018/08/04/RK3399-Tengine/
参考官方安装指南: https://github.com/OAID/Tengine/blob/master/doc/install.md

Ubuntu基本环境安顿好之后,接下来可以开始搭建Tengine的环境。

  1. 安装git
sudo apt-get install git
  1. 用git下载源码
git clone https://github.com/OAID/tengine
  1. 安装编译源码时需要依赖的包
 sudo apt install libprotobuf-dev protobuf-compiler libboost-all-dev libgoogle-glog-dev
 sudo apt install libopencv-dev
  1. 进入Tengine目录,复制编译的配置文件
 cd ~/tengine
 cp makefile.config.example makefile.config
  1. 编辑makefile.config文件
 gedit makefile.config

注释了 makefile.config里面中有关arm的CONFIG_ARCH_ARM64=y(猜测是因为intel上运行的),然后将 CONFIG_ARCH_BLAS=y取消注释,保存退出。具体可以参考下图。
Tengine在Ubuntu16.04上的部署及MobileNet_SSD的实时测试_第1张图片
6. 在tengine目录下编译

 make
 make install
  1. 测试
    这里的 -1的1指的是重复次数
./build/tests/bin/bench_sqz -1

0.2763 - “n02123045 tabby, tabby cat”
0.2673 - “n02123159 tiger cat”
0.1766 - “n02119789 kit fox, Vulpes macrotis”
0.0827 - “n02124075 Egyptian cat”
0.0777 - “n02085620 Chihuahua”

./build/tests/bin/bench_mobilenet -1

8.5976 - “n02123159 tiger cat”
7.9550 - “n02119022 red fox, Vulpes vulpes”
7.8679 - “n02119789 kit fox, Vulpes macrotis”
7.4274 - “n02113023 Pembroke, Pembroke Welsh corgi”
6.3647 - “n02123045 tabby, tabby cat”

3. 运行Tengine自带的MobileNet SSD并实时测试

  1. 在tengine目录下编译

1) 打开CMakeLists.txt,修改增加一句打开CMakeLists.txt,在set( INSTALL_DIR ${TENGINE_DIR}/install/)前增加一句(这里的the是指电脑名字,要根据自己的电脑名修改)

	 set( TENGINE_DIR /home/the/tengine ) 

2)没有模型文件,下载一个!
Tengine提供了一些训练好的模型——Tengine_models | 百度云(提取码:57vb)
找到mobilenet_ssd文件夹把其中的MobileNetSSD_deploy.prototxt和MobileNetSSD_deploy.caffemodel下载下来放到./models目录下就行。
3)编译

cd ~/tengine/examples/mobilenet_ssd
sudo apt-get install cmake
cmake .
make

4)运行

./MSSD

在cmake-build-debug文件夹里有一张识别结果的图片。
Tengine在Ubuntu16.04上的部署及MobileNet_SSD的实时测试_第2张图片
5)实时检测
修改mssd.cpp的代码为下,再make一下。

/*
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership.  The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * License); you may not use this file except in compliance
 * with the License.  You may obtain a copy of the License at
 *
 *   http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing,
 * software distributed under the License is distributed on an
 * AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 * KIND, either express or implied.  See the License for the
 * specific language governing permissions and limitations
 * under the License.
 */

/*
 * Copyright (c) 2018, Open AI Lab
 * Author: [email protected]
 */

#include 
#include 
#include 
#include 
#include 
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "tengine_c_api.h"
#include 
#include 
#include "common.hpp"

#define DEF_PROTO "models/MobileNetSSD_deploy.prototxt"
#define DEF_MODEL "models/MobileNetSSD_deploy.caffemodel"
#define DEF_IMAGE "tests/images/ssd_dog.jpg"

struct Box
{
    float x0;
    float y0;
    float x1;
    float y1;
    int class_idx;
    float score;
};

// void get_input_data_ssd(std::string& image_file, float* input_data, int img_h,  int img_w)
void get_input_data_ssd(cv::Mat img, float* input_data, int img_h,  int img_w)
{
    // cv::Mat img = cv::imread(image_file);

    if (img.empty())
    {
        // std::cerr << "Failed to read image file " << image_file << ".\n";
        std::cerr << "Failed to read image from camera.\n";
        return;
    }

    cv::resize(img, img, cv::Size(img_h, img_w));
    img.convertTo(img, CV_32FC3);
    float *img_data = (float *)img.data;
    int hw = img_h * img_w;

    float mean[3]={127.5,127.5,127.5};
    for (int h = 0; h < img_h; h++)
    {
        for (int w = 0; w < img_w; w++)
        {
            for (int c = 0; c < 3; c++)
            {
                input_data[c * hw + h * img_w + w] = 0.007843* (*img_data - mean[c]);
                img_data++;
            }
        }
    }
}

// void post_process_ssd(std::string& image_file,float threshold,float* outdata,int num,std::string& save_name)
void post_process_ssd(cv::Mat img, float threshold,float* outdata,int num)
{
    const char* class_names[] = {"background",
                            "aeroplane", "bicycle", "bird", "boat",
                            "bottle", "bus", "car", "cat", "chair",
                            "cow", "diningtable", "dog", "horse",
                            "motorbike", "person", "pottedplant",
                            "sheep", "sofa", "train", "tvmonitor"};
    // cv::Mat img = cv::imread(image_file);
    int raw_h = img.size().height;
    int raw_w = img.size().width;
    std::vector boxes;
    int line_width=raw_w*0.002;
    printf("detect ruesult num: %d \n",num);
    for (int i=0;i=threshold)
        {
            Box box;
            box.class_idx=outdata[0];
            box.score=outdata[1];
            box.x0=outdata[2]*raw_w;
            box.y0=outdata[3]*raw_h;
            box.x1=outdata[4]*raw_w;
            box.y1=outdata[5]*raw_h;
            boxes.push_back(box);
            printf("%s\t:%.0f%%\n", class_names[box.class_idx], box.score * 100);
            printf("BOX:( %g , %g ),( %g , %g )\n",box.x0,box.y0,box.x1,box.y1);
        }
        outdata+=6;
    }
    for(int i=0;i<(int)boxes.size();i++)
    {
        Box box=boxes[i];
        cv::rectangle(img, cv::Rect(box.x0, box.y0,(box.x1-box.x0),(box.y1-box.y0)),cv::Scalar(255, 255, 0),line_width);
        std::ostringstream score_str;
        score_str<> frame;

        for (int i = 0; i < repeat_count; i++)
        {
            get_input_data_ssd(frame, input_data, img_h,  img_w);

            gettimeofday(&t0, NULL);
            set_tensor_buffer(input_tensor, input_data, img_size * 4);
            run_graph(graph, 1);

            gettimeofday(&t1, NULL);
            float mytime = (float)((t1.tv_sec * 1000000 + t1.tv_usec) - (t0.tv_sec * 1000000 + t0.tv_usec)) / 1000;
            total_time += mytime;

        }
        std::cout << "--------------------------------------\n";
        std::cout << "repeat " << repeat_count << " times, avg time per run is " << total_time / repeat_count << " ms\n";
        tensor_t out_tensor = get_graph_output_tensor(graph, 0,0);//"detection_out");
        get_tensor_shape( out_tensor, out_dim, 4);
        outdata = (float *)get_tensor_buffer(out_tensor);

        int num=out_dim[1];
        float show_threshold=0.5;
        post_process_ssd(frame, show_threshold, outdata, num);
        cv::imshow("MSSD", frame);
        if( cv::waitKey(10) == 'q' )
            break;
    }

    postrun_graph(graph);
    free(input_data);
    destroy_runtime_graph(graph);
    remove_model(model_name);

    return 0;
}

出现这样的错误

/home/the/tengine/examples/mobilenet_ssd/MSSD
/home/the/tengine/examples/mobilenet_ssd/MSSD
proto file not specified,using /home/the/tengine/models/MobileNetSSD_deploy.prototxt by default
model file not specified,using /home/the/tengine/models/MobileNetSSD_deploy.caffemodel by default
image file not specified,using /home/the/tengine/tests/images/ssd_dog.jpg by default
tensor: detection_out created by node: detection_out is not consumed
add the node: detection_out into output list
load model done!
HIGHGUI ERROR: V4L: index 1 is not correct!
Failed to read image from camera.
--------------------------------------
repeat 1 times, avg time per run is 263.055 ms
detect ruesult num: 0 
OpenCV Error: Assertion failed (size.width>0 && size.height>0) in imshow, file /build/opencv-ys8xiq/opencv-2.4.9.1+dfsg/modules/highgui/src/window.cpp, line 269
terminate called after throwing an instance of 'cv::Exception'
  what():  /build/opencv-ys8xiq/opencv-2.4.9.1+dfsg/modules/highgui/src/window.cpp:269: error: (-215) size.width>0 && size.height>0 in function imshow


Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

发现可能是摄像头没有正确读取,找到对应代码为cv::VideoCapture capture(1);查了一下,参考OpenCV3】视频读写——cv::VideoCapture和cv::VideoWriter详解这篇博文,我们需要的功能是

从摄像机中读取视频,这种情况下,我们会给出一个标识符,用于表示我们想要访问的摄像机,及其与操作系统的握手方式。对于摄像机而言,这个标志符就是一个标志数字——如果只有1个摄像机,那么就是0

故只有一个摄像头,修改代码中的cv::VideoCapture capture(1);为
cv::VideoCapture capture(0);
再make 和. /MSSD即可实现实时检测。

你可能感兴趣的:(Arm)