OADI/Tengine | github
Tengine 是OPEN AI LAB 为嵌入式设备开发的一个轻量级、高性能并且模块化的引擎。
Tengine在嵌入式设备上支持CPU,GPU,DLA/NPU,DSP异构计算的计算框架,实现异构计算的调度器,基于ARM平台的高效的计算库实现,针对特定硬件平台的性能优化,动态规划计算图的内存使用,提供对于网络远端AI计算能力的访问支持,支持多级别并行,整个系统模块可拆卸,基于事件驱动的计算模型,吸取已有AI计算框架的优点,设计全新的计算图表示。
参考博文链接: http://hey-yahei.cn/2018/08/04/RK3399-Tengine/
参考官方安装指南: https://github.com/OAID/Tengine/blob/master/doc/install.md
Ubuntu基本环境安顿好之后,接下来可以开始搭建Tengine的环境。
sudo apt-get install git
git clone https://github.com/OAID/tengine
sudo apt install libprotobuf-dev protobuf-compiler libboost-all-dev libgoogle-glog-dev
sudo apt install libopencv-dev
cd ~/tengine
cp makefile.config.example makefile.config
gedit makefile.config
注释了 makefile.config里面中有关arm的CONFIG_ARCH_ARM64=y(猜测是因为intel上运行的),然后将 CONFIG_ARCH_BLAS=y取消注释,保存退出。具体可以参考下图。
6. 在tengine目录下编译
make
make install
./build/tests/bin/bench_sqz -1
0.2763 - “n02123045 tabby, tabby cat”
0.2673 - “n02123159 tiger cat”
0.1766 - “n02119789 kit fox, Vulpes macrotis”
0.0827 - “n02124075 Egyptian cat”
0.0777 - “n02085620 Chihuahua”
./build/tests/bin/bench_mobilenet -1
8.5976 - “n02123159 tiger cat”
7.9550 - “n02119022 red fox, Vulpes vulpes”
7.8679 - “n02119789 kit fox, Vulpes macrotis”
7.4274 - “n02113023 Pembroke, Pembroke Welsh corgi”
6.3647 - “n02123045 tabby, tabby cat”
1) 打开CMakeLists.txt,修改增加一句打开CMakeLists.txt,在set( INSTALL_DIR ${TENGINE_DIR}/install/)前增加一句(这里的the是指电脑名字,要根据自己的电脑名修改)
set( TENGINE_DIR /home/the/tengine )
2)没有模型文件,下载一个!
Tengine提供了一些训练好的模型——Tengine_models | 百度云(提取码:57vb)
找到mobilenet_ssd文件夹把其中的MobileNetSSD_deploy.prototxt和MobileNetSSD_deploy.caffemodel下载下来放到./models目录下就行。
3)编译
cd ~/tengine/examples/mobilenet_ssd
sudo apt-get install cmake
cmake .
make
4)运行
./MSSD
在cmake-build-debug文件夹里有一张识别结果的图片。
5)实时检测
修改mssd.cpp的代码为下,再make一下。
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* License); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
/*
* Copyright (c) 2018, Open AI Lab
* Author: [email protected]
*/
#include
#include
#include
#include
#include
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "tengine_c_api.h"
#include
#include
#include "common.hpp"
#define DEF_PROTO "models/MobileNetSSD_deploy.prototxt"
#define DEF_MODEL "models/MobileNetSSD_deploy.caffemodel"
#define DEF_IMAGE "tests/images/ssd_dog.jpg"
struct Box
{
float x0;
float y0;
float x1;
float y1;
int class_idx;
float score;
};
// void get_input_data_ssd(std::string& image_file, float* input_data, int img_h, int img_w)
void get_input_data_ssd(cv::Mat img, float* input_data, int img_h, int img_w)
{
// cv::Mat img = cv::imread(image_file);
if (img.empty())
{
// std::cerr << "Failed to read image file " << image_file << ".\n";
std::cerr << "Failed to read image from camera.\n";
return;
}
cv::resize(img, img, cv::Size(img_h, img_w));
img.convertTo(img, CV_32FC3);
float *img_data = (float *)img.data;
int hw = img_h * img_w;
float mean[3]={127.5,127.5,127.5};
for (int h = 0; h < img_h; h++)
{
for (int w = 0; w < img_w; w++)
{
for (int c = 0; c < 3; c++)
{
input_data[c * hw + h * img_w + w] = 0.007843* (*img_data - mean[c]);
img_data++;
}
}
}
}
// void post_process_ssd(std::string& image_file,float threshold,float* outdata,int num,std::string& save_name)
void post_process_ssd(cv::Mat img, float threshold,float* outdata,int num)
{
const char* class_names[] = {"background",
"aeroplane", "bicycle", "bird", "boat",
"bottle", "bus", "car", "cat", "chair",
"cow", "diningtable", "dog", "horse",
"motorbike", "person", "pottedplant",
"sheep", "sofa", "train", "tvmonitor"};
// cv::Mat img = cv::imread(image_file);
int raw_h = img.size().height;
int raw_w = img.size().width;
std::vector boxes;
int line_width=raw_w*0.002;
printf("detect ruesult num: %d \n",num);
for (int i=0;i=threshold)
{
Box box;
box.class_idx=outdata[0];
box.score=outdata[1];
box.x0=outdata[2]*raw_w;
box.y0=outdata[3]*raw_h;
box.x1=outdata[4]*raw_w;
box.y1=outdata[5]*raw_h;
boxes.push_back(box);
printf("%s\t:%.0f%%\n", class_names[box.class_idx], box.score * 100);
printf("BOX:( %g , %g ),( %g , %g )\n",box.x0,box.y0,box.x1,box.y1);
}
outdata+=6;
}
for(int i=0;i<(int)boxes.size();i++)
{
Box box=boxes[i];
cv::rectangle(img, cv::Rect(box.x0, box.y0,(box.x1-box.x0),(box.y1-box.y0)),cv::Scalar(255, 255, 0),line_width);
std::ostringstream score_str;
score_str<> frame;
for (int i = 0; i < repeat_count; i++)
{
get_input_data_ssd(frame, input_data, img_h, img_w);
gettimeofday(&t0, NULL);
set_tensor_buffer(input_tensor, input_data, img_size * 4);
run_graph(graph, 1);
gettimeofday(&t1, NULL);
float mytime = (float)((t1.tv_sec * 1000000 + t1.tv_usec) - (t0.tv_sec * 1000000 + t0.tv_usec)) / 1000;
total_time += mytime;
}
std::cout << "--------------------------------------\n";
std::cout << "repeat " << repeat_count << " times, avg time per run is " << total_time / repeat_count << " ms\n";
tensor_t out_tensor = get_graph_output_tensor(graph, 0,0);//"detection_out");
get_tensor_shape( out_tensor, out_dim, 4);
outdata = (float *)get_tensor_buffer(out_tensor);
int num=out_dim[1];
float show_threshold=0.5;
post_process_ssd(frame, show_threshold, outdata, num);
cv::imshow("MSSD", frame);
if( cv::waitKey(10) == 'q' )
break;
}
postrun_graph(graph);
free(input_data);
destroy_runtime_graph(graph);
remove_model(model_name);
return 0;
}
出现这样的错误
/home/the/tengine/examples/mobilenet_ssd/MSSD
/home/the/tengine/examples/mobilenet_ssd/MSSD
proto file not specified,using /home/the/tengine/models/MobileNetSSD_deploy.prototxt by default
model file not specified,using /home/the/tengine/models/MobileNetSSD_deploy.caffemodel by default
image file not specified,using /home/the/tengine/tests/images/ssd_dog.jpg by default
tensor: detection_out created by node: detection_out is not consumed
add the node: detection_out into output list
load model done!
HIGHGUI ERROR: V4L: index 1 is not correct!
Failed to read image from camera.
--------------------------------------
repeat 1 times, avg time per run is 263.055 ms
detect ruesult num: 0
OpenCV Error: Assertion failed (size.width>0 && size.height>0) in imshow, file /build/opencv-ys8xiq/opencv-2.4.9.1+dfsg/modules/highgui/src/window.cpp, line 269
terminate called after throwing an instance of 'cv::Exception'
what(): /build/opencv-ys8xiq/opencv-2.4.9.1+dfsg/modules/highgui/src/window.cpp:269: error: (-215) size.width>0 && size.height>0 in function imshow
Process finished with exit code 134 (interrupted by signal 6: SIGABRT)
发现可能是摄像头没有正确读取,找到对应代码为cv::VideoCapture capture(1);查了一下,参考OpenCV3】视频读写——cv::VideoCapture和cv::VideoWriter详解这篇博文,我们需要的功能是
从摄像机中读取视频,这种情况下,我们会给出一个标识符,用于表示我们想要访问的摄像机,及其与操作系统的握手方式。对于摄像机而言,这个标志符就是一个标志数字——如果只有1个摄像机,那么就是0
故只有一个摄像头,修改代码中的cv::VideoCapture capture(1);为
cv::VideoCapture capture(0);
再make 和. /MSSD即可实现实时检测。