转载地址
前言
当前,在目标检测领域,基于深度学习的目标检测方法在准确度上碾压传统的方法。基于深度学习的目标检测先后出现了RCNN,FastRCNN,FasterRCNN, 端到端目标检测方法YOLO,YOLO-9000,YOLO-v3, MobileNet-SSD,以及Mask-RCNN等。MobileNet是一种轻量级的网络,本文基于MobileNet-SSD+opencv实现目标检测。
MobileNet-SSD caffe
目前MobileNet有基于caffe框架训练好的,caffe本身就是C++实现的,因此网上的大部分opencv调用MobileNet都是C++代码。本人先采用vs+opencv3.4.1成功测试之后,再用Java代码进行移植。
Visual Stuido 2017配置opencv过程就不赘述了
MobileNet-SSD训练好的caffe模型在上面的MobileNet-SSD caffe链接下载
#include
#include
#include
using namespace std;
using namespace cv;
using namespace cv::dnn;
class Object
{
public:
Object();
Object(int index, float confidence, String name, Rect rect);
~Object();
public:
int index;
String name;
float confidence;
Rect rect;
private:
};
Object::Object() {
}
Object::Object(int index,float confidence,String name,Rect rect) {
this->index = index;
this->confidence = confidence;
this->name = name;
this->rect = rect;
}
Object::~Object() {
}
//----------------------------全局常量----------------------------------
//配置好protxt文件,网络结构描述文件
//配置好caffemodel文件,训练好的网络权重
const String PROTOTX_FILE ="MobileNetSSD\\MobileNetSSD_deploy.prototxt";
const String CAFFE_MODEL_FILE = "MobileNet-SSD\\MobileNetSSD_deploy.caffemodel";
const String classNames[] = { "background", "aeroplane", "bicycle", "bird", "boat", "bottle", "bus", "car", "cat", "chair",
"cow", "diningtable", "dog", "horse", "motorbike", "person", "pottedplant", "sheep", "sofa", "train", "tvmonitor" };
const float CONF_THRESH = 0.7f;
int main() {
//------------------------实例化网络----------------------------
Net mobileNetSSD = readNetFromCaffe(PROTOTX_FILE, CAFFE_MODEL_FILE);
if (mobileNetSSD.empty()) {
cerr << "加载网络失败!" << endl;
return -1;
}
TickMeter t;
//----------------------设置网络输入-----------------------
Mat srcImg = imread("D:\\小可爱\\Java学习\\day-6\\代码\\opencv调用MobileNet-SSD\\MobileNet-test.jpg");
//将二维图像转换为CNN输入的张量Tensor,作为网络的输入
mobileNetSSD.setInput(blobFromImage(srcImg, 1.0 / 127.5, Size(300, 300), Scalar(127.5, 127.5, 127.5), true, false));
t.start();
//--------------------CNN网络前向计算----------------------
Mat netOut = mobileNetSSD.forward();
t.stop();
cout << "检测时间=" << t.getTimeMilli() << "ms" << endl;
//----------------------解析计算结果-----------------------
vector
测试图像:
代码说明
blobFromImage()
将输入的二维图像转换为一个4维的张量/高维矩阵,4维张量的顺序为NCHW(N个数,C通道数,H高度,W宽度 ) /** @brief Creates 4-dimensional blob from series of images. Optionally resizes and
* crops @p images from center, subtract @p mean values, scales values by @p scalefactor,
* swap Blue and Red channels.
* @param images input images (all with 1-, 3- or 4-channels).
* @param size spatial size for output image
* @param mean scalar with mean values which are subtracted from channels. Values are intended
* to be in (mean-R, mean-G, mean-B) order if @p image has BGR ordering and @p swapRB is true.
* @param scalefactor multiplier for @p images values.
* @param swapRB flag which indicates that swap first and last channels
* in 3-channel image is necessary.
* @param crop flag which indicates whether image will be cropped after resize or not
* @details if @p crop is true, input image is resized so one side after resize is equal to corresponing
* dimension in @p size and another one is equal or larger. Then, crop from the center is performed.
* If @p crop is false, direct resize without cropping and preserving aspect ratio is performed.
* @returns 4-dimansional Mat with NCHW dimensions order.
*/
CV_EXPORTS_W Mat blobFromImages(const std::vector& images, double scalefactor=1.0,
Size size = Size(), const Scalar& mean = Scalar(), bool swapRB=true, bool crop=true);
net.forward()
深度神经网络的前向传播计算,返回值也是一个4D的张量
/** @brief Runs forward pass to compute output of layer with name @p outputName.
* @param outputName name for layer which output is needed to get
* @return blob for first output of specified layer.
* @details By default runs forward pass for the whole network.
*/
CV_WRAP Mat forward(const String& outputName = String());
调试步骤,输入图像为RGB图像,包含dog,person
调试步骤,网络的最终计算输出是一个张量,需要转换为一个2维矩形Mat,是一个7(width)*6(height)的float矩阵,第2列的整数代表目标类别的索引,第3列代表检测结果的置信度,最后4列是归一化的目标矩形框。网络只输出置信度最大的前6个目标,因此输出矩阵的形状为7x6.
运行结果
检测到了5个目标,2个dog+3个person,后面坐着的2个人以及椅子上中间的人露检了。
笔者查看了好几遍Java封装的Mat,没有发现如何将一个高维/4D张量转换为2D矩阵的方法,比较坑。在C++ Mat中,Mat有一个成员变量size
,进一步查看之后发现是一个结构体,里面封装了指针,在Java Mat中找不到对应的。卡在这里了,只能用JNI了,好麻烦。
以后有时间再继续倒腾Java版本的 MobileNet-SSD。