图像特征匹配

简介

本文主要演示利用opencv自带的特征检测算子做图像的特征匹配。

检测算子包括

• SIFT
• SURF
• ORB

特征描述子提取算子包括

• SIFT
• SURF
• ORB
• VGG

匹配算法

• FlannBasedMatcher

本文不对相关原理做介绍，只演示其用法，如果对原理感兴趣可以查阅相关文档学习。

首先，包含所需要的头文件

#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 

定义几个辅助函数，其声明如下

template 
void detect_keypoint(const cv::Ptr det, const cv::Mat &src, std::vector &kpts);

template 
void extract_feature_desc(const cv::Ptr feat, const cv::Mat &src, std::vector &kpts, cv::Mat &desc);

template 
void detectAndCompute(const cv::Ptr det, const cv::Mat &src, std::vector &kpts, cv::Mat &desc);

/*********************************************************
 * @brief         两幅图像特征匹配
 * @Inparam       desc1
 * @Inparam       desc2
 * @Outparam      good_matches
 * @return
 * @author        hscoder
 * @date          2020-03-28
********************************************************/
void match(const cv::Mat &desc1, const cv::Mat &desc2, std::vector &good_matches , bool orb = false);

void show_match(cv::Mat &left, const std::vector &kpt1, cv::Mat &right, const std::vector &kpt2, std::vector &good_matches, const char *winName);

这里定义了三个模板函数，分别是detect_keypoint、extract_feature_desc、detectAndCompute

• detect_keypoint 检测关键点函数，可以传入sift、surf、orb等检测对象
• extract_feature_desc 提取特征描述子，可以传入sift、surf、orb、vgg等
• detectAndCompute 检测关键点并提取特征描述子，支持sift、surf、orb

另外两个

• match 特征匹配，利用Lowe’s algorithm,获取优秀匹配点
• show_match 图像匹配可视化显示

主要代码如下

        cv::Mat left_image = imread("/Users/han/Deeplearning/opencv_tutorial/dnn_tutorial/left.jpeg");
	cv::Mat right_image = imread("/Users/han/Deeplearning/opencv_tutorial/dnn_tutorial/right.jpeg");

	// 定义sift检测器
	cv::Ptr sift = cv::xfeatures2d::SIFT::create();
	// 定义surf检测器
	cv::Ptr surf = cv::xfeatures2d::SURF::create(1000);
	
	// 定义描述子提取器
	// sift
	cv::Ptr sift_desc = cv::xfeatures2d::SiftDescriptorExtractor::create();
	// surf
	cv::Ptr surf_desc = cv::xfeatures2d::SurfDescriptorExtractor::create();
	// vgg
	cv::Ptr vgg_desc = cv::xfeatures2d::VGG::create();

	std::vector kpts1, kpts2;
	cv::Mat desc1, desc2;
	std::vector good_matches;
	auto start_time = std::chrono::steady_clock::now();
	auto end_time = std::chrono::steady_clock::now();
	auto cost_time = std::chrono::duration_cast(end_time - start_time).count();

	start_time = std::chrono::steady_clock::now();
	std::cout << "-----------------sift------------------" << std::endl;
	detectAndCompute(sift, left_image, kpts1, desc1);
	detectAndCompute(sift, right_image, kpts2, desc2);
	std::cout << "left image detect " << kpts1.size() << " keypoint" << std::endl;
	std::cout << "right image detect " << kpts2.size() << " keypoint" << std::endl;

	std::cout << "desc1 size: (" << desc1.rows << " , " << desc1.cols << ")" << std::endl;
	std::cout << "desc2 size: (" << desc2.rows << " , " << desc2.cols << ")" << std::endl;

	end_time = std::chrono::steady_clock::now();
	cost_time = std::chrono::duration_cast(end_time - start_time).count();
	std::cout << "cost time: " << cost_time << ("ms") << std::endl;

	match(desc1, desc2, good_matches);
	std::cout << "经过匹配之后仍有： " << good_matches.size() << "对匹配对" << std::endl;
	show_match(left_image, kpts1, right_image, kpts2, good_matches, "sift");
	std::cout << "---------------------------------------";

	kpts1.clear();
	kpts2.clear();
	good_matches.clear();

	std::cout << "-----------------surf------------------" << std::endl;
	start_time = std::chrono::steady_clock::now();
	detectAndCompute(surf, left_image, kpts1, desc1);
	detectAndCompute(surf, right_image, kpts2, desc2);
	std::cout << "left image detect " << kpts1.size() << " keypoint" << std::endl;
	std::cout << "right image detect " << kpts2.size() << " keypoint" << std::endl;

	std::cout << "desc1 size: (" << desc1.rows << " , " << desc1.cols << ")" << std::endl;
	std::cout << "desc2 size: (" << desc2.rows << " , " << desc2.cols << ")" << std::endl;

	match(desc1, desc2, good_matches);
	std::cout << "good match number is: " << good_matches.size() << std::endl;
	end_time = std::chrono::steady_clock::now();
	cost_time = std::chrono::duration_cast(end_time - start_time).count();
	std::cout << "cost time: " << cost_time << ("ms") << std::endl;
	show_match(left_image, kpts1, right_image, kpts2, good_matches, "surf");
	std::cout << "---------------------------------------";

	kpts1.clear();
	kpts2.clear();
	good_matches.clear();
	std::cout << "---------------surf+vgg----------------" << std::endl;
	start_time = std::chrono::steady_clock::now();

	detect_keypoint(surf, left_image , kpts1);
	detect_keypoint(surf , right_image , kpts2);
	std::cout << "left image detect " << kpts1.size() << " keypoint" << std::endl;
	std::cout << "right image detect " << kpts2.size() << " keypoint" << std::endl;

	extract_feature_desc(vgg_desc , left_image , kpts1 , desc1);
	extract_feature_desc(vgg_desc , right_image , kpts2 , desc2);

	std::cout << "desc1 size: (" << desc1.rows << " , " << desc1.cols << ")" << std::endl;
	std::cout << "desc2 size: (" << desc2.rows << " , " << desc2.cols << ")" << std::endl;

	match(desc1, desc2, good_matches);
	std::cout << "good match number is: " << good_matches.size() << std::endl;
	end_time = std::chrono::steady_clock::now();
	cost_time = std::chrono::duration_cast(end_time - start_time).count();
	std::cout << "cost time: " << cost_time << ("ms") << std::endl;
	show_match(left_image, kpts1, right_image, kpts2, good_matches, "surf+vgg");
	std::cout << "---------------------------------------";

	return 0;

贴上不同算法的特征匹配图
Sift

Surf

Orb

Sift+VGG

Surf+VGG

上述匹配我采用的FlannBasedMatcher，opencv中有一个匹配器的抽象基类，其具体类有：FlannBasedMatcher、BFMatcher
因此匹配器可以由静态工厂函数方法直接创建，如下

cv::Ptr matcher = cv::DescriptorMatcher::create(const string& descriptorMatcherType)

支持的匹配类型有

• BruteForce (it uses L2 )
• BruteForce-L1
• BruteForce-Hamming
• BruteForce-Hamming(2)
• FlannBased

如有兴趣，可以用一个swich实现多种方法切换，比较一下效果。
本人以后会在个人公众号上定期更新一些cv相关的内容，如有兴趣可以关注本人公众号