SLAM十四讲-ch11-回环检测-代码注释

一、创建字典

字典：是由许多单词构成
一个单词：是由特征点的集合构成
词袋：由许多【单词以及单词的权重】构成 （每个单词都会有自己对应的权重）
权重：其实就是其单词的区分度 ，因为有些单词在图片中很常见（出现的频率高），那么它的区分度就应该低，相反不常见的单词（出现的频率低），区分度应该高。
权重的公式：见slambook2 p295页 公式11.7
图片与图片之间的评分依赖于每个图片
源码如下：

//
// Created by nnz on 2020/11/15.
//
#include 
#include 
#include 
#include 
#include 
#include 
#include "DBoW3/DBoW3.h"
#include 

using namespace std;
using namespace cv;

/****************************

该程序用来创建一组图片（10张）的字典

步骤:
1、首先要用cv的特征检测器 找出图片的特征点（包括检测角点，以及计算描述子----ORB）
2、用DBoW3库的creat()函数 创建词典
*****************************/


int main(int argc,char **argv)
{

    //读取图片
    cout<<" reading images ...... "<<endl;
    vector<Mat> images;//10张图片放在容器里面
    for (int i = 0; i < 10; i++) {
        string path = "/home/nnz/data/slam_pratice/loop_closing/data/" + to_string(i + 1) + ".png";
        images.push_back(imread(path));
    }
   /* for ( int i=0; i<10; i++ )
    {
        string path = "../data/"+to_string(i+1)+".png";
        images.push_back( imread(path) );
    }*/
    //检测ORB特征点
    cout <<" detecting ORB ...... "<< endl;
    Ptr< Feature2D > detector = ORB::create();
    vector<Mat> descriptors;
    //得到各个图片的ORB特征点
    for( Mat& image:images )
    {
        vector<KeyPoint> keypoints;
        Mat descriptor;
        detector->detectAndCompute(image,Mat(),keypoints,descriptor);
        descriptors.push_back(descriptor);
    }
    //创建字典
    cout<<" creating vcabulary ......"<<endl;
    DBoW3::Vocabulary vocab;
    vocab.create(descriptors);//创建10张图片的字典
    cout<<"vocabulary info: "<<vocab<<endl;
    vocab.save("vocabulary.yml.gz");//保存字典至vocabulary.yml.gz
    cout<<" done "<<endl;
    return 0;

}

运行结果如下：

reading images ...... 
 detecting ORB ...... 
[ INFO:0] Initialize OpenCL runtime...
 creating vcabulary ......
vocabulary info: Vocabulary: k = 10, L = 5, Weighting = tf-idf, Scoring = L1-norm, Number of words = 4970
 done 

其结果就是创建了一个字典 vocabulary.yml.gz ，该字典由data中的10张图片的单词构成 （单词由一部分的特征点的集合构成）

二、利用已经创建的vocabulary.yml.gz字典，我们去得到图片与图片之间相似性的评分

源码如下：

//
// Created by nnz on 2020/11/15.
//
#include "DBoW3/DBoW3.h"
#include 
#include 
#include 
#include 
#include 
#include 
#include 

using namespace cv;
using namespace std;
/****************************

该程序用创建好的字典进行图片之间相似性的评分

*****************************/


int main()
{
    //读取字典
    cout<<" reading vocabulary ..... "<<endl;
    DBoW3::Vocabulary vocab("./vocabulary.yml.gz");//使用creat_vb.cpp程序创建的字典
    if(vocab.empty())
    {
        cerr<<"Vocabulary does not exit !"<<endl;
        return 1;
    }
    cout<<"reading images ....."<<endl;
    vector<Mat> images;//10张图片放在容器里面
    //用 format可能有点问题
    /*boost::format format_("../data/%d.png");
    for(int i=1;i<=10;i++)
    {
        Mat im=imread((format_ % i).str(),0);
        images.push_back(im);
    }*/
    for (int i = 0; i < 10; i++) {
        string path = "/home/nnz/data/slam_pratice/loop_closing/data/" + to_string(i + 1) + ".png";
        images.push_back(imread(path));
    }
    //检测ORB特征点
    cout <<" detecting ORB ...... "<< endl;
    Ptr< Feature2D > detector = ORB::create();
    vector<Mat> descriptors;
    //得到各个图片的ORB特征点
    for( Mat &image:images )
    {
        vector<KeyPoint> keypoints;
        Mat descriptor;
        detector->detectAndCompute(image,Mat(),keypoints,descriptor);
        descriptors.push_back(descriptor);
    }
    //直接图片比较
    cout<<" comparing images with images "<<endl;
    for(int i=0;i<images.size();i++)
    {

        DBoW3::BowVector V1;//定义词袋V1
        vocab.transform(descriptors[i],V1);//把第i个图片的ORB特征点的对应的词袋(bag of words)
        for(int j=i;j<images.size();j++)
        {
            DBoW3::BowVector V2;//定义词袋V2
            vocab.transform(descriptors[j],V2);//把第i个图片之后的第j个图片的ORB特征点的对应的词袋(bag of words)
            double score = vocab.score(V1, V2);//score是两个图片的相似性评分
            cout << "image " << i << " vs image " << j << " : " << score << endl;
        }
        cout << endl;
    }
    //或者 与 字典 比较
    cout<<" comparing images with vocabulary "<<endl;
    DBoW3::Database db(vocab, false,0);
    for(int i=0;i<descriptors.size();i++)
    {
        db.add(descriptors[i]);//把10张图片的ORB特征都放在一个database里
    }
    cout<<"database info:"<<db<<endl;
    for(int i=0;i<descriptors.size();i++)
    {
        DBoW3::QueryResults ret;
        db.query(descriptors[i],ret,4);//四个图片的评分 找到与第i个图片评分最高的4个图片
        cout << "searching for image " << i << " returns " << ret << endl << endl;
    }
    cout << "done." << endl;
    return  0;
}

运行结果如下：

reading vocabulary ..... 
reading images .....
 detecting ORB ...... 
[ INFO:0] Initialize OpenCL runtime...
 comparing images with images 
image 0 vs image 0 : 1
image 0 vs image 1 : 0.0238961
image 0 vs image 2 : 0.0198893
image 0 vs image 3 : 0.0339413
image 0 vs image 4 : 0.0160477
image 0 vs image 5 : 0.0300543
image 0 vs image 6 : 0.0222911
image 0 vs image 7 : 0.0197256
image 0 vs image 8 : 0.0228385
image 0 vs image 9 : 0.0583173

image 1 vs image 1 : 1
image 1 vs image 2 : 0.0422068
image 1 vs image 3 : 0.0282324
image 1 vs image 4 : 0.0225322
image 1 vs image 5 : 0.025146
image 1 vs image 6 : 0.0161122
image 1 vs image 7 : 0.0166048
image 1 vs image 8 : 0.0350047
image 1 vs image 9 : 0.0326933

image 2 vs image 2 : 1
image 2 vs image 3 : 0.0361374
image 2 vs image 4 : 0.0330401
image 2 vs image 5 : 0.0189786
image 2 vs image 6 : 0.0182909
image 2 vs image 7 : 0.0137075
image 2 vs image 8 : 0.0270761
image 2 vs image 9 : 0.0219518

image 3 vs image 3 : 1
image 3 vs image 4 : 0.0304209
image 3 vs image 5 : 0.03611
image 3 vs image 6 : 0.0205856
image 3 vs image 7 : 0.0208058
image 3 vs image 8 : 0.0312382
image 3 vs image 9 : 0.0329747

image 4 vs image 4 : 1
image 4 vs image 5 : 0.0498645
image 4 vs image 6 : 0.0345081
image 4 vs image 7 : 0.0227451
image 4 vs image 8 : 0.0208472
image 4 vs image 9 : 0.0266803

image 5 vs image 5 : 1
image 5 vs image 6 : 0.0198106
image 5 vs image 7 : 0.0162269
image 5 vs image 8 : 0.0259153
image 5 vs image 9 : 0.0220667

image 6 vs image 6 : 1
image 6 vs image 7 : 0.0212959
image 6 vs image 8 : 0.0188494
image 6 vs image 9 : 0.0208606

image 7 vs image 7 : 1
image 7 vs image 8 : 0.012798
image 7 vs image 9 : 0.0205016

image 8 vs image 8 : 1
image 8 vs image 9 : 0.0228186

image 9 vs image 9 : 1

 comparing images with vocabulary 
database info:Database: Entries = 10, Using direct index = no. Vocabulary: k = 10, L = 5, Weighting = tf-idf, Scoring = L1-norm, Number of words = 4970
searching for image 0 returns 4 results:
<EntryId: 0, Score: 1>
<EntryId: 9, Score: 0.0583173>
<EntryId: 3, Score: 0.0339413>
<EntryId: 5, Score: 0.0300543>

searching for image 1 returns 4 results:
<EntryId: 1, Score: 1>
<EntryId: 2, Score: 0.0422068>
<EntryId: 8, Score: 0.0350047>
<EntryId: 9, Score: 0.0326933>

searching for image 2 returns 4 results:
<EntryId: 2, Score: 1>
<EntryId: 1, Score: 0.0422068>
<EntryId: 3, Score: 0.0361374>
<EntryId: 4, Score: 0.0330401>

searching for image 3 returns 4 results:
<EntryId: 3, Score: 1>
<EntryId: 2, Score: 0.0361374>
<EntryId: 5, Score: 0.03611>
<EntryId: 0, Score: 0.0339413>

searching for image 4 returns 4 results:
<EntryId: 4, Score: 1>
<EntryId: 5, Score: 0.0498645>
<EntryId: 6, Score: 0.0345081>
<EntryId: 2, Score: 0.0330401>

searching for image 5 returns 4 results:
<EntryId: 5, Score: 1>
<EntryId: 4, Score: 0.0498645>
<EntryId: 3, Score: 0.03611>
<EntryId: 0, Score: 0.0300543>

searching for image 6 returns 4 results:
<EntryId: 6, Score: 1>
<EntryId: 4, Score: 0.0345081>
<EntryId: 0, Score: 0.0222911>
<EntryId: 7, Score: 0.0212959>

searching for image 7 returns 4 results:
<EntryId: 7, Score: 1>
<EntryId: 4, Score: 0.0227451>
<EntryId: 6, Score: 0.0212959>
<EntryId: 3, Score: 0.0208058>

searching for image 8 returns 4 results:
<EntryId: 8, Score: 1>
<EntryId: 1, Score: 0.0350047>
<EntryId: 3, Score: 0.0312382>
<EntryId: 2, Score: 0.0270761>

searching for image 9 returns 4 results:
<EntryId: 9, Score: 1>
<EntryId: 0, Score: 0.0583173>
<EntryId: 3, Score: 0.0329747>
<EntryId: 1, Score: 0.0326933>

done.

三、利用规模更大的字典进行图片与图片之间相似性的评分，（该字典已经给出，不用自己去创建 vocab_larger.yml.gz）

源码如下：

//
// Created by nnz on 2020/11/15.
//

#include "DBoW3/DBoW3.h"
#include 
#include 
#include 
#include 
#include 
#include 
#include 

using namespace cv;
using namespace std;
/****************************

该程序用创建好的字典进行图片之间相似性的评分

*****************************/


int main()
{
    //读取字典
    cout<<" reading vocabulary ..... "<<endl;
    DBoW3::Vocabulary vocab("./vocab_larger.yml.gz");//使用creat_vb.cpp程序创建的字典
    if(vocab.empty())
    {
        cerr<<"Vocabulary does not exit !"<<endl;
        return 1;
    }
    cout<<"reading images ....."<<endl;
    vector<Mat> images;//10张图片放在容器里面
    //用 format可能有点问题
    /*boost::format format_("../data/%d.png");
    for(int i=1;i<=10;i++)
    {
        Mat im=imread((format_ % i).str(),0);
        images.push_back(im);
    }*/
    for (int i = 0; i < 10; i++) {
        string path = "/home/nnz/data/slam_pratice/loop_closing/data/" + to_string(i + 1) + ".png";
        images.push_back(imread(path));
    }
    //检测ORB特征点
    cout <<" detecting ORB ...... "<< endl;
    Ptr< Feature2D > detector = ORB::create();
    vector<Mat> descriptors;
    //得到各个图片的ORB特征点
    for( Mat &image:images )
    {
        vector<KeyPoint> keypoints;
        Mat descriptor;
        detector->detectAndCompute(image,Mat(),keypoints,descriptor);
        descriptors.push_back(descriptor);
    }
    //直接图片比较
    cout<<" comparing images with images "<<endl;
    for(int i=0;i<images.size();i++)
    {

        DBoW3::BowVector V1;//定义词袋V1
        vocab.transform(descriptors[i],V1);//把第i个图片的ORB特征点的对应的词袋(bag of words)
        for(int j=i;j<images.size();j++)
        {
            DBoW3::BowVector V2;//定义词袋V2
            vocab.transform(descriptors[j],V2);//把第i个图片之后的第j个图片的ORB特征点的对应的词袋(bag of words)
            double score = vocab.score(V1, V2);//score是两个图片的相似性评分
            cout << "image " << i << " vs image " << j << " : " << score << endl;
        }
        cout << endl;
    }
    //或者 与 字典 比较
    cout<<" comparing images with vocabulary "<<endl;
    DBoW3::Database db(vocab, false,0);
    for(int i=0;i<descriptors.size();i++)
    {
        db.add(descriptors[i]);//把10张图片的ORB特征都放在一个database里
    }
    cout<<"database info:"<<db<<endl;
    for(int i=0;i<descriptors.size();i++)
    {
        DBoW3::QueryResults ret;
        db.query(descriptors[i],ret,4);//四个图片的评分 找到与第i个图片评分最高的4个图片
        cout << "searching for image " << i << " returns " << ret << endl << endl;
    }
    cout << "done." << endl;
    return  0;
}

运行结果如下：

 reading vocabulary ..... 
reading images .....
 detecting ORB ...... 
[ INFO:0] Initialize OpenCL runtime...
 comparing images with images 
image 0 vs image 0 : 1
image 0 vs image 1 : 0.00201064
image 0 vs image 2 : 0.0029546
image 0 vs image 3 : 0.00785156
image 0 vs image 4 : 0.00681504
image 0 vs image 5 : 0.00629842
image 0 vs image 6 : 0.00452653
image 0 vs image 7 : 0.00196114
image 0 vs image 8 : 0.00849321
image 0 vs image 9 : 0.0306572

image 1 vs image 1 : 1
image 1 vs image 2 : 0.025082
image 1 vs image 3 : 0.00597156
image 1 vs image 4 : 0.00364751
image 1 vs image 5 : 0.00458698
image 1 vs image 6 : 0.00587741
image 1 vs image 7 : 0.00521632
image 1 vs image 8 : 0.00661743
image 1 vs image 9 : 0.0059525

image 2 vs image 2 : 1
image 2 vs image 3 : 0.00678883
image 2 vs image 4 : 0.0039245
image 2 vs image 5 : 0.0104254
image 2 vs image 6 : 0.00791604
image 2 vs image 7 : 0.00499522
image 2 vs image 8 : 0.00497627
image 2 vs image 9 : 0.00686568

image 3 vs image 3 : 1
image 3 vs image 4 : 0.00176185
image 3 vs image 5 : 0.00881794
image 3 vs image 6 : 0.00879057
image 3 vs image 7 : 0.00876193
image 3 vs image 8 : 0.00799292
image 3 vs image 9 : 0.00389382

image 4 vs image 4 : 1
image 4 vs image 5 : 0.00712535
image 4 vs image 6 : 0.00795609
image 4 vs image 7 : 0.0055488
image 4 vs image 8 : 0.00165572
image 4 vs image 9 : 0.00515797

image 5 vs image 5 : 1
image 5 vs image 6 : 0.0027527
image 5 vs image 7 : 0.00130352
image 5 vs image 8 : 0.00576963
image 5 vs image 9 : 0.00325172

image 6 vs image 6 : 1
image 6 vs image 7 : 0.0054425
image 6 vs image 8 : 0.00473875
image 6 vs image 9 : 0.00410202

image 7 vs image 7 : 1
image 7 vs image 8 : 0.00664312
image 7 vs image 9 : 0.00350397

image 8 vs image 8 : 1
image 8 vs image 9 : 0.00685142

image 9 vs image 9 : 1

 comparing images with vocabulary 
database info:Database: Entries = 10, Using direct index = no. Vocabulary: k = 10, L = 5, Weighting = tf-idf, Scoring = L1-norm, Number of words = 99566
searching for image 0 returns 4 results:
<EntryId: 0, Score: 1>
<EntryId: 9, Score: 0.0306572>
<EntryId: 8, Score: 0.00849321>
<EntryId: 3, Score: 0.00785156>

searching for image 1 returns 4 results:
<EntryId: 1, Score: 1>
<EntryId: 2, Score: 0.025082>
<EntryId: 8, Score: 0.00661743>
<EntryId: 3, Score: 0.00597156>

searching for image 2 returns 4 results:
<EntryId: 2, Score: 1>
<EntryId: 1, Score: 0.025082>
<EntryId: 5, Score: 0.0104254>
<EntryId: 6, Score: 0.00791604>

searching for image 3 returns 4 results:
<EntryId: 3, Score: 1>
<EntryId: 5, Score: 0.00881794>
<EntryId: 6, Score: 0.00879057>
<EntryId: 7, Score: 0.00876193>

searching for image 4 returns 4 results:
<EntryId: 4, Score: 1>
<EntryId: 6, Score: 0.00795609>
<EntryId: 5, Score: 0.00712535>
<EntryId: 0, Score: 0.00681504>

searching for image 5 returns 4 results:
<EntryId: 5, Score: 1>
<EntryId: 2, Score: 0.0104254>
<EntryId: 3, Score: 0.00881794>
<EntryId: 4, Score: 0.00712535>

searching for image 6 returns 4 results:
<EntryId: 6, Score: 1>
<EntryId: 3, Score: 0.00879057>
<EntryId: 4, Score: 0.00795609>
<EntryId: 2, Score: 0.00791604>

searching for image 7 returns 4 results:
<EntryId: 7, Score: 1>
<EntryId: 3, Score: 0.00876193>
<EntryId: 8, Score: 0.00664312>
<EntryId: 4, Score: 0.0055488>

searching for image 8 returns 4 results:
<EntryId: 8, Score: 1>
<EntryId: 0, Score: 0.00849321>
<EntryId: 3, Score: 0.00799292>
<EntryId: 9, Score: 0.00685142>

searching for image 9 returns 4 results:
<EntryId: 9, Score: 1>
<EntryId: 0, Score: 0.0306572>
<EntryId: 2, Score: 0.00686568>
<EntryId: 8, Score: 0.00685142>

done.