多目标跟踪MOT踩坑记录
文章目录
- tracking MOT log
- 0. 数据集
- 1.Github
- 2.资源综述
- 3.工具包Code
- MOT 工具包
- MOT 评估 Python版
- 4.算法Code
- deep sort `python`
- 计算多目标跟踪性能
- 1 实验：Deep SORT
- 2 实验：Resnet50 MOT16
- 3 实验：Resnet50 MOT17
- 4 实验：计算MOT17的gt的指标
- 5 实验：Deep SORT(MOT16用gt.txt代替det.txt)
- 代码笔记
- iou-tracker `python`
- rnntracking
- 5.实验结论
- 6.附录
- Deep SORT MOT17 SDP
- Deep SORT MOT17 SDP mindet0.8
tracking MOT log
综述
一篇论文综述：翻译
https://blog.csdn.net/yuhq3/article/details/78742658
arxiv搜索：
http://www.arxiv-sanity.com/search?q=recurrent+tracking
quora:https://www.quora.com/Why-is-no-visual-tracking-algorithm-using-RNN-LSTM
一个斯坦福大学教授主页-专做行人分析-多目标跟踪等
使用多目标跟踪的数据集测试单目标跟踪算法（DaSiamRPN、ECO等），发现跟踪效果较差。原因是多目标跟踪数据集中多个目标相似度很高、并且存在大量的遮挡和目标消失现象。
DaSiamRPN-ECCV-2018VOT单目标跟踪冠军
ECO-CVPR-2017
0. 数据集
- MOT：https://motchallenge.net/
多目标跟踪数据集
检测文件标注格式:
,
1, -1, 794.2, 47.5, 71.2, 174.8, 67.5, -1, -1
1, -1, 164.1, 19.6, 66.5, 163.2, 29.4, -1, -1
标注格式:第一个数字表示帧号,第二个-1表示还没有分配ID,后面四个数字表示左上角x,y,w,h,接着的一个数字表示检测器得到的置信度,最后两个-1对检测文件来说是忽略的.
GroundTrue标注格式
,
1, 1, 794.2, 47.5, 71.2, 174.8, 1, 1, 0.8
第7个数字表示这个实体是否被评估,0=忽略,1=评估;第8个数字表示类别
Label ID
Pedestrian 1
Person on vehicle 2
Car 3
Bicycle 4
Motorbike 5
Non motorized vehicle 6
Static person 7
Distractor 8
Occluder 9
Occluder on the ground 10 Occluder full 11
Reflection 12
- MOT数据集解析、绘制、预处理
按照det.txt将MOT序列图像中的检测框绘制出来：目录=/media/han/E/mWork/datasets/MOT/DataSetParser
按照gt.txt将MOT序列图像的跟踪结果绘制出来：例如目录=/media/han/E/mWork/datasets/MOT/DataSetParser/MOT17/train/MOT17-02-GT
按照gt.txt去掉忽略的框，绘制跟踪结果：例如目录=/media/han/E/mWork/datasets/MOT/DataSetParser/MOT17/train/MOT17-02-GT_ignore0
- DukeMTMC : http://vision.cs.duke.edu/DukeMTMC/
杜克大学多相机多目标跟踪项目组
1.Github
- DeepCC
https://github.com/ergysr/DeepCC 杜克大学项目组，论文里提到了很多技巧，很杂，我虽然代码开源了，但是数据集太大100G以上，有点难训练。
- MOT竞赛 devkit
https://bitbucket.org/amilan/motchallenge-devkit/src
- MDP_Tracking 2015年ICCV
https://github.com/yuxng/MDP_Tracking
MDP_Tracking is a online multi-object tracking framework based on Markov Decision Processes (MDPs).
多目标跟踪 MDP Tracking 代码配置与运行
- ICCV 2015论文代码开源
http://rehg.org/mht/
2.资源综述
-
多目标跟踪论文清单 http://perception.yale.edu/Brian/refGuides/MOT.html
-
相关知识点
评价指标：
多目标跟踪竞赛结果摘要：Multiple Object Tracking Challenge 2017 Results
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
% metrics contains the following
% [1] recall - percentage of detected targets
% [2] precision - percentage of correctly detected targets
% [3] FAR - number of false alarms per frame
% [4] GT - number of ground truth trajectories
% [5-7] MT, PT, ML - number of mostly tracked, partially tracked and mostly lost trajectories
% [8] falsepositives- number of false positives (FP)
% [9] missed - number of missed targets (FN)
% [10] idswitches - number of id switches (IDs)
% [11] FRA - number of fragmentations
% [12] MOTA - Multi-object tracking accuracy in [0,100]
% [13] MOTP - Multi-object tracking precision in [0,100] (3D) / [td,100] (2D)
% [14] MOTAL - Multi-object tracking accuracy in [0,100] with log10(idswitches)
ICCV 2015 Multiple Hypothesis Tracking Revisited 阅读笔记
ECCV2018论文:Multi-object Tracking with Neural Gating Using Bilinear LSTM
作者Fuxin Li主页 没有开源代码，比清楚具体训练细节
3.工具包Code
MOT 工具包
https://bitbucket.org/amilan/motchallenge-devkit/
例子:
% 输入序列list,GT路径,自己的算法跟踪结果路径,评估集名称
benchmarkGtDir = '/media/han/E/mWork/mCode/tracking-mot/MOT16/train/';
[allMets, metsBenchmark] = evaluateTracking('c5-train.txt', '/media/han/E/mWork/mCode/tracking-mot/deep_sort/results/', benchmarkGtDir, 'MOT16');
没有可视化的程序,可视化的程序可以在/media/han/E/mWork/mCode/tracking-mot/MDP_Tracking/show_groundtruth.m,我把这个可视化脚本和其依赖函数提取出来,放在了
/media/han/E/mWork/mCode/tracking-mot/show_MOT_groundTrue.m
Deep SORT算法代码中还有一个可视化工具,Python写的脚本:/media/han/E/mWork/mCode/tracking-mot/deep_sort/show_results.py
- MOT评估指标
注意对于MOT16、MOT17，devkit评估时会进行一些预处理
MOT 评估 Python版
https://github.com/cheind/py-motmetrics
运行速度非常慢,可能是我没有开加速优化,但是相对来说比MATLAB工具包难用.
4.算法Code
deep sort python
Simple Online and Realtime Tracking with a Deep Association Metric
[2017 IEEE International Conference on Image Processing (ICIP)]
https://github.com/nwojke/deep_sort
主要是用CNN提取特征,转换成了128维度的特征向量,在跟踪过程中使用了卡尔曼滤波和两种损失:IOU损失和特征向量余弦损失.
代码中没有使用神经网络检测特征,而是预先提取了特征保存在文件中,代码运行过程中加载文件中的128维度特征向量,然后进行跟踪匹配.
https://github.com/nwojke/cosine_metric_learning 是作者关联的另一个相关的repo,包含了如何训练特征向量.
计算多目标跟踪性能
1 实验：Deep SORT
论文中：MOTA 61.4
自己计算：MOTA 60.3
%/media/han/E/mWork/mCode/tracking-mot/amilan-motchallenge-devkit/demo_evalMOT16.m
benchmarkGtDir = '/media/han/E/mWork/datasets/MOT/MOT16/train/';
[allMets, metsBenchmark] = evaluateTracking('c5-train.txt', '/media/han/E/mWork/mCode/tracking-mot/deep_sort/results/', benchmarkGtDir, 'MOT16');
以下是Deep SORT代码中的模型
%Deep Sort
% 这是和论文作者使用的检测器一致，都是一篇论文POI提供的检测器
%********************* Your MOT16 Results *********************
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
64.7 76.9 55.8| 66.7 91.9 1.22| 517 176 254 87| 6492 36747 586 1168| 60.3 81.7 60.8
这是使用Deep SORT从MOT16原始检测文件中提取特征进行多目标跟踪的效果
%% Deep Sort
% by MOT16 original detection
%********************* Your MOT16 Results *********************
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
37.6 46.0 31.8| 43.9 63.4 5.26| 517 65 232 220| 27964 61905 729 1680| 17.9 76.9 18.6
以下是使用Deep SORT从MOT17原始检测文件中提取特征进行多目标跟踪的效果
%% Deep Sort
% by MOT17 SDP original detection
%********************* Your MOT17 Results *********************
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
64.4 78.1 54.7| 67.7 96.7 0.49| 546 186 248 112| 2619 36248 619 1405| 64.8 84.6 65.4
%% Deep Sort
% by MOT17 DPM+FRCNN+SDP original detection
%********************* Your MOT17 Results *********************
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
52.3 66.6 43.0| 54.6 84.6 2.11| 1638 357 764 517| 33615152859 1827 4049| 44.1 83.2 44.6
2 实验：Resnet50 MOT16
尝试：看了几篇行人重识别的论文，发现一个较好的baseline是直接用Resnet50+softmax分类来训练特征提取，我现在正在尝试将其他方法提取的特征方法替换掉deepsort中的特征提取过程，看看多目标跟踪指标有没有较大提升。
%resnet50 Re-ID baseline
%********************* Your MOT16 Results *********************
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
31.9 37.8 27.6| 44.8 61.4 5.86| 517 68 243 206| 31142 60902 1172 1946| 15.6 76.6 16.6
上面的表中，MOTA非常低，跟Deep SORT的结果相差很大。但是我突然意识到，Deep SORT的结果并不是基于MOT challenge的官方检测的，是基于POI论文提出的检测框，因此虚假框比较少。
3 实验：Resnet50 MOT17
紧接着，试验在MOT17上的MOTA结果：
%% Re-ID baseline resnet50
% MOT17 DPM+FRCNN+SDP平均，MOTchallenge排行榜也是类似的综合成绩
%********************* Your MOT17 Results *********************
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
46.9 58.6 39.1| 55.3 82.9 2.40| 1638 383 768 487| 38351150518 2635 4587| 43.2 83.0 43.9
使用更好的检测器，可以得到更好的跟踪效果，这是显而易见的。MOT17包括DPM、FRCNN、SDP三种检测器得到的检测bboxs，这三种检测器得到的跟踪指标MOTA有较大差别。
3.1 DPM 效果
%% Re-ID baseline resnet50
% MOT17 DPM
********************* Your MOT17 Results *********************
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
32.5 38.9 27.9| 44.2 61.7 5.80| 546 64 251 231| 30841 62676 1171 1964| 15.7 76.6 16.7
3.2 FRCNN 效果
%% Re-ID baseline resnet50
% MOT17 FRCNN
%********************* Your MOT17 Results *********************
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
49.5 68.3 38.8| 53.2 93.6 0.77| 546 120 273 153| 4103 52522 638 1030| 49.0 86.5 49.6
3.3 SDP 效果
%% Re-ID baseline resnet50
% MOT17 SDP
********************* Your MOT17 Results *********************
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
58.8 70.5 50.5| 68.5 95.8 0.64| 546 199 244 103| 3407 35320 826 1593| 64.8 84.4 65.5
从3.1、3.2、3.3可以很容易看出来，即使是相同的特征提取技术，如果检测器的精度高，那么多目标跟踪效果就越好。
4 实验：计算MOT17的gt的指标
Sequences:
'MOT17-02-SDP'
'MOT17-04-SDP'
'MOT17-05-SDP'
'MOT17-09-SDP'
'MOT17-10-SDP'
'MOT17-11-SDP'
'MOT17-13-SDP'
... MOT17-02-SDP
Preprocessing (cleaning) MOT17-02-SDP...
......
Removing 8020 boxes from solution...
MOT17-02-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
91.6 84.5 100.0|100.0 84.5 5.67| 62 62 0 0| 3402 0 0 0| 81.7 100.0 81.7
... MOT17-04-SDP
Preprocessing (cleaning) MOT17-04-SDP...
..........
Removing 4798 boxes from solution...
MOT17-04-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
63.1 46.1 100.0|100.0 46.1 53.00| 83 83 0 0| 55650 0 0 0| -17.0 100.0 -17.0
... MOT17-05-SDP
Preprocessing (cleaning) MOT17-05-SDP...
........
Removing 339 boxes from solution...
MOT17-05-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
94.8 90.1 100.0|100.0 90.1 0.90| 133 133 0 0| 757 0 0 0| 89.1 100.0 89.1
... MOT17-09-SDP
Preprocessing (cleaning) MOT17-09-SDP...
.....
Removing 4036 boxes from solution...
MOT17-09-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
91.0 83.5 100.0|100.0 83.5 2.00| 26 26 0 0| 1050 0 0 0| 80.3 100.0 80.3
... MOT17-10-SDP
Preprocessing (cleaning) MOT17-10-SDP...
......
Removing 1846 boxes from solution...
MOT17-10-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
90.3 82.3 100.0|100.0 82.3 4.23| 57 57 0 0| 2765 0 0 0| 78.5 100.0 78.5
... MOT17-11-SDP
Preprocessing (cleaning) MOT17-11-SDP...
.........
Removing 585 boxes from solution...
MOT17-11-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
96.9 94.1 100.0|100.0 94.1 0.66| 75 75 0 0| 596 0 0 0| 93.7 100.0 93.7
... MOT17-13-SDP
Preprocessing (cleaning) MOT17-13-SDP...
.......
Removing 126 boxes from solution...
MOT17-13-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
73.4 58.0 100.0|100.0 58.0 11.25| 110 110 0 0| 8434 0 0 0| 27.6 100.0 27.6
********************* Your MOT17 Results *********************
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
75.6 60.7 100.0|100.0 60.7 13.67| 546 546 0 0| 72654 0 0 0| 35.3 100.0 35.3
上面的结果发现，虽然MOTP都是100，ID和FN、FM都是0，但是MOTA仍然不是特别高。为什么会出现这样的情况？实际上，MOT challenge devkit(MATLAB) 在读取gt/gt.txt文件中的数据后，将所有静止的目标全部删除了，然后在和跟踪到的结果数据进行对比计算，也就是说，对检测数据并没有删除其中的静止bbox。如果我们使用完全删除静止框的gt.txt来作为跟踪结果，那么会得到：
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
100.0 100.0 100.0|100.0 100.0 0.00| 62 62 0 0| 0 0 0 0| 100.0 100.0 100.0
启发：如果多目标跟踪算法对静止目标有强的抑制作用（检测出更少的静止目标轨迹），那么MOTA指标应该会增大。
5 实验：Deep SORT(MOT16用gt.txt代替det.txt)
MOT16用gt.txt代替det.txt，为什么做这样的实验，因为gt.txt中的框有很多在det.txt中并没有，由于Deep SORT并没有检测环节，所以这些没有检测到的框将限制MOTA指标值。
Sequences:
'MOT16-02'
'MOT16-04'
'MOT16-05'
'MOT16-09'
'MOT16-10'
'MOT16-11'
'MOT16-13'
... MOT16-02
Preprocessing (cleaning) MOT16-02...
......
Removing 0 boxes from solution...
MOT16-02
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
97.0 97.2 96.8| 99.4 99.7 0.07| 54 53 1 0| 45 115 14 9| 99.0 96.3 99.1
... MOT16-04
Preprocessing (cleaning) MOT16-04...
..........
Removing 3 boxes from solution...
MOT16-04
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
99.8 99.9 99.7| 99.7 99.9 0.03| 83 83 0 0| 28 166 0 0| 99.6 97.4 99.6
... MOT16-05
Preprocessing (cleaning) MOT16-05...
........
Removing 0 boxes from solution...
MOT16-05
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
87.3 87.9 86.7| 96.5 97.9 0.17| 125 109 16 0| 142 242 43 15| 93.7 91.3 94.3
... MOT16-09
Preprocessing (cleaning) MOT16-09...
.....
Removing 0 boxes from solution...
MOT16-09
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
85.1 85.3 84.8| 98.8 99.5 0.05| 25 25 0 0| 26 62 11 4| 98.1 94.4 98.3
... MOT16-10
Preprocessing (cleaning) MOT16-10...
......
Removing 0 boxes from solution...
MOT16-10
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
91.4 91.7 91.1| 98.9 99.6 0.08| 54 52 2 0| 54 139 15 9| 98.3 92.8 98.4
... MOT16-11
Preprocessing (cleaning) MOT16-11...
.........
Removing 0 boxes from solution...
MOT16-11
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
98.7 99.2 98.3| 98.5 99.4 0.06| 69 66 3 0| 52 138 1 1| 97.9 94.8 97.9
... MOT16-13
Preprocessing (cleaning) MOT16-13...
.......
Removing 0 boxes from solution...
MOT16-13
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
87.9 88.6 87.2| 97.4 99.0 0.15| 107 101 4 2| 110 299 9 8| 96.3 91.5 96.4
********************* Your MOT16 Results *********************
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
95.6 95.9 95.3| 98.9 99.6 0.09| 517 489 26 2| 457 1161 93 46| 98.5 95.4 98.5
以上的实验结果很惊人，MOTA得到98.5，这也就意味着，如果输入的检测框和GT中的检测框一致，将会极大的提高MOTA分数。那么是否意味着，现有的方法在轨迹关联方面已经够用了，想要在官方给的det.txt基础上继续提高MOTA，只能想办法减少det.txt中和gt.txt中框不一致的现象。
思考：为什么MOTA不等于100，原因可能是：Deep SORT关联候选框出现错误；Deep SORT需要连续框出现3帧。
代码笔记
deep sort代码中有nms预处理过程，但是其nms不是计算IOU，而是overlay，并且作者设置nms阈值为1.0，当大于1.0时抑制。
实际上，这个nms无法发挥作用，overlay>0.1是几乎不会出现的。
iou-tracker python
https://github.com/bochinski/iou-tracker
MOT17 SDP数据结果
Sequences:
'MOT17-02-SDP'
'MOT17-04-SDP'
'MOT17-05-SDP'
'MOT17-09-SDP'
'MOT17-10-SDP'
'MOT17-11-SDP'
'MOT17-13-SDP'
... MOT17-02-SDP
Preprocessing (cleaning) MOT17-02-SDP...
......
Removing 1774 boxes from solution...
MOT17-02-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
36.8 55.3 27.6| 46.4 93.2 1.05| 62 7 38 17| 628 9954 439 499| 40.7 80.5 43.0
... MOT17-04-SDP
Preprocessing (cleaning) MOT17-04-SDP...
..........
Removing 1007 boxes from solution...
MOT17-04-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
60.9 70.8 53.4| 75.1 99.7 0.10| 83 41 29 13| 102 11824 487 542| 73.9 84.9 74.9
... MOT17-05-SDP
Preprocessing (cleaning) MOT17-05-SDP...
........
Removing 40 boxes from solution...
MOT17-05-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
54.5 68.6 45.3| 63.5 96.4 0.20| 133 35 79 19| 166 2522 193 182| 58.3 84.1 61.1
... MOT17-09-SDP
Preprocessing (cleaning) MOT17-09-SDP...
.....
Removing 80 boxes from solution...
MOT17-09-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
45.5 57.9 37.5| 64.2 99.1 0.06| 26 11 14 1| 30 1907 106 98| 61.6 85.4 63.6
... MOT17-10-SDP
Preprocessing (cleaning) MOT17-10-SDP...
......
Removing 110 boxes from solution...
MOT17-10-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
47.5 54.4 42.1| 72.9 94.2 0.87| 57 29 25 3| 571 3484 363 348| 65.6 79.7 68.4
... MOT17-11-SDP
Preprocessing (cleaning) MOT17-11-SDP...
.........
Removing 148 boxes from solution...
MOT17-11-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
56.5 65.4 49.8| 73.3 96.1 0.31| 75 29 32 14| 280 2523 163 149| 68.6 85.1 70.3
... MOT17-13-SDP
Preprocessing (cleaning) MOT17-13-SDP...
.......
Removing 0 boxes from solution...
MOT17-13-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
52.2 67.8 42.5| 56.4 90.0 0.98| 110 44 24 42| 732 5071 307 247| 47.5 78.9 50.1
********************* Your MOT17 Results *********************
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
53.4 65.4 45.1| 66.8 96.8 0.47| 546 196 241 109| 2509 37285 2058 2065| 62.7 83.2 64.6
rnntracking
/media/han/E/mWork/mCode/tracking-mot/amilan-rnntracking
http://torch.ch/docs/getting-started.html
安装过程中出现torch>=7.0依赖问题，参考https://github.com/deepmind/torch-hdf5/issues/95#issuecomment-442821334
5.实验结论
MOT评估方法中有一些细节：
（1）MOT16/17会对gt.txt中的数据中的忽略框进行删除（方法是：直接判断gt.txt中每行的第7位标志位）；
（2）也会对跟踪结果数据中的数据进行清洗，并自动生成在clear文件夹下；但是！仍然有很多框没有清洗完，导致直接使用gt.txt数据测试，MOTA并不是100，参见实验4.
图像对比
MOT17-02-GT-ignore0
MOT17-02-SDP-det
6.附录
Deep SORT MOT17 SDP
Sequences:
'MOT17-02-SDP'
'MOT17-04-SDP'
'MOT17-05-SDP'
'MOT17-09-SDP'
'MOT17-10-SDP'
'MOT17-11-SDP'
'MOT17-13-SDP'
... MOT17-02-SDP
Preprocessing (cleaning) MOT17-02-SDP...
......
Removing 1901 boxes from solution...
MOT17-02-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
43.3 63.3 32.9| 48.4 93.1 1.11| 62 9 37 16| 669 9591 162 366| 43.9 81.4 44.8
... MOT17-04-SDP
Preprocessing (cleaning) MOT17-04-SDP...
..........
Removing 1015 boxes from solution...
MOT17-04-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
73.2 84.4 64.6| 76.2 99.6 0.13| 83 42 28 13| 141 11318 109 343| 75.7 86.7 75.9
... MOT17-05-SDP
Preprocessing (cleaning) MOT17-05-SDP...
........
Removing 38 boxes from solution...
MOT17-05-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
64.9 81.7 53.8| 63.1 95.7 0.23| 133 21 84 28| 194 2555 76 134| 59.2 84.0 60.2
... MOT17-09-SDP
Preprocessing (cleaning) MOT17-09-SDP...
.....
Removing 85 boxes from solution...
MOT17-09-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
56.7 71.4 47.1| 65.5 99.3 0.05| 26 9 16 1| 25 1838 32 57| 64.4 86.4 65.0
... MOT17-10-SDP
Preprocessing (cleaning) MOT17-10-SDP...
......
Removing 149 boxes from solution...
MOT17-10-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
63.1 71.9 56.2| 73.2 93.6 0.98| 57 29 25 3| 644 3443 102 255| 67.4 80.7 68.2
... MOT17-11-SDP
Preprocessing (cleaning) MOT17-11-SDP...
.........
Removing 148 boxes from solution...
MOT17-11-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
64.4 74.3 56.9| 73.8 96.4 0.29| 75 30 33 12| 263 2471 63 101| 70.4 86.3 71.0
... MOT17-13-SDP
Preprocessing (cleaning) MOT17-13-SDP...
.......
Removing 0 boxes from solution...
MOT17-13-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
61.7 80.1 50.2| 56.8 90.6 0.91| 110 46 25 39| 683 5032 75 149| 50.3 79.9 50.9
********************* Your MOT17 Results *********************
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
64.4 78.1 54.7| 67.7 96.7 0.49| 546 186 248 112| 2619 36248 619 1405| 64.8 84.6 65.4
Deep SORT MOT17 SDP mindet0.8
Sequences:
'MOT17-02-SDP'
'MOT17-04-SDP'
'MOT17-05-SDP'
'MOT17-09-SDP'
'MOT17-10-SDP'
'MOT17-11-SDP'
'MOT17-13-SDP'
... MOT17-02-SDP
Preprocessing (cleaning) MOT17-02-SDP...
......
Removing 1650 boxes from solution...
MOT17-02-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
43.8 67.5 32.5| 46.2 96.2 0.57| 62 9 35 18| 342 9989 127 298| 43.7 82.0 44.4
... MOT17-04-SDP
Preprocessing (cleaning) MOT17-04-SDP...
..........
Removing 1004 boxes from solution...
MOT17-04-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
73.5 86.7 63.8| 73.5 99.8 0.05| 83 37 31 15| 55 12625 129 348| 73.1 86.8 73.3
... MOT17-05-SDP
Preprocessing (cleaning) MOT17-05-SDP...
........
Removing 20 boxes from solution...
MOT17-05-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
63.6 83.4 51.4| 60.5 98.1 0.10| 133 18 78 37| 83 2733 56 112| 58.5 84.6 59.3
... MOT17-09-SDP
Preprocessing (cleaning) MOT17-09-SDP...
.....
Removing 16 boxes from solution...
MOT17-09-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
57.1 72.8 46.9| 64.2 99.7 0.02| 26 8 16 2| 9 1904 29 55| 63.5 86.7 64.0
... MOT17-10-SDP
Preprocessing (cleaning) MOT17-10-SDP...
......
Removing 43 boxes from solution...
MOT17-10-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
63.6 74.8 55.4| 71.1 96.0 0.58| 57 25 29 3| 379 3715 92 244| 67.4 81.1 68.1
... MOT17-11-SDP
Preprocessing (cleaning) MOT17-11-SDP...
.........
Removing 125 boxes from solution...
MOT17-11-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
64.7 76.6 56.0| 71.3 97.5 0.19| 75 24 38 13| 174 2711 46 100| 68.9 86.9 69.4
... MOT17-13-SDP
Preprocessing (cleaning) MOT17-13-SDP...
.......
Removing 0 boxes from solution...
MOT17-13-SDP
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
62.4 87.0 48.6| 52.5 94.0 0.52| 110 39 21 50| 390 5528 31 89| 48.9 81.1 49.2
********************* Your MOT17 Results *********************
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
64.7 81.1 53.8| 65.1 98.1 0.27| 546 160 248 138| 1432 39205 510 1246| 63.4 84.9 63.8
Deep SORT MOT17 DPM+FRCNN+SDP mindet0.8
********************* Your MOT17 Results *********************
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
51.9 81.4 38.1| 46.0 98.3 0.17| 1638 266 647 725| 2712181970 983 2990| 44.9 85.0 45.2
Deep SORT MOT17 DPM+FRCNN+SDP mindet0.3
********************* Your MOT17 Results *********************
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
53.4 78.0 40.6| 50.1 96.1 0.43| 1638 319 696 623| 6886168277 1249 3432| 47.6 84.2 48.0