deepSORT对低帧率视频的追踪，纯代码
deepSORT依据的是两个匹配机制，马氏距离下的特征层匹配以及IOU匹配，而当IOU匹配失去作用时，也就是多目标追踪任务中目标帧与帧之间差距较大而使得IOU匹配无法实现时，就需要考虑将其替换成其他的匹配机制，比如欧氏距离匹配
IOU匹配替换代码
iou_track_candidates = unconfirmed_tracks + [
k for k in unmatched_tracks_a if
self.tracks[k].time_since_update == 1]
unmatched_tracks_a = [
k for k in unmatched_tracks_a if
self.tracks[k].time_since_update != 1]
# matches_b, unmatched_tracks_b, unmatched_detections = \
# linear_assignment.min_cost_matching(
# iou_matching.iou_cost, self.max_iou_distance, self.tracks,
# detections, iou_track_candidates, unmatched_detections)
# matches = matches_a + matches_b
# unmatched_tracks = list(set(unmatched_tracks_a + unmatched_tracks_b))
matches=matches_a
unmatched_tracks=list(set(unmatched_tracks_a+iou_track_candidates))
#欧氏距离匹配
Euclidean_distance_candidates=unmatched_tracks
matches_c, unmatched_tracks_c, unmatched_detections = \
linear_assignment.min_cost_matching(
iou_matching.Euclidean_distance_cost, self.max_iou_distance, self.tracks,
detections, Euclidean_distance_candidates, unmatched_detections)
matches+=matches_c
unmatched_tracks=list(set(unmatched_tracks_c))Euclidean_cost_matching是欧氏距离的方法
修改的位置在deep_sort_pytorch/deep_sort/sort/tracker.py
track属性相关修改
在原本的 deepSORT 中是没有 detection 这个属性的，我们需要在 track 中增加这一属性，来对后续的欧氏距离匹配铺路，在这里，唯一的改动只是增加了一个 detection 的属性
def __init__(self, mean, covariance, track_id, n_init, max_age,
feature=None):
self.mean = mean
self.covariance = covariance
self.track_id = track_id
self.hits = 1
self.age = 1
self.time_since_update = 0
self.state = TrackState.Tentative
self.features = []
if feature is not None:
self.features.append(feature)
self.detections = []
self._n_init = n_init
self._max_age = max_age之后需要在 track 的更新函数中将每一帧检测匹配上的 detection 加上去。
def update(self, kf, detection):
"""Perform Kalman filter measurement update step and update the feature
cache.
Parameters
----------
kf : kalman_filter.KalmanFilter
The Kalman filter.
detection : Detection
The associated detection.
"""
self.mean, self.covariance = kf.update(
self.mean, self.covariance, detection.to_xyah())
self.features.append(detection.feature)
self.detections.append(detection)
self.hits += 1
self.time_since_update = 0
if self.state == TrackState.Tentative and self.hits >= self._n_init:
self.state = TrackState.Confirmed欧氏距离匹配代码增加
由于是将IOU匹配替换成了欧氏距离匹配，所以在 deep_sort_pytorch/deep_sort/sort/iou_matching.py中增加方法 Euclidean_distance_cost
def Euclidean_distance(bbox,candidates):
bbox_cx=bbox[0]+bbox[2]/2
bbox_cy=bbox[1]+bbox[3]/2
candidates_cx=candidates[:,0]+candidates[:,2]/2
candidates_cy=candidates[:,1]+candidates[:,3]/2
distance=np.sqrt((candidates_cx[:]-bbox_cx)**2+(candidates_cy-bbox_cy)**2)
return distance
def Euclidean_distance_cost(tracks, detections, track_indices=None,
detection_indices=None):
if track_indices is None:
track_indices = np.arange(len(tracks))
if detection_indices is None:
detection_indices = np.arange(len(detections))
cost_matrix=np.zeros((len(track_indices),len(detection_indices)))
for row,track_idx in enumerate(track_indices):
bbox=tracks[track_idx].to_tlwh()
candidates=np.asarray(
[detections[i].tlwh for i in detection_indices])
cost_matrix[row,:]=Euclidean_distance(bbox,candidates)
return cost_matrix此函数返回的是距离矩阵
最后在距离 linear_assignment.py 下将iou匹配的阈值替换成合适的欧氏距离阈值
if (len(cost_matrix_1)==2 and cost_matrix_1[1]!='cas') or len(cost_matrix_1)!=2 :
#默认的是max_distance = 0.7
# cost_matrix = cost_matrix
# cost_matrix[cost_matrix > max_distance] = max_distance + 1e-5
#使用欧氏距离替换iou
cost_matrix = cost_matrix
max_distance=50
cost_matrix[cost_matrix > max_distance] = max_distance + 1e+5如果是简单的修改能够使算法满足项目要求的话下半部分就不用去看了
欧氏距离下的路径规划
在实际的项目中，IOU发挥不了作用的情况下，欧氏距离匹配能够勉强发挥作用，但仍不能达到高帧数下的追踪效果
这种情况下就应该加入运动规划了，规范化目标的运行路径，使得不合理的匹配结果尽可能的少。
在这里就需要利用之前增加的 detection 属性了，修改如下
def Euclidean_distance(bbox,candidates):
bbox_cx=bbox[0]+bbox[2]/2
bbox_cy=bbox[1]+bbox[3]/2
candidates_cx=candidates[:,0]+candidates[:,2]/2
candidates_cy=candidates[:,1]+candidates[:,3]/2
distance=np.sqrt((candidates_cx[:]-bbox_cx)**2+(candidates_cy-bbox_cy)**2)
return distance
#距离长宽比矩阵
def distance_xy_a(bbox,candidates):
#上一帧的位置
bbox_cx = bbox[0]
bbox_cy = bbox[1]
candidates_cx=candidates[:,0]+candidates[:,2]/2
candidates_cy=candidates[:,1]+candidates[:,3]/2
juzhen=[]
for i in range(len(candidates_cy)):
# if abs(candidates_cx[i]-bbox_cx)>=10 and abs(candidates_cy[i]-bbox_cy)>=10 :
# juzhen.append(True)
# continue
if abs(candidates_cy[i]-bbox_cy)>=50:
juzhen.append(True)
continue
if candidates_cy[i]-bbox_cy<-20:
juzhen.append(True)
continue
if candidates_cx[i]-bbox_cx<-10:
juzhen.append(True)
continue
juzhen.append(False)
return juzhen
def Euclidean_distance_cost(tracks, detections, track_indices=None,
detection_indices=None):
if track_indices is None:
track_indices = np.arange(len(tracks))
if detection_indices is None:
detection_indices = np.arange(len(detections))
cost_matrix=np.zeros((len(track_indices),len(detection_indices)))
for row,track_idx in enumerate(track_indices):
track=tracks[track_idx]
bbox=tracks[track_idx].to_tlwh()
candidates=np.asarray(
[detections[i].tlwh for i in detection_indices])
cost_matrix[row,:]=Euclidean_distance(bbox,candidates)
if len(track.detections)!=0:
ret=track.detections[-1].to_xyah()
cost_matrix[row,distance_xy_a(ret,candidates)]+=1000distance_xy_a是设定的路径规划，使得 ID 切换和 ID 丢失的情况尽可能的下降，可根据实际的项目进行修改
总结
以上就是对 deepSORT 进行的所有修改，能够有效的解决帧数过低或目标移动过快的问题，在实际的项目中效果良好，欢迎在评论指出并提出修改意见。
祝大家所愿皆成！