NMS是one-stage和two-stage目标检测任务中常用的一种后处理方法，用来过滤无效重叠的检测框。
NMS
NMS全称非极大值抑制，出自ICPR2006的论文《Efficient Non-Maximum Suppression》。其基本思想很简单，就是保留局部最大值而去除局部非最大值。
NMS对所有的类别的检测框进行循环过滤。对于某个类别C，首先对这些矩形框按照概率降序排列，选中概率最大的框作为候选框，对于剩下的框，依次与候选框求IOU，如果IOU大于某个阈值（超参），则将这些框丢弃（置0），并标记保留最大概率框。
以此类推，最终所有的框相互之间的IOU都是小于超参阈值的，或者概率被置为0了。剩下的所有概率非0的框就是最终的检测框。
基于这种计算逻辑的NMS有两个缺点。首先，NMS算法需要一个超参即IOU Threshold，这个阈值在不同任务中很难平衡。其次，NMS会将相邻或者重叠的两个物体对应的两个大概率目标框去掉一个，造成漏检。
实现：
def nms(dets, threshold):
x1 = dets[:, 0]
y1 = dets[:, 1]
x2 = dets[:, 2]
y2 = dets[:, 3]
areas = (y2 - y1 + 1) * (x2 - x1 + 1)
scores = dets[:, 4]
keep = []
index = scores.argsort()[::-1]
while index.size > 0:
i = index[0] # every time the first is the biggst, and add it directly
keep.append(i)
x11 = np.maximum(x1[i], x1[index[1:]]) # calculate the points of overlap
y11 = np.maximum(y1[i], y1[index[1:]])
x22 = np.minimum(x2[i], x2[index[1:]])
y22 = np.minimum(y2[i], y2[index[1:]])
w = np.maximum(0, x22 - x11 + 1) # the weights of overlap
h = np.maximum(0, y22 - y11 + 1) # the height of overlap
overlaps = w * h
ious = overlaps / (areas[i] + areas[index[1:]] - overlaps)
idx = np.where(ious <= threshold)[0]
index = index[idx + 1] # because index start from 1
return keep
Soft-NMS
Soft-NMS出自CVPR2017的论文《Improving Object Detection With One Line of Code》，对NMS做了一些改进。
Soft-NMS总体算法流程同NMS相同，主要差别循环过程中对阈值的判断部分。NMS是简单的对IOU大于阈值的检测框进行删除出来，而Soft-NMS则是通过权重来降低检测框原有的置信度。对于有重叠的框，重叠区域越大，置信度衰减越严重。
Soft-NMS计算降低置信度的权重常用两种方法：线性法和高斯法。
线性法：
高斯法：
实现如下，方法1为线性法，方法2为高斯法，其他参数的话Soft-NMS退化为NMS：
def soft_nms(dets, sigma=0.5, threshold1=0.7, threshold2=0.1, method=1):
n = dets.shape[0]
x1 = dets[:, 0]
y1 = dets[:, 1]
x2 = dets[:, 2]
y2 = dets[:, 3]
scores = dets[:, 4]
areas = (y2 - y1 + 1) * (x2 - x1 + 1)
new_scores = scores.copy()
index = [i for i in range(n)]
keep = []
while len(index) > 0:
# get max box position of current based new scores
max_score = 0
max_pos = -1
for i in index:
if new_scores[i] >= max_score:
max_pos = i
max_score = new_scores[i]
if max_pos == -1:
break
keep.append(max_pos)
index.remove(max_pos)
# calculate ious between current max box and others
x11 = np.maximum(x1[max_pos], x1[index])
y11 = np.maximum(y1[max_pos], y1[index])
x22 = np.minimum(x2[max_pos], x2[index])
y22 = np.minimum(y2[max_pos], y2[index])
w = np.maximum(0, x22 - x11 + 1)
h = np.maximum(0, y22 - y11 + 1)
overlaps = w * h
ious = overlaps / (areas[max_pos] + areas[index] - overlaps)
# adjust score of others
new_index = []
for i, ids in enumerate(index):
iou = ious[i]
weight = 1
if method == 1:
# linear
if iou >= threshold1:
weight = 1 - iou
elif method == 2:
# gaussian
weight = np.exp(-(iou * iou) / sigma)
else:
# normal nms
if iou >= threshold1:
weight = 0
new_scores[ids] = new_scores[ids] * weight
if new_scores[ids] > threshold2:
new_index.append(ids)
index = new_index
return keep
实验
模拟的5个候选框：
if __name__ == '__main__':
boxes = np.array([[100, 100, 210, 210, 0.72],
[250, 250, 420, 420, 0.8],
[220, 220, 320, 330, 0.92],
[100, 100, 210, 210, 0.72],
[230, 240, 325, 330, 0.81],
[220, 230, 315, 340, 0.9]]) # (x1,y1,x2,y2,score)
keep = nms(boxes, threshold=0.7)
print(keep)
keep = soft_nms(boxes, threshold1=0.7, threshold2=0.2, method=0)
print(keep)
keep = soft_nms(boxes, threshold1=0.7, threshold2=0.2, method=1)
print(keep)
keep = soft_nms(boxes, threshold1=0.7, threshold2=0.2, method=2)
print(keep)
[2, 1, 3]
[2, 1, 3]
[2, 1, 3, 4]
[2, 1, 3, 4]
实验可以看出，在相同IOU阈值的情况下，Soft-NMS相比NMS保留了一个检测结果，这在同类别物体重叠的情况下能够提升其召回率。但是Soft-NMS又多引入了一个超参，这个参数的设置也会显著影响后处理的结果；而且，由于Soft-NMS在每次迭代都会修改score值，其最大值是在动态变化的需要在每次迭代都寻找一次，因此Soft-NMS相比NMS计算效率有所降低。