Real-time ghost removal for foreground segmentation methods/实时前景分割时的鬼影消除

Fei Yin, Dimitrios Makris, Sergio A Velastin
Digital Imaging Research Centre (DIRC)
Faculty of Computing, Information Systems and Mathematics
Kingston University, Surrey, KT1 2EE, UK
{fei.yin, d.makris, sergio.velastin}@kingston.ac.uk

Abstract

Among all the challenges for foreground detection, we focus on detection of ghosts caused by the starting or stopping of objects. We propose a fast and effective method for ghost detection that compares the similarity between the edges of the detected foreground objects and those of the current frame based on object-level knowledge of moving objects. Finally we demonstrate the performance of ghost detection algorithm using a series of urban traffic video sequences including car parking and departure which normally cause ghost problems. The performance evaluation results show that the proposed method can detect and reduce ghost objects efficiently, brings little computational load, and no significant side-effect to the surveillance system.

摘要
在所有前景检测所遇到的挑战中,我们重点关注与检测对象进入和移出时产生的鬼影.我们试图通过比较前景对象的边缘和基于运动对象的对象层次知识基础上的当前帧的边缘之间的相似性来寻找一种快速有效的叠影检测方法.最后,我们用城市内交通监控序列来测试了这个算法的有效性,这些序列包含了会导致出现叠影问题的车辆停泊和开走事件.测试结果表明这个算法能有效的检测并消除叠影,增加少量计算量并且不会对监控系统带来显著的副作用.

1. Introduction
Visual surveillance in dynamic scenes, especially for tracking humans and vehicles moving through the field of view (FOV) of a CCTV camera has received much attention in the past decades [1]. Because detection of moving objects is a key preliminary, it ultimately limits successful tracking in surveillance applications such as traffic monitoring and analysis, access control in special areas, human and vehicle identification, and detection of anomalous behaviours.

1.介绍
对动态场景的视觉监控,特别是对通过监控摄像头视野(FOV)的运动的行人和车辆的跟踪,在过去的几十年里,已经得到了很多关注[1].对运动的检测是关键性的,它会对监控应用中跟踪的成功与否产生影响,如交通监测和分析,在特定区域的访问控制,人体和车辆识别,异常行为检测等.

The most common approach for detecting moving objects is background subtraction, in which each frame is compared against a background model. The pixels in the current frame that have significant difference from the background are then considered as moving pixels or foreground pixels. These foreground pixels will be grouped to form objects which are then tracked. A large number of background subtraction algorithms have been proposed so far [1], but problems still remain for moving objects identification under certain conditions.

最常见的运动检测算法是背景减法,每一帧都会与一个背景模型相比较.当前帧与背景有显著差异的像素被视为运动的像素或前景像素.这些前景像素会组成前景对象,然后再进行跟踪.至今已经提出了很多背景减法的算法,但对于某些条件下的运动对象识别,问题依然存在.

One of the toughest problems is that background subtraction causes the detection of false objects when an object that belongs to the background starts to move away, often referred to as “ghost”. It is important to address the problem because ghost objects will adversely affect many tasks such as object classification, tracking and event analysis (e.g. abandoned item detection). Therefore, ghost object needs to be separated from other objects and eliminated.

一个严重的问题是,当一个对象属于背景并移除后,背景减法会检测到一个错误的对象,通常称为"鬼影".这是一个严重的问题,因为鬼影影响到对象分类,跟踪和事件分析(比如遗弃物检测)等任务,因此,必须要区分和消除鬼影.

This paper focuses on the problem of ghost identification and elimination. We used a state-of-theart industrial tracker [6] which includes basic background subtraction and object tracking. Then we included our ghost detection algorithm into the basic tracker to identify and eliminate ghosts. Finally, we systematically evaluate and compare performance on some urban traffic video sequences. The ground truth for all videos was manually generated using Viper GT[8].

本文主要关注鬼影的识别和消除.我们使用了最先进的工业跟踪(包括基本的背景减法和对象跟踪)系统[6].然后再加入我们的鬼影识别和消除算法.最后,我们用一些城市内的交通视频序列来系统的评估和比较性能.所有的地面实况视频使用Viper GT制作[8].

This paper is organized as follows: Section 2 defines the problem of ghost detection and surveys some previous methods in the literature. Section 3 describes the methodology of ghost detection. Results are presented and discussed in section 4. Finally section 5 concludes the paper.

本文组织如下:第二节定义鬼影检测问题并介绍一些文献背景,第三节介绍鬼影检测的方法,第四节说明测试结论和一些讨论.第五节是对论文的总结.

2. Background
Generally, a robust background model should be able to automatically recover and update itself from a dynamic sequence and be insensitive to illumination changes, shadows, weather conditions such as rain or snow. One of the most popular existing background modelling algorithms was proposed by Stauffer and Grimson [2] in 1998 and further refined e.g. in [9].

2.背景
一般来说,一个健壮的背景模型应该能够从动态序列中自动恢复和更新自己,并且对光亮变化,阴影,天气条件(雨雪等)不敏感.现在最受欢迎的背景建模算法是Stauffer和Grimson[2]在1998年提出并进一步优化的[9].

Besides background modelling, the accuracy of motion detection also depends on rules of motion segmentation and object classification. Normally, motion segmentation methods are based on the notion that differences in consecutive images arise from moving objects. The motion is measured using either image differencing or flow. There are some drawbacks of using a background model. One problem related to our work is that, to account for changing illumination conditions, objects that become stationary are relatively quickly incorporated into the background and when they move again they leave behind an area of foreground (or “ghost”) that can be mistakenly taken as a new object. So the ghost problem is a direct result of the approach to background modelling. An approach that would feedback high level object detection/tracking information to the background model might be able to reduce these effects, for example by enforcing constraints such as that slow or stopped objects cannot simply disappear (merged into the background).

除了背景建模以外,运动检测的准确性也取决于运动分割的规则和对象分类.一般情况下,运动分割是基于这样一种观念:连续图像的差异是源于运动对象.通过图像差分或光流来检测运动.使用背景模型有一些缺点.我们工作有关的一个问题是,在光照条件改变时,静止下来的对象会逐渐的融入背景,当它重新开始运动时,就会在前景留下一个空洞("鬼影")区域,这个会被误认为是一个新的对象.所以"鬼影"问题是背景模型的一个直接结果.将高级别的对象检测/跟踪信息反馈到背景模型可以减少这些影响,比如增加强制约束变慢或者静止下来的对象不能简单的消失(合并到背景).

Cucchiara et al [3] proposed a general frame work of background subtraction called Sakbot which combines statistical assumptions to detect moving objects, apparent objects (ghosts), and shadows with the object level knowledge of those from previous frames. For ghost detection, they calculate the average optical flow, over all the detected moving pixels. They assumed that moving objects should have significant motion, while ghosts should have a near-to-zero average optical flow since their motion is only apparent. However, optical flow is computational expensive for real time processing and also there is a danger of inaccurate classification of stationary object into ghost.

Cucchiara等人[3]提出了一个叫Sakbot的简单背景减法,结合统计假设从之前的帧内获取对象层次的知识来检测运动对象,已经存在的对象(鬼影)和阴影.通过在所有检测到的移动像素中计算出平均光流来检测鬼影.他们假定移动对象应该有显著的运动,而鬼影因为其运动是唯一表面化的所以其平均光流几乎为0.然而,对于实时处理来说,光流计算太耗时了,而且也存在静止对象被认为是鬼影的危险.

Cheung et al [4], proposed a different method to detect ghost in traffic surveillance. They used a frame difference mask which was computed using the incoming frame and its previous frame. Objects with no corresponding blob in the frame difference mask were identified as ghosts. However, the method can not distinguish between ghosts and abandoned objects.

Cheung等人[4]提出了一种用在交通监控中的不同算法.他们使用的差分掩码是传入的帧及其前一帧计算出来的.差分掩码中不存在对应块的对象被视为鬼影.然而,这种方法不能区分鬼影和遗弃物.

In Guler’s method [5], a specific processing layer is used for detection of objects that become stationary. The detected stationary foreground objects are maintained in the specific layer, and the motion history of the immobile objects are recorded. However, the method needs motion history of the object which is not available if the object belonged to background from the start of the video sequence.

在Guler的方法[5]中,保留一个特殊的处理层来检测静止下来的对象.被检测到的对象都维护在这个特殊的层中,固定物的运动历史记录也被保存了下来.然而,该方法需要被检测对象的运动历史记录,如果对象从视频开始就存在背景中,这些是不可用的.

Lu et al [10] proposed to recover the "real" background by using a revised image inpainting method. Thus they could differentiate between ghost and abandoned objects: The object will be declared as ghost if there is no difference between an incoming frame and the “real” background image, otherwise it is an abandoned item. However, image inpainting is computational expensive for real time processing, and their method has not been evaluated quantitatively.

Lu等人[10]提出使用改进的图像修复方法来重新获取"真正"背景.这样就可以区分鬼影和遗弃物:如果当前帧和"真实"背景之间没有差别,就是鬼影;否则就是遗弃物.然而,对于实时处理来说,图像修复也是很耗时的操作,并且他们的方法没有进行定量的评估.

Desurmont et al [12] proposed a general framework for video surveillance systems and calculated the edge mean square gradient (EMSG) for each blob. They find out ghost blobs based on the assumption that in the background the gradient is typically low. Therefore, if the EMSG for a blob is very small (lower than a threshold), the blob will be classified as ghost, otherwise it will be classified as real object. However, this is not always true, e.g. a cluttered scene may result in a high textured background with high EMSG.

Desurmont等[12]提出了一个用于视频监控的简单框架,通过计算每个块的边缘均方梯度(EMSG),他们发现在背景上出现的假定的鬼影块的梯度通常都较低.因此,如果一个块的EMSG非常低(低于一个阈值),就可以视为鬼影;否则就视为真实的物体.但是,这个方法也并不总是正确的,比如,一个凌乱的场景会导致一个高EMSG的背景.

In this paper, we propose using edge comparison between extracted regions from multiple images to find out whether the detected foreground belongs to the current image or not, and furthermore to identify detected foreground objects as real objects (whether moving or stationary) or ghosts. The method is discussed in detail in section 3 and evaluated in section 4.

在本文中,我们通过比较从多张图像中提取的区域边缘,来确定检测到的前景是否属于当前图像,从而识别出该对象是真是存在的(运动的/静止的)还是鬼影.在第三节中详细讨论了这个方法,在第四节中对其进行了评估.

3. Methodology
We have developed a framework that we use to detect ghosts during tracking and furthermore to eliminate them.

3.方法
我们已经开发了一个用于鬼影检测跟踪和消除的框架.

Ghosts mainly appear in two cases: In the first case, when a moving object becomes stationary, it will be adapted (merged) into the background, and then, when it starts to move again some time later, there will be a ghost left behind. In the second case, an existing object that belongs to the background starts to move (e.g parked vehicle) and will also cause a ghost problem. In the first case, we may have the history of object trajectory, but in the second case, we may not have any previous information about the object. So it is preferable to make the tracker able to recognise ghosts in time without additional history information rather than analyse the history of the object.

有两种情况会出现鬼影:第一种情况,当一个运动物体变得静止不动时,它会逐渐融入到背景中,然后当它再次移动时,在原来的位置就会出现鬼影;第二种情况,一个原来属于背景的物体开始运动(例如停放的车辆),也会导致鬼影的出现.在第一种情况下,我们可能会有物体的历史运动轨迹,但是在第二种情况下,我们没有任何关于此物体的历史信息.所以,最好是让跟踪器能够在没有历史信息的情况下识别出鬼影而不是分析这个物体的历史信息.

We consider that a basic tracker is available with the following assumptions:

我们假定基本的跟踪器已经符合以下条件:

.Tracking is performed at pixel level and no calibration information is needed.

.在没有校准信息的情况下进行像素级跟踪.

.Tracking is performed on gray-level image sequences (although extension of the method to colour image sequences is relatively simple).

.在灰度图像序列下跟踪(尽管扩展到彩色序列也很容易).

.The basic tracker contains a motion detection module that can provide a difference map which is the difference of the values of each pixel between the background image and the incoming frame .

.包含一个运动检测模块,可以提供当前帧与背景图之间的差分图像.

.The basic tracker is able to estimate the bounding box and the velocity of moving objects at each frame.

.能计算出每帧中的运动物体的边框(bbox)和速度.

Our proposal of ghost detection is described follows:

建议鬼影检测应符合以下条件:

.For every k frames (k=12 in our implementation), we check the speed of each track. If the track’s speed is below a threshold Tv (e.g. Tv= 0.1 pixel/frame), we proceed further to the ghost detection.

.每经过k帧(在我们的实现中k=12),检查每个跟踪的速度.如果跟踪的速度低于一个阈值Tv(例如Tv=0.1像素/每帧),就需要进一步对鬼影进行检测.

.We apply Gaussian smoothing and Canny edge detection in regions of the bounding boxes (bbox) on both the difference map and the incoming frame. and we get two edge maps for every track at the same time (see Figure 1).

.在当前帧和差分图像中,对每个跟踪的区域(bbox)做Gaussian平滑和Canny边缘检测,这样我们在同一时间对每个跟踪物体有两幅边缘检测图像(见图1).

图1: KND2-JULY-EAST-S2.avi, 5880帧.注意红色和黄色框标明了不同的跟踪区域.

a)车辆开始沿红色箭头从其停泊处移出

b)背景减法得到的当前帧的运动区域

c)图像a)跟踪区域的Canny边缘检测

d)图像b)跟踪区域的Canny边缘检测

.Then, we compare the difference between the edges from the two maps (incoming frame and the frame difference map) in the area of the bounding boxes (bbox) for each track respectively. (e.g. in figure 1 we compare the right yellow bbox with the left yellow bbox since they are of the same track). We get a score for edge similarity (S) between the edges for each track in each frame, and then set up a threshold for making the decision of whether they are the same or different (usually, the score is from 0% (do not match at all) to 100% (perfectly matched).

.然后,我们比较每个跟踪区域(bbox)的两幅图像(当前帧和差分图像)的边缘(如图1,比较左图和右图的同色边框).我们可以为每帧的每个跟踪物体得到一个边缘相似度(S),然后设置一个阈值决定它们是否相同或不同的(通常,分数从0%(完全不匹配)到100%(完全匹配).

Suppose for track i, frame j, we get a set of edge points from the difference map (see Figure 2):

对于帧j,跟踪物体i,我们从差分图中得到一组边缘点(图2):

D_{i,j =} {(x₁^D, y₁^D), (x₂^D, y₂^D)...(x_n^D, y_n^D)}　　　　　　(1)

图2:差分图像的边缘检测

and another set of edge points from the current image for track i (see Figure 3):

和对于当前帧,得到跟踪的物体i的另一组边缘点(图3):

C_{i,j =} {(x₁^C, y₁^C), (x₂^C, y₂^C)...(x_m^C, y_m^C)}　　　　　　(2)

图3:当前帧的边缘检测

We calculate the score of S_ijbetween two edge sets as follows:

可以用下面的公式来获取相似度S_ij:

S_{i,j =} size( D_i,j ∩ C_i,j )/size( D_i,j ∪ C_i,j )  S_i,j∈ [0 ~ 1]　　　　　　(3)

.Normally, similarity scores are very high when the tracks are from real existing objects (typically much higher than 30%). If S_i,jis larger than a threshold T (set to 30% according to experiments), a ghost condition has been detected for track i.

.通常,当跟踪的对象真实存在时,相似度会非常高(远大于30%).如果设定S_i,j必须大于阈值T(根据实验结果,T设置为30%),我们就为跟踪的目标设定了一个鬼影检测条件.

.We accumulate evidence over time using a counter G_iwhich indicates how many times the track i has been detected as ghost. The initial value for G_iis zero.

.我们使用一个累积计数器G_i来记录被跟踪的对象i随着时间的推移,被检测为鬼影的次数,G_i初始化为0.

If S_i,j > T, G_i <- G_i + 1　　　　　　　　　　　　　　　　　　　　　　(4)

As long as G_i> 0, we label the track as "ghost", the track is prevented from being "valid" or "real moving object" and does not trigger a false alarm. After several times of ghost suspicion (using the criterion of equation 4), if G_iis larger than a predefined number E (set to 5 in our work), then we confirm the track as "ghost" and vanish the track (i.e. feedback the confirmation to the motion detector so that the motion detector enforces the ghost blobs to become part of the background).

如果G_i>0,我们标记这个对象是可能的"鬼影",这样就不会被视为"有效的"或者"真正的移动物体"而触发错误的警报.被标记的对象经过几次判别(使用公式(4)计数)后,如果G_i大于一个预定的值E(我们设置为5),我们就确认这个对象是鬼影并去掉这个跟踪对象(即将这个结果反馈到运动检测器,运动检测器会将鬼影块视为背景的一部分).

.In order to makes sure that we do not detect a "real" object as ghost and vanish it by mistake, we also take the motion of the track into consideration. Let (x_i⁰, y_i⁰) be the coordinates of the centroid of track i where it appears firstly. (x_i,j, y_i,j) be the coordinates of the centroid of track i in current frame j.(W_i, H_i) be the average width and height of track i during its life time. Then,

.为了确保不会将真实的物体视为鬼影并错误的消除它,我们也将跟踪目标的运动轨迹考虑在内.假设(x_i⁰, y_i⁰)是跟踪目标i初始时的质心坐标,(x_i,j, y_i,j)是目标i在当前帧j的质心坐标,(W_i, H_i)是目标i在生存周期内的平均宽度和高度,然后

If V_i = [(x_i,j - x_i⁰)² + (y_i,j - y_i⁰)²]/(W_i² + H_i²) >= 1, and G_i > 0   G_i <- G_i - 1　　　　(5)

If the track has moved a long enough distance from where it appeared, then it is not be identified as "ghost". The counter G_iwill be decremented to zero gradually.

如果目标i在出现后已经运动了足够长的距离,就不应该再被认为是鬼影,对它的计数G_i应逐渐递减到0.

Our ghost detection algorithm is summarized in Figure 4:

图4是我们的鬼影检测算法的总结.

图4:鬼影检测算法的框架

 

4. Results
We demonstrate the practical value of the proposed method by using an experimental industrial tracker from BARCO [11] called the "basic tracker" here and then the same tracker with the addition of our ghost detection algorithm. We run the two trackers on a standard PC with Pentium IV 3.4 GHz processor with 1GB RAM. The tracking and ghost detection were carried out on 4 video sequences with 6 ghost events and 2 abandoned items. We managed to remove all the 6 ghost objects, and avoid all false alarms caused by ghosts. (see Figures 5,6,7)

4.实验结果
我们使用BARCO公司的一个实验性的工业跟踪系统[11]来证明我们算法的实用价值,对比只使用这套系统的跟踪器(我们成为"基本跟踪器")与同样的跟踪器但是添加了我们的鬼影检测算法的跟踪器.我们在同一台PC(Pentium IV 3.4GHZ, 1GB RAM)上运行这两套系统,使用了4个测试序列,包含6个鬼影事件和2个遗弃物事件.我们设法消除了所有的6个鬼影并避免了鬼影所造成的误报(图5,6,7).

Figure 5 presents visual results from the basic motion tracker as well as from the tracker with ghost detection algorithm using a PETS2001 sequence. Red bboxes represent the valid tracks generated by the tracker, while blue bboxes represent invalid tracks (ghost tracks). The coloured patches inside each bbox present the foreground pixels which are detected as moving pixels (the colours of each patch are randomly assigned by the tracker). In Figures 5c and 5d, the white minibus moved away from where it had stopped, leaving a ghost (red bbox with green patch in Figure 5d) behind. In Figures 5e and 5f, by using the proposed ghost detection method, we successfully identified the ghost object (blue bounding bbox with green patch with the text "ghost" on it).

图5显示了使用基本跟踪器和添加鬼影检测器的跟踪器来测试PETS2001视频序列的结果.红色框(bboxes)表示跟踪器生成的有效跟踪,蓝色框表示无效的跟踪(对鬼影的跟踪).框内的色块显示的是检测到的前景的移动像素(色块颜色是跟踪器随机分配的).在图5c和图5d中,白色面包车从开始停止的地方开走以后,留下一个鬼影(图5d中红色框内的绿色色块).在图5e和图5f中,通过使用鬼影检测,我们成功的确定了鬼影(蓝色框内的绿色色块,并标记了"ghost").

图5 PETS2001 PetsD1TeC1.avi 2686帧(时长00:01:29),包含4个(2组)人, 3辆车.主要挑战是多物体的交叉和鬼影.

a)第2360帧　　　　b)第2635帧

c)基本跟踪器对2360帧的检测结果

d)基本跟踪器对2635帧的检测结果

e)带鬼影检测的基本跟踪器对2360帧的检测结果

f)带鬼影检测的基本跟踪器对2635帧的检测结果

Figure 6 presents results from the KND2-JULY-EAST- S2.avi sequence. In Figure 6c frame 5856 a car is about to start. After a while, in frame 5880 (Figure 6d), the car has left its parked area and a ghost track has been formed (red box with yellow patch). Figures 6e and 6f present the tracker with ghost detection: in frame 5880 (Figure 6f), the car left its parked area, however, the ghost patch has been successfully identified (blue bbox with yellow patch) and furthermore prevented from becoming valid and eliminated after a few frames.

图6是KND2-JULY-EAST-S2.avi的测试结果.图6c(5856帧),一辆车开始启动.过了一会,到5880帧(图6d),这辆车已经离开其停放区并留下一个鬼影(红色框内的黄色色块).图6e和6f显示在鬼影检测器的作用下,在5880帧(图6f),汽车离开其停放区,并确认了鬼影区域(蓝色框内的黄色色块),这片区域不会被识别为有效的,并会在数帧后被消除.

图6 KND2-JULY-EAST-S2.avi 21794帧(时长00:12:55),包含23个目标.主要挑战是快速光照变化和2个鬼影.

a)第5856帧　　b)第5880帧

c)基本跟踪器对5856帧的检测结果

d)基本跟踪器对5880帧的检测结果

e)带鬼影检测的基本跟踪器对5856帧的检测结果

f)带鬼影检测的基本跟踪器对5880帧的检测结果

Figures 7c and 7d present the results by the basic motion tracker and Figures 7e and 7f are produced by Ghost detection tracker. As we can see Figure7d, after the vehicle (frame 9556, red bbox with light blue patch) has left, a ghost track was left behind (red bbox with purple patch). With ghost detection in Figures 7e and 7f, the ghost object (blue bbox with blue patch) has been successfully detected.

图7c和7d是基本跟踪器的检测结果,图7e和7f是带鬼影检测的基本跟踪器检测结果.正如我们在图7d中看到,车辆离开后并留下一个鬼影(第9556帧,红色框内的亮蓝色色块).在图7e和图7f中看到,鬼影检测器成功的检测到了鬼影(蓝色框内蓝色色块).

图7 KND2-JULY-GATE1-S2.avi 18180帧(时长00:12:32),包含29个目标.主要挑战是快速光照变化和3个鬼影.

a)第9496帧　　　　b)第9556帧

c)基本检测器对9496帧的检测结果

d)基本检测器对9556帧的检测结果

e)带鬼影检测的基本检测器对9496帧的检测结果

e)带鬼影检测的基本检测器对9556帧的检测结果

Figures 8e and 8f present the result from the Ghost detection tracker. A box (frame 13510, red bbox with green patch) was dropped from the car, and our algorithm can still successfully detect the abandoned item without miss classifying as a ghost.

图8e和图8f显示带鬼影检测的跟踪器的检测结果.一个箱子从车上掉下来(第13510帧,红色框内的绿色色块),我们的算法任然可以准确的检测出遗弃物,而不会将其分类成鬼影.

图8 KND2-JULY-EAST-S3.avi 18102帧(时长00:10:04),包含29个目标.主要挑战是快速光照变化和遗弃的箱子.

a)第13460帧　　　　b)第13510帧

c)基本检测器对13460帧的检测结果

d)基本检测器对13510帧的检测结果

e)带鬼影检测的基本检测器对13460帧的检测结果

f)带鬼影检测的基本检测器对13510帧的检测结果

From the figures shown above, we can see that ghost detection is effective enough to detect ghost objects, to prevent the tracks from becoming "valid", and furthermore to avoid false alarms. At the same time, the algorithm can differentiate between ghost objects and abandoned items.

从上面的图可以看出鬼影检测是有效的,足以发现鬼影,防止其被视为"有效的"目标而触发假警报.同时,该算法能区分鬼影和遗弃物.

We experimented with the frequency of ghost detection by using different values of k (e.g. k = 1, 2, 5, 10, 12, 15, 20…) and testing our algorithm. There is no effect on the performance as long as k is under 20. Our decision to set k to 12 ensures fast and reliable ghost detection.

我们尝试用不同的鬼影检测频率值k(如k = 1、2、5、10、12、15、20…)来测试我们的算法.我们发现只要k小于20,都不会影响性能.我们设定k为12以确保快速和可靠的鬼影检测.

We also present some quantitative evaluation results (Tables 1 - 3) based on the performance evaluation framework described in detail in [6]. The improvement is mainly reflected by the metric: False alarm tracks (FATs). In the PETS2001 sequence (table 1), there are two false alarms caused by ghost objects in the results of the basic tracker. In the results of ghost detection tracker, the 2 false alarms caused by ghosts have been avoided. Similarly, in KND2-JULY-EAST-S2 sequence (table 2), we had 2 cases of false alarms caused by ghosts, but the tracker with ghost detection avoided these 2 FATs. Hence, the number of FAT is reduced from 5 to 3. Similarly, in KND2-JULY-GATE1-S2 sequence (table 3), we had 3 cases of FATs caused by ghosts, but the tracker with ghost detection avoided these 3 FATs. Hence, the number of FAT is reduced from 14 to 11.

我们还提供一些基于效果评估框架(在[6]中详细描述)的定量评估结果.改进主要体现在减少了假警报(FATs).在PETS 2001测试序列中(表1),基本跟踪器有2次假警报,在带鬼影检测的结果中,这两例假警报都被取消了.同样的,在KND2-JULY-EAST-S2序列(表2),也有2次假警报,带鬼影检测的检测结果中,也被取消了,因此,警报的次数从5次减到3次.在KND2-JULY-GATE1-S2序列(表3)中,我们有3次警报是由于鬼影引起的,鬼影检测器取消了这些警报,因此,警报次数从14次减少为11次.

表1 使用Pets2001.avi的效果评估

表1 使用KND2-JULY-EAST-S2.avi的效果评估

表3 使用KND2-JULY-GATE1-S2.avi的效果评估

The speeds of the basic tracker and ghost detection tracker are fast enough to operate in real-time, although there is about 10% to 15% decrease in speed when adding ghost detection into the basic tracker. In addition, except false alarm track, other metrics such as correct detected track, track detection failure, track fragmentation, average latency, distance error remain more or less the same for both basic tracker and ghost detection tracker, which means that the ghost detection does not cause any undesirable side-effects on tracking the objects.

尽管鬼影检测会降低10%~15%的检测速度,但是基本跟踪器和鬼影检测器的速度都足够快,可以满足实时处理的需求.此外,除了处理假警报,在其它的指标上(如正确的检测跟踪,跟踪失败,...平均延时,距离误差等),带和不带鬼影检测的基本跟踪器都差不多,这意味着鬼影检测器没有对目标检测带来任何副作用.

According to the overall evaluation results, the ghost detection algorithm can effectively eliminate ghost objects and brings no significant side-effects caused by the ghost detection algorithm to the surveillance system.

根据综合评价结果,鬼影检测算法能有效地消除监测系统中的鬼影,没有明显的副作用.

5. Conclusion
We have presented an efficient ghost detection and removal algorithm that can be applied to any motion detection and tracking system that produces a difference map per frame and bounding boxes for foreground objects.

5.结论
我们提出了一种高效的鬼影检测和去除算法,可以应用于任何能够为每帧图像生成差分图像和每个前景目标生成跟踪边框的运动检测和跟踪系统.

According to the visual results and performance evaluation results, we can effectively avoid false alarms caused by ghost objects while still differentiating between ghost objects and abandoned objects. The method works well in several scenarios: PETS2001 and another two car park scenarios. The Ghost removal version of basic tracker runs almost as fast as the original and do not have any negative sideeffect on detection and tracking.

根据视觉结果和效果评估的结果,我们可以有效地避免由鬼影引发的假警报,同时仍然能区分鬼影和遗弃物.该方法在PETS2001和另两个停车场场景下工作良好.带鬼影检测的基本跟踪器运行速度几乎与不带鬼影检测的基本跟踪器在检测和跟踪方面一样快,并且没有任何副作用.

However, our method is limited by the performance of the tracker, e.g. if the tracker can not recognise individual objects in crowd scenes, our method will not be able to detect ghosts either.

然而,我们的方法受跟踪器的性能限制,例如,如果跟踪器不能在复杂的场景下识别单个对象,我们的方法将无法检测到鬼影.

In future work, we want to test our algorithm in videos with textured background to check if that would cause confusion between the edges of the pattern and the object.

在未来的工作中,我们要在视频使用纹理背景来测试我们的算法,以检查是否会混淆图案和对象的边缘.

6. Acknowledgements
The authors would like to acknowledge financial support from BARCO View, a major multinational manufacturer of CCTV display systems, whose headquarters are in Belgium. They are also grateful to the Engineering and Physical Sciences Research Council's REASON project for providing access to data.

6.感谢
作者要感谢BARCO View公司(一个大型跨国公司的闭路电视显示系统制造商，其总部设在比利时)的金融支持.他们还感谢工程和物理科学研究委员会提供对其数据的访问.

7. References
[1] W.Hu, T.Tan, L.Wang, and S.Maybank. "A survey on visual surveillance of object motion and behaviors". IEEE Trans. on Systems, Man, and Cybernetics - Part C, 34(3):334-352, August 2004
[2] C.Stauffer, W.E.L.Grimson, "Adaptive background mixture models for real-time tracking", in: Computer Vision and Pattern Recognition, vol 2, p.2246, Santa Barbara, CA, June, 1998
[3] R.Cucchiara, M.Piccardi, and A.Prati, "Detecting moving objects, ghosts, and shadows in video streams," IEEE Transactions on Pattern Analysis and Machine Intelligence 25, pp. 1337-1342, Oct 2003
[4] Sen-Ching S.Cheung, Chandrika Kamath, "Robust Background Subtraction with Fore-ground Validation for Urban Traffic Video", EURASIP Journal on Applied Signal Processing, pp. 2330-2340, 2005.
[5] Sadiye Guler, Jason A. Silverstein, Ian H. Pushee,
"Stationary Objects in Multiple Object Tracking", IEEE
International Conference on AVSS, pp.248-253, 2007
[6] Fei Yin, D.Makris, S.A.Velastin, "Performance Evaluation of Object Tracking Algorithms", 10th IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS2007), October, Rio de Janeiro, Brazil, 2007
[7] OpenCV Computer Vision Library http://www.intel.com/technology/computing/opencv/index.htm, [Last accessed: August 2007]
[8] Guide to Authoring Media Ground Truth with ViPERGT, http://viper-toolkit.sourceforge.net/docs/gt/, [Last accessed:July 2007]
[9] D.S.Lee, "Effective Gaussian mixture learning for video background substraction", IEEE Transactions on Pattern Analysis and Machine Intelligence 27 (5) pp.827–832, 2005
[10] Sijun Lu Jian Zhang Feng, D. "An efficient method for detecting ghost and left objects in surveillance video", IEEE Conference on Advanced Video and Signal Based Surveillance, pp.540-545, London, Sept. 2007
[11] Barco Display and visualization solutions, http://www.barco.com/ [Last accessed: May 2008]
[12] X.Desurmont, C.Chaudy, A.Bastide, C.Parisot, J.F.Delaigle, B.Macq, "Image analysis architectures and techniques for intelligent systems", IEE Proceedings on Vision, Image and Signal Processing, Special issue on Intelligent Distributed Surveillance Systems, pp. 224-231, April 2005.

转载于:https://www.cnblogs.com/zzywan/archive/2013/03/13/2955958.html