论文阅读＞烟雾检测：Smoke detection based on imageprocessing by using grey and transparency features

Smoke detection based on imageprocessing by using grey and transparency features

通过利用灰度和透明度特征基于图像处理的烟雾检测

Smoke detection based on imageprocessing by using grey and transparency features

AHMED FAKHIR MUTAR, DR. HAZIM GATI’ DWAY

摘要

在本研究中，我们通过分析早期烟雾的特点，改进了基于帧运动的烟雾检测方法。
利用背景和不同的建模方法对每一帧中的运动目标进行精确检测。
然后，将图像转换为二值化模式，并将不需要的亮度像素从图像中去除。
通过灰度和透明度两种特征检测烟雾。
第一个特征取决于物体的标准偏差，第二个特征测量图像的透明度。
实验结果表明，该算法能实现接近92%的烟雾检测率。
这些结果是通过使用准确度标度作为数学基础进行分类观察得到的。

In this study, we improve smoke detection approach based on frame movement by analyzing the characteristics of early smoke. Background and different modeling methods are used to detect moving objects in every frame accurately. Sequentially, the image was converted to binary mode, and while undesirable lightness pixels are removed from the image. Smoke was detected by using two features, namely, gray and transparency. The first feature depends on the standard deviation of the object, and the second one measures image transparency. Experimental results show that the suggested algorithm can achieve a high detection rate of smoke approach to 92%. These results were observed by using accuracy scale as a mathematical base for classification.

关键词

运动检测，烟雾检测，标准偏差，透明度。

Keyword: Motion Detection; Smoke Detection; Standard Deviation; Transparency.

1 介绍

烟雾探测有时代表着大多数火灾的早期迹象。监测空气感染及其对人类健康和自然的影响非常重要；因此，必须有一种有效的烟雾探测方法[1]。尽早发现火灾对于减少火灾损失十分重要。火焰或烟雾通常代表着火灾或森林大火发出的第一警报。如果火焰出现的地方很远或被诸如山脉或建筑物之类的障碍物遮挡，在监视摄像机上就有可能看不到火焰。烟雾是森林火灾的有效指标，但在图像中识别烟雾是困难的，因为它缺乏特定的形状或颜色模式[2]。

Celik等人[3]提出了一种利用图像处理的烟雾检测方法。他们使用颜色和饱和度信息来检测图像中的烟雾。通常，灰色的烟是非常可靠的。烟雾信息用于早期火灾探测系统；因此，当烟雾显示出低热量和低饱和度时，就可以检测到烟雾特征。 Chen[4]提出了一种基于视频处理的烟雾探测算法，用于早期火灾报警系统。该算法依赖于基于静态决策规则的着色效应，而扩散受动态决策准则的影响。静态决策依赖于烟的灰色，而动态决策规则基于烟雾的扩散属性，例如烟的无序度和生长速率。灰色是用HSI色彩系统的强度成分来描述的。 Kopilovic等人[5]观察到，由于烟的非硬度，物体的运动会出现不规则现象。他们在贝叶斯分类器中应用多层光流计算和运动分布熵来检测烟雾的特殊运动。Yuan[6]提出了一种快速定位模型，该模型可以有效地找到运动特征，从而节省了计算时间。尽管在该领域的开发方面已取得了重大进展，但是在一般的监视系统中采用这些方法的报道却很少。基于外观的方法也被用于检测烟雾。 Simone Calderara[7]提出了一种基于运动分割算法的烟雾检测的系统，利用小波变换能量系数和图像颜色特性进行检测处理。

使用小波变换系数分析能量，评估其时间演变。根据烟雾颜色模型分析目标的颜色属性，检测场景中的颜色变化是否为自然变化，采用两种贡献可能性度量解决了每个特征选择和出现的大部分问题，并且大大提高了检测过程的效率。在这里，我们提出了在视频中实施的新技术，以提供烟雾检测的最佳结果。

本文的其余部分结构如下。
运动对象检测和区域中的不同点在第（2）节。
第（3）节介绍了每个区域的图像分析和特征提取。
在第（4）节中，我们使用了一种形态学方法，通过考虑物体颜色的平均标准偏差来检测烟雾，并应用傅里叶变换来区分透明烟雾并跟踪物体。

Smoke detection some time represents an early sign of most fires. It is important to monitor air infection and its effects on human health and nature; therefore, an effective method to detect smoke is necessary [1] The Early detection of fires is important to reduce fire damage. The flame or smoke usually represents the first alarm that a fire or forest fire gives it. Flames may not be visible at monitoring cameras if the flames appear at along distance far away or they are obscured by obstacles, such as mountains or buildings. Smoke is an effective indicator of a forest fire, but identifying smoke in images is difficult because it lacks a specific shape or color pattern [2].
Celik et al. pro[3] proposed a method for smoke detection using image processing. They used color and saturation information in order to detect smoke in images. In general, a grayish color of smoke is reasonably reliable. Smoke information is used for early fire detection systems; therefore, smoke features are detected when smoke exhibits low heat and low saturation. Chen [4] proposed a smoke detection algorithm based on video processing for early fire alarm systems. The algorithm depends on a chromatically effect which is based on static decision rule, while diffusion-affected by dynamic decision criterion. The static decision relies on the grayish color of smoke, and the dynamic decision rule is based on the spreading attributes of smoke, such as smoke disorder and growth rate. The grayish color is described using the intensity component of the HSI color system. Kopilovic et al. [5] observed that irregularities occur in the motion of the objects because of the non-hardness of smoke. They applied a multilevel optical flow computation and the entropy of the motion distribution in the Bayesian classifier to detect the special motion of smoke. Yuan [6] proposed a fast orientation model that effectively finds the motion characteristics, thereby saving computational time. Although significant advances have been made in the development of this area, the adoption of these methods in general monitoring systems is not widely reported. Appearancebased approaches are also used to detect smoke. Simone Calderara [7]proposed a system able to smoke detecting by using means of the motion segmentation algorithm and both Wavelet Transform energy coefficients and image color properties were used to detect process.
Where the energy is analyzed using the wavelet Transform coefficients evaluating its temporal evolution. The color properties of the objects are analyzed accordingly to a smoke color model to detect if color changes in the scene are due to a natural variation or not and the adoption of a two contributions likelihood measure solves most of the emerged problems of each chosen feature and boosts up significantly the detection process Here, we propose new techniques to implement in video to provide optimized results in smoke detection. The remainder of the paper is structured as follows. Objects motion detection and the difference in regions are defined in Section (2). Image analysis and feature extraction in each region are presented in Section (3). In Section (4), we used a morphological method for smoke detection by accounting average standard deviation to object color and applied Fourier transform to distinguish transparent smoke and track objects.

2 颜色模型和烟雾特征分析

在烟雾检测中，确定烟雾的颜色模型和分析由烟雾样本组成的图像是非常重要的[8]。HSV颜色空间是有意选择的，因为与其他颜色空间相比，它能够更有效地从色度中分离光(或光度)信息。烟雾像素不表明定殖（？）属性，如火焰像素，其中烟雾大多是灰色的，并包含一些颜色细节。因此，亮度信息比消色差信息要大得多。图1显示了烟区RGB和HSV分量的直方图。值得注意的是，RGB组件的直方图大致相同。在HSV情况下，光分量与RGB均匀分布，且大量分布。因此，我们可以用公式表示烟雾像素：


其中，α是一个范围从0到1的全局阈值。

图1 烟雾区域的RGB和HSV色彩空间的直方图

Determining the color model for smoke and analyzing the images that consist of smoke samples are important in the smoke detection [8]. HSV color space is chosen intentionally because of its capability to separate light (or luminosity) information from chrominance more effectively compared with other color spaces. Smoke pixels do not show colonization properties, such as fire pixels, in which the smoke is mostly gray and contains a few color details. Thus, the luminance information is much larger than the achromatic information. Figure 1 shows the histogram of the RGB and HSV components for the smoke region. Notably, the histograms of the RGB components are approximately equal. In the case of HSV, the light component is distributed equally with RGB and it presents a large distribution.

3 系统

该系统由固定摄像机提供的视频组成，可通过视频分析检测烟雾。将视频数据转换为一系列帧，并对每一帧进行分析。当火灾开始时，烟或火焰的特征很难识别；但是随着火灾强度的增加，属性的识别将变得容易。通过图像分析来确定烟雾的特征。图像中的所有物体都是通过改变运动来识别的。下一步，对图像中的每个物体进行测试，在此基础上确定其冒烟行为。

The system consists of video given by a fixed camera to detect smoke by video analysis. The video data are converted to a series of frames, and each frame is analyzed. As fires begin, the characteristics of smoke or flame are difficult to identify; however, as the intensity of fire increases, the identification of properties becomes easy. Determining the characteristics of smoke is performed by image analysis. All the objects in the image are identified by changing the movement. The next step, each object is tested in the image to determine the smoke behavior on the basis of the following steps.

3.1 烟雾检测算法

烟雾探测对于确定火灾的早期探测至关重要。烟的特性可以由它的五个主要部分来决定。首先，我们将输入的视频转换成一组帧，通过减去帧来确定图像中运动区域的目标。也需要利用运动检测，除了转换图像到HSV颜色，我们也使用模型来分析颜色和强烟候选特征。此外，我们删除或减少不需要的亮度像素，并跟踪满足阈值和条件的对象。计算得到的对象颜色的每个颜色分量(RGB)的标准偏差。我们采用最高值，并将其与烟的特性进行比较，以确定是否存在烟雾。最后利用傅里叶变换计算烟雾过程中所有目标的透明度，从而提高了系统的精度。

Smoke detection is essential in defining the early detection of a fire. Smoke can be determined by the characteristics of its five main parts. First, we convert input video into a sequence of frames and determine objects of moving areas in an image by subtracting the frames to find all objects in each frame. Motion detection is also used, In addition to converting the image to HSV color, we also use models to analyze color and intensity smoke candidate features. Furthermore, we remove or reduce undesirable lightness pixels and track the objects that meet the threshold and conditions. The standard deviation was calculated for each color component (RGB) of The colors of the resulting object. We adopt the highest values and compare them with the characteristics of smoke to determine whether smoke is present. Finally, Fourier transform was used to calculate the transparency of all objects during the smoke, thereby the accuracy of the system increases.

3.1.1 视频帧转换

预处理视频在分辨率的基础上转换为顺序帧，从而可以进行火灾之间的比较并确定处理方法的差异。该过程可以提高所提出的检测算法的性能并减少错误警报。 该算法包括运动检测和将帧转换为二进制，减少不希望的亮度像素。

The pre-processing video is converted to a sequential frame on the basis of resolution, thereby, enabling comparison between fires and determining the differences in processing methods. This process may increase the performance of the proposed detection algorithm and reduce false alarms. The algorithm includes motion detection and conversion of the frame to binary, thereby decreasing undesirable lightness pixels.

3.1.2 运动检测和颜色变换

运动检测是提取每一帧目标的必要步骤。在这个过程中，第一个帧将被存储为背景，并查找顺序帧之间的差异。基于特定的步骤在序列帧之间使用减法，提高了处理速度，减少了处理时间，从而方便地检测帧之间的差异，确定每一帧的最终目标。应用每个对象的跟踪机制来确定和存储其坐标，并将其标记为：
其中，C表示图像序号，n表示帧数。
如果d>t，则i = i + 1。
其中，值应为d>t以提取对象。
例如，如果连续两幅图像相似，则它们的d值将接近0；如果它们不相同，那么它们的d值就会很大。
在提出的算法中，每个对象都将在帧内进行测试。帧将被转换为HSV颜色空间，从而尽可能从帧内所有对象的强度中确定值。
V值可以由下式获得：

其中，V是RGB颜色值中的最大值。

3.1.3 去除不需要的亮度像素和标记目标

照明水平的变化在决定物体的属性方面起着关键的作用。对每个物体的属性进行确定和测试，以确定该物体是否为烟。提出了一种利用阈值亮度像素去除二值图像中不需要的亮度像素和小目标的算法。在算法中，将亮度值小于阈值的像素删除，如图3所示。

图3 组(1) (A,B)表示背景图像，(C,D)表示烟雾状态图像，(E,F)表示相减后运动检测目标的二值图像。

利用包括扩张和腐蚀在内的形态闭合过程来填充孔洞，并删除不需要的小目标和亮度像素，以准确地确定目标，并在帧内跟踪所有物体。 图2流程图表示检测和跟踪目标的主要步骤。

图2 目标检测和跟踪步骤流程图

此过程如等式(7)所示，可提高检测的准确性和速度。我们使用了一个形态闭合过程，包括膨胀和侵蚀来填补洞，以准确确定对象和所有对象被跟踪在框架内。
亮度不佳的小像素和小物体的形态膨胀和腐蚀面积可通过以下方法获得：

如果不止一个对象满足阈值，则将存储并标记每个对象的坐标。 然后，使用背景图像中的坐标来清楚地选择对象，并通过以下算法应用统计计算。
标记区域算法1：

1: binary image; V(x, y) = 0: background,  V(x, y) = 1: foreground 
The image V is labeled (destructively modified) and returned. 
2: Let m ← 2  …     value of the next label to be assigned 
3: for all image coordinates (x, y) do 
4: if V(x, y) = 1 then 
5: Set V(x, y) ← label 
6:  Check all pixels neighbors: V(x+1,y),  V (x, y+1), V (x-1, y), V (x, y-1) 
7: m ← m + 1. 
8: return the labeled image V. 
9. End 

3.1.4 通过标准偏差值检测烟雾

目标提取完成后，需要单独进行烟雾检测。烟雾可以在大范围内改变其亮度-颜色性能的值，从透明的灰色到灰蓝色。因此，我们分析了强度区域。在烟雾目标分析[11]中，RGB颜色空间中的烟雾颜色值由条件R±a= G±a =B±a得到。该规则意味着烟雾像素的三个组成部分(RGB)是相等的。
跟踪技术应用于每个对象，以定位和存储所有对象的坐标。
为了确定每个目标的颜色属性，我们使用等式(8)计算每个分量(R, G，and B)的标准差来测量各值之间的近似值。我们还通过等式(9)计算颜色组件(RGB)的最大(std)值，用于基于阈值的测试。因此，有色的烟雾目标被识别，如下面的等式。

其中，X为向量，i表示RGB分量，μ是平均值。
获得RGB分量的最大标准偏差。

其中，如果标准偏差的差很小，则目标表示烟雾属性。 下一步，当满足以下条件时，应用阈值。目标代表感兴趣区域，其中该区域包含用于灰烟的灰度渐变和用于有色烟的非灰色渐变。

· Fourier transform 傅里叶变换

傅里叶分析通常用于数据分析，因为它将信号分解成不同频率的正弦分量。快速傅里叶变换(FFT)被认为是一种有效的算法，它特别适用于图像处理和信号处理等领域，其应用范围从卷积、滤波、频率分析到功率谱、估计。离散傅里叶变换（DFT）通常是在计算机上与傅里叶变换一起使用的，其中涉及一种变换形式。图像代表两个图像的第一傅里叶变换。

图6：(a)无烟雾图像的快速傅里叶变换(FFT)， (b)透明烟雾图像的快速傅里叶变换(FFT)。

3.1.5 利用傅里叶变换测量透明烟雾

傅里叶变换在透明检测中非常重要。在图7、8中，我们注意到烟雾图像中某些颜色(颜色信息)的存在表现了透明度。该算法利用公式(11)对每个通道(RGB)各色分量的最大值进行傅里叶变换，从而计算透明度系数。因此，本研究提高了对透明烟雾的检测能力和准确性。根据(12,13)采用傅里叶变换检测透明度。

其傅里叶域性质如下：

nj是频率，j = 1,2,3…k，K是频率数。
我们可以使用两个因素作为特征来确定对象的透明度。第一个是直方图的最大峰值的值，第二个是两幅图像之间移动的变化值(a,b)。如果图像(b)的最大值大于图像(a)的最大值，则偏移值尽可能小。
该目标可视为烟雾，如图7、8和表1所示。

图7 组(3) 移动值变化和识别透明对象的最大峰值

图8 显示移动变化值和最大峰值来识别透明目标

其中透明度比例越大，峰值越小，移动值越大，
通过公式(14)得到的最大概率为：

特征1表示烟雾，有色和无色区域，根据等式11可以仅包含灰色渐变。
由于烟雾具有透明性，必须始终满足第二个特征才能将物体识别为烟雾，如表1所示。

表1 将Group1(A,B)和Group4(B)图像与背景图像E(Group1(C,D)和Group4(A)进行傅里叶变换后的两个特征，即最高峰值和移动值，以确定透明对象

4 目标追踪与边缘

目标跟踪是计算机视觉中的一项重要任务。在视频分析中有三个重要的步骤，即感兴趣的运动目标的检测、帧跟踪和目标轨迹的分析来识别目标的行为。目标跟踪复杂性的原因是图像噪声、场景光照变化、复杂的目标运动以及部分和全部目标遮挡。大多数跟踪算法都假设目标运动平稳，没有突然变化[9]。在获得所有条件并在目标周围画出边界后，我们依靠帧来跟踪这些目标，如图9所示。

图9 组(4)，表示跟踪的烟雾目标和绘制的边界

使用算法2可以总结出上述步骤。

算法2：建议法检测烟雾和目标跟踪

并在接下来的图表中给出了烟雾检测和目标跟踪的建议方法和步骤。

图10 建议方法流程图

我们使用MATLAB对选定的视频进行测试，其中一些视频由图11[14,15]中的火灾探测提供。
计算速度为每秒25帧，所有视频归一化为320像素乘240像素。烟雾对象显示在绿色边框中。准确的计算结果[10]由此获得。


图11 组(2)，表示本系统在不同条件下对多个在线视频的工作快照。在图像中选择的区域被检测为烟雾。

表2 该方法的烟雾检测状态

Object tracking is a critical task in computer vision. Three important steps are used in video analysis, namely, detection of interested moving objects, tracking of such objects by frame, and analysis of object tracks to recognize their behavior. The complexity of object tracking is due to image noises, scene illumination changes, complex object motion, and partial and full object occlusion. Most tracking algorithms assume that the motion of the object moves smoothly and indicates no sudden change [9]. We rely on tracking of such objects by frame to track our object after attaining every condition and drawing boundaries around the object, as shown in Figure 9.

5 结果与讨论

在本研究中，我们通过分析早期烟雾的特征，改进了一种基于帧运动的烟雾检测方法。利用背景和不同的建模方法检测每一帧的运动目标。运动检测后，将图像转换为二值模式，去除图像中不需要的亮度像素。烟雾检测使用两种特征，即灰度特征和透明度特征。通过傅里叶变换揭示了透明烟，通过计算均值和标准差揭示了灰色烟。本文方法在多个视频上进行了测试，如图10所示。结果inTable 3解释了系统中每段视频在帧变换次数、帧烟雾和第一次烟雾帧检测次数方面的一个特征，出现率为92.6%，如表3所示。

In this study, we improve a smoke detection approach based on frame movement by analyzing the characteristics of early smoke. Background and different modeling methods are used to detect moving objects in every frame. After motion detection, the image is converted to binary mode, and undesirable lightness pixels are removed from the image. Smoke is detected using two features, namely, gray and transparency features. We reveal the transparent smoke by Fourier transform and gray smoke by calculating mean and standard deviation. The proposed method is tested on several videos, as shown in Figure10.and the result inTable 3 explain a feature of every video in term number of frame video convert and a number of the frame that present frame smoke and first smoke frame detect in the system and the result appear is 92.6 % in Table 3

6 结果

该方法的精度达到92.6%。将所提方法的结果与[11,12,13]中的方法进行比较，表4显示所提方法的检测率在所比较的方法中是最高的。因此，该方法具有较高的检测精度。

The proposed approach obtains an accuracy of 92.6%. The results of the proposed method are compared with those of the methods in[11] [12] [13], Table 4 shows that the proposed method obtains the highest percentage of detection among the compared methods. Therefore, the proposed method is superior in terms of detection accuracy.

参考文献

[1] Memane, S. and V. Kulkarni. (2015)." A review on flame and smoke detection techniques in video’s". Int. J. of Advanced Research in Electrical, Electronics and Instrumentation Engineering. 4(2): p. 885889.
[2] Kim, D. and Y.-F. Wang.(2009).“Smoke detection in video”. in Computer Science and Information Engineering, WRI World Congress on. 2009. IEEE.
[3] Celik, T.,et al ( 2007). “Fire detection using statistical color model in video sequences”. Journal of Visual Communication and Image Representation. 18(2): p. 176-185.
[4] Chen, T.-H., et al(2006). “The smoke detection for early fire-alarming system base on video processing”. in Intelligent Information Hiding and Multimedia Signal Processing. IIH-MSP’06. International Conference on. 2006. IEEE.
[5] Kopilovic, I., B. Vagvolgyi, and T. Szirányi.( 2000)." Application of panoramic annular lens for motion analysis tasks: surveillance and smoke detection". in Pattern Recognition,. Proceedings. 15th International Conference on. 2000. IEEE
[6] Yuan, F., A(2008)." fast accumulative motion orientation model based on integral image for video smoke detection".Pattern Recognition Letters. 29(7): p. 925-932.
[7] Calderara, S., P. Piccinini, and R. Cucchiara, Vision based smoke detection system using image energy and color information. Machine Vision and Applications, 2011. 22(4): p. 705-719.
[8] Çelik, T., H. Özkaramanlı, and H. Demirel.(2007). “Fire and smoke detection without sensors: image processing based approach”. in Signal Processing Conference, 15th European. 2007. IEEE.
[9] Shantaiya, S., K. Verma, and K. Mehta.( 2015). “Multiple object tracking using kalman filter and optical flow”. European Journal of Advances in Engineering and Technology,. 2(2): p. 34-39.
[10] Kong, S.G., et al.(2016). “Fast fire flame detection in surveillance video using logistic regression and temporal smoothing”. Fire Safety Journal,. 79: p. 37-43.
[11] Ye, S., et al.(2017)." An effective algorithm to detect both smoke and flame using color and wavelet analysis". Pattern Recognition and Image Analysis. 27(1): p. 131-138.
[12] Avgerinakis, K., A. Briassouli, and I. Kompatsiaris.(2012.). “Smoke detection using temporal HOGHOF descriptors and energy color statistics from video”. in International Workshop on Multi-Sensor Systems and Networks for Fire Detection and Management.
[13] Lee, C.-Y., et al(2012)." Smoke detection using spatial and temporal analyses". International Journal of Innovative Computing, Information and Control,. 8(7): p. 4749-4770.
[14]http://signal.ee.bilkent.edu.tr/VisiFire/Demo/ SampleClips.html
[15]http://imagelab.ing.unimore.it/visor/video_vi deosInCategory.asp?idcategory=8