基于少样本学习的森林火灾烟雾检测方法

贾一鸣; 张长春; 胡春鹤; 张军国

doi:10.12171/j.1000-1522.20230044

基于少样本学习的森林火灾烟雾检测方法

doi: 10.12171/j.1000-1522.20230044

北京林业大学工学院，林木资源高效生产全国重点实验室，林业装备与自动化国家林业和草原局重点实验室，北京 100083

基金项目: 森林防火智能巡检装备项目（TC210H00L/40），中央高校基本科研业务费专项资金资助（BLX202129）。

详细信息

作者简介:
贾一鸣。主要研究方向：森林防火。 Email：jiayiming1997@163.com　地址：100083 北京市海淀区清华东路35号北京林业大学工学院

责任作者:
胡春鹤，博士，副教授。主要研究方向：机器学习、无人机和智能控制。 Email：huchunhe@bjfu.edu.cn　地址：同上

张军国，博士，教授。主要研究方向：物联网与无线传感器网络、图像处理以及深度学习。Email：zhangjunguo@bjfu.edu.cn　地址：同上。

中图分类号: TP181；S762.3；TP212
计量
- 文章访问数: 52
- HTML全文浏览量: 26
- PDF下载量: 15
- 被引次数: 0
出版历程
- 收稿日期: 2023-03-03
- 修回日期: 2023-07-24
- 网络出版日期: 2023-07-31

Forest fire smoke detection method based on few-shot learning

School of Technology, Beijing Forestry University, State Key Laboratory of Efficient Production of Forest Resources, Key Laboratory of State Forestry and Grassland Administration on Forestry Equipment and Automation, Beijing 100083, China

摘要

摘要: 目的为解决由于森林火灾烟雾数据集样本量小、样本特征分散、烟雾图像占比小等特点导致的林火烟雾检测模型识别效果差、准确率低等问题，实现快速、准确识别检测森林火灾烟雾。方法针对少样本森林火灾烟雾图像数据集的样本特征，本研究提出了一种基于多头注意力机制的森林火灾烟雾图像检测方法。该方法首先在训练阶段采用数据增强方法，扩充训练数据的数量同时降低过拟合风险；然后设计特征提取模块与特征聚合模块，在特征提取模块中引入多头注意力机制并探讨引入的合适位置，使模型更多地关注火灾局部特征，解决烟雾图像少造成的信息缺失问题；在特征聚合模块中使用FPN-PAN模块对图像的深层与浅层语义信息进行特征融合；最后，设置检测头模块输出实验结果。利用测试准确率、召回率、误报率、检测率和F₁值等评价指标在少样本公共数据集和自建火灾烟雾少样本数据集上测试本方法的有效性。结果在数据增强阶段同时增加mosaic数据增强和多尺度变换，可以得到更好的检测效果。在特征提取模块的第4个卷积模块后面添加1处多头注意力机制的模型性能最好。相较于现有的元学习长短时记忆网络、匹配网络和轻量级目标检测网络等方法，本方法有更好的检测效果，具体表现为准确率达到了98.79%，召回率98.28%，检测率97.33%，误报率仅为6.36%。结论与现有的火灾烟雾检测模型相比，本方法具有更好的判别能力和泛化能力。
- 森林火灾烟雾检测 /
- 少样本 /
- 小目标 /
- 特征提取 /
- 多头注意力机制
Abstract: Objective In order to solve the problems of poor recognition effect and low accuracy of forest fire smoke detection model caused by small sample size, scattered sample features and small proportion of smoke images in forest fire smoke data set, rapid and accurate recognition and detection of forest fire smoke were realized. Method Aiming at the sample characteristics of the forest fire smoke image dataset with few samples, this study proposed a forest fire smoke image detection method based on multi-head attention mechanism. Firstly, data enhancement method was used in the training stage to expand the number of training data and reduce the risk of overfitting. Then the feature extraction module and the feature aggregation module were designed. The multi-head attention mechanism was introduced into the feature extraction module and the appropriate location was discussed to make the model pay more attention to the local features of the fire and solve the problem of information loss caused by fewer smoke images. In the feature aggregation module, FPN-PAN module was used for feature fusion of deep and shallow semantic information of images. Finally, the detection head module was set to output the experimental results. Test precision rate, recall rate, false alarm rate, detection rate and F1 value were used to test the effectiveness of this method on the small sample public data set and the self-built fire smoke small sample data set. Result The experimental results show that adding mosaic data enhancement and multi-scale transformation at the same time in the data enhancement stage can get better detection results. Models that add a multi-head attention mechanism after the fourth convolutional module of the feature extraction module performed best. Compared with the existing methods such as meta-learning short-duration memory network, matching network and lightweight target detection network, the proposed method had better detection effect. The accuracy rate reached 98.79%, the recall rate was 98.28%, the detection rate was 97.33% and the false positive rate was only 6.36%. Conclusion The experimental results show that the proposed method has better discriminative ability and generalization ability than the existing fire smoke detection models.
- forest fire smoke detection /
- a small sample /
- a small goal /
- feature extraction /
- multi-head attention mechanism

HTML全文

图 1 基于特征学习的森林火灾烟雾检测整体结构

Figure 1. Overall structure of forest fire smoke detection based on few-shot learning

下载: 全尺寸图片幻灯片

图 2 特征提取网络结构图

Figure 2. Feature extraction network structure diagram

下载: 全尺寸图片幻灯片

图 3 多头注意力机制

Figure 3. Multi-head attention mechanism

下载: 全尺寸图片幻灯片

图 4 点积注意力机制（SDPA）示意图

Figure 4. Sketch diagram of the dot product attention mechanism

下载: 全尺寸图片幻灯片

图 5 特征聚合模块

Figure 5. Feature aggregation module

下载: 全尺寸图片幻灯片

图 6 自建森林火灾烟雾数据集样本图像

Figure 6. Sample images of self-built forest fire smoke dataset

下载: 全尺寸图片幻灯片

图 7 烟雾图像Grad-CAM和Guided Grad-CAM可视化结果

色柱数值代表能量数值，单位为J，颜色越红代表该像素区域对分类结果的重要性越高。The color column value is the energy value, and the unit is J. The redder the color is, the more important the pixel region is to the classification result.

Figure 7. Grad-CAM and guided grad-CAM visualizations of smoke images

下载: 全尺寸图片幻灯片

图 8 不同注意力机制的准确率

Figure 8. Accuracy rates of different attention mechanisms

下载: 全尺寸图片幻灯片

表 1 数据增强等不同训练方式实验结果对比

Table 1. Comparison of experimental results of different training methods such as data enhancement

编号 No.	图像尺寸 Image size	批量大小 Batch size	迭代轮数 Epoch	随机旋转 Rotate	平移 Shift	缩放 Resize	混合 Mixup	马赛克 Mosaic	多尺度变换 Multi-scale	准确率 Accuracy rate/%	召回率 Recall rate/%	平均准确率 Mean average precision (mAP)/%
1	1280 × 1280	32	5 000	√						96.81	96.31	96.46
2	1280 × 1280	32	5 000		√					96.26	97.07	96.78
3	1280 × 1280	32	5 000			√				96.51	97.00	96.15
4	1280 × 1280	32	5 000				√			96.82	97.31	97.46
5	1280 × 1280	32	5 000					√		97.23	97.86	98.27
6	1280 × 1280	32	5 000						√	98.00	97.66	97.87
7	1280 × 1280	32	5 000					√	√	98.43	98.02	98.57

下载: 导出CSV

表 2 不同学习网络的性能对比

Table 2. Performance comparison of different learning networks

方法 Method	准确率 Accuracy rate/%	误报率 False alarm rate/%	检测率 Detection rate/%	召回率 Recall rate/%	F₁值 F₁-score
元学习长短时记忆网络 Meta-learning long and short-term memory networks	85.34	12.98	94.54	93.13	89.23
匹配网络 Matching Networks	80.16	15.96	89.62	87.75	83.95
Faster RCNN	92.46	11.33	93.05	93.76	93.11
Yolov5	96.69	9.28	96.24	95.89	96.29
Yolov7	98.55	6.41	95.69	96.78	97.81
多头注意力原型网络 Multi-head attention-based prototypical network	98.79	6.36	97.33	98.28	98.53

下载: 导出CSV

表 3 不同添加位置的性能对比

Table 3. Performance comparison for different add locations

添加位置 Add location	平均准确率 Average precision rate (mAP)/%	召回率 Recall rate/%	准确率 Accuracy rate/%
Ori_Net	93.69	91.35	95.28
Multi_head_1	93.61	92.15	95.79
Multi_head_2	96.20	93.98	96.06
Multi_head_3	96.10	95.12	97.3
Multi_head_4	98.43	98.28	98.79
Multi_head_1-2	95.56	93.18	97.19
Multi_head_1-3	96.88	94.46	97.52
Multi_head_1-4	96.97	94.55	97.61
Multi_head_2-3	96.06	94.64	97.71
Multi_head_2-4	96.25	94.82	97.90
Multi_head_3-4	96.21	94.78	97.86
Multi_head_1-2-3	92.94	90.62	94.52
Multi_head_2-3-4	92.66	90.35	94.23
Multi_head_1-2-3-4	91.35	89.07	92.90

下载: 导出CSV

表 4 不同注意力模块的性能对比

Table 4. Performance comparison of different learning networks

方法 Method	准确率 Accuracy rate/%	误报率 False alarm rate/%	检测率 Detection rate/%	召回率 Recall rate/%	F₁值 F₁-score	CPU推理速度 CPU speed/ms	GPU推理速度 GPU speed/ms
CBAM	95.84	15.93	95.23	87.15	91.28	98	9.6
CTAM	96.82	13.94	95.63	91.73	94.21	96	9.5
MHA	98.79	6.36	97.33	98.28	98.53	90	9.3

下载: 导出CSV

参考文献(29)

[1]	陈小华. 由高被引论文的研究主题看森林防火的科研−以《森林防火》为例[J]. 国家林业和草原局管理干部学院学报, 2022, 21(4): 48−53. Chen X H. Scientific research on forest fire prevention from the research themes of highly cited papers: take Forest Fire Prevention as an example[J]. National Academy of Forestry and Grassland Administration Journal, 2022, 21(4): 48−53.
[2]	刘楠, 钟少波, 朱伟. 国内外森林火灾研究的知识图谱分析[J]. 林业科技通讯, 2023: 1−13. Liu N, Zhong S B, Zhu W. Knowledge mapping analysis of domestic and international forest fire research[J]. Forest Science and Technology, 2023: 1−13.
[3]	祝玉华, 司艺艺, 李智慧. 基于深度学习的烟雾与火灾检测算法综述[J]. 计算机工程与应用, 2022, 58(23): 1−11. Zhu Y H, Si Y Y, Li Z H. Overview of smoke and fire detection algorithms based on deep learning[J]. Computer Engineering and Applications, 2022, 58(23): 1−11.
[4]	肖潇, 孔凡芝, 刘金华. 基于动静态特征的监控视频火灾检测算法[J]. 计算机科学, 2019, 46(6): 284−286. Xiao X, Kong F Z, Liu J H. Monitoring video fire detection algorithm based on dynamic characteristics and static characteristics[J]. Computer Science, 2019, 46(6): 284−286.
[5]	Li P, Zhao W D. Image fire detection algorithms based on convolutional neural networks[J]. Case Studies in Thermal Engineering, 2020, 19: 100625. doi: 10.1016/j.csite.2020.100625
[6]	Shahid M, Chien I F, Sarapugdi W, et al. Deep spatial-temporal networks for flame detection[J]. Multimedia Tools and Applications, 2021, 80(28/29): 35297−35318.
[7]	Lee Y, Shim J. False positive decremented research for fire and smoke detection in surveillance camera using spatial and temporal features based on deep learning[J]. Electronics, 2019, 8(10): 1167. doi: 10.3390/electronics8101167
[8]	Liu Z C, Zhang K, Wang C Y, et al. Research on the identification method for the forest fire based on deep learning[J]. Op-tik, 2020, 223: 165491.
[9]	Chen Y Y, Chen K, Bai H, et al. An efficient fire detection algorithm based on multi-scale convolutional neural network[J]. Fire and Materials, 2021(1): 1−12.
[10]	Gu K, Zhang Y, Qiao J. Vision-based monitoring of flare soot[C]//Yan R Q. Transactions on instrumentation and measurement. Tampa: IEEE, 2020: 7136−7145.
[11]	Gu K, Xia Z, Qiao J. Deep dual-channel neural network for image-based smoke detection[C]//Yan R Q. Transactions on multimedia. Tampa: IEEE, 2020: 311−323.
[12]	Gu K, Zhang Y, Qiao J. Ensemble meta-learning for few-shot soot density recognition[C]//Yan R Q. Transactions on industrial informatics. Tampa: IEEE, 2021: 2261−2270.
[13]	Majid S, Alenezi F, Masood S, et al. Attention based CNN model for fire detection and localization in real-world images[J]. Expert Systems with Applications, 2022, 189: 116114. doi: 10.1016/j.eswa.2021.116114
[14]	Li T T, Zhu H W, Hu C H, et al. An attention-based prototypical network for forest fire smoke few-shot detection[J]. Journal of Forestry Research, 2022, 33: 1493−1504. doi: 10.1007/s11676-022-01457-6
[15]	祝钧桃, 姚光乐, 张葛祥. 深度神经网络的小样本学习综述[J]. 计算机工程与应用, 2021, 57(7): 22−33. Zhu J T, Yao G L, Zhang G X. Survey of few shot learning of deep neural network[J]. Computer Engineering and Applications, 2021, 57(7): 22−33.
[16]	张毓, 高雅月, 常峰源. 小样本条件下基于数据扩充和ResNeSt的雪豹检测[J]. 北京林业大学学报, 2021, 43(10): 89−99. Zhang Y, Gao Y Y, Chang F Y. Panthera unica recognition based on data expansion and ResNeSt with few samples[J]. Journal of Beijing Forestry University, 2021, 43(10): 89−99.
[17]	Dimitropoulos K, Barmpoutis P, Grammalidis N. Higher order linear dynamical systems for smoke detection in video surveillance applications[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2017, 27(5): 1143−1154. doi: 10.1109/TCSVT.2016.2527340
[18]	Lee G, Hong S, Cho D. Self-supervised feature enhancement networks for small object detection in noisy images[J]. IEEE Signal Processing Letters, 2021, 28: 1026−1030. doi: 10.1109/LSP.2021.3081041
[19]	王晓松, 杨刚. 一种融合聚类和分类算法的树木图像多目标优化分割方法[J]. 北京林业大学学报, 2018, 40(12): 124−131. Wang X S, Yang G. A multi-objective optimization segmentation method for tree image based on fusion clustering and classification algorithm[J]. Journal of Beijing Forestry University, 2018, 40(12): 124−131.
[20]	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Advances in Neural Information Processing Systems, 2017, 30: 1−15.
[21]	Krstini D, Stipanicev D, Jakovcevi T. Histogram-based smoke segmentation in forest fire detection system[J]. Information Technology & Control, 2009, 38(3): 237−244.
[22]	Russakovsky O, Deng J, Su H, et al. Imagenet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015, 115(3): 211−252. doi: 10.1007/s11263-015-0816-y
[23]	崔艳鹏, 王元皓, 胡建伟. 一种改进YOLOv3 的动态小目标检测方法[J]. 西安电子科技大学学报, 2020, 47(3): 1−7. Cui Y P, Wang Y H, Hu J W. Detection method for a dynamic small target using the improved YOLOv3[J]. Journal of Xidian University, 2020, 47(3): 1−7.
[24]	鞠默然, 罗海波, 王仲博, 等. 改进的YOLO V3 算法及其在小目标检测中的应用[J]. 光学学报, 2019, 39(7): 245−252. Ju M R, Luo H B, Wang Z B, et al. Improved YOLO V3 algorithm and its application in small target detection[J]. Acta Optica Sinica, 2019, 39(7): 245−252.
[25]	Sun C, Ai Y, Wang S, et al. Mask-guided SSD for small- object detection[J]. Applied Intelligence, 2021, 51(20): 1−12.
[26]	Lian J, Yin Y, Li L, et al. Small object detection in traffic scenes based on attention feature fusion[J]. Sensors, 2021, 21(9): 3031. doi: 10.3390/s21093031
[27]	Selvaraju R R, Cogswell M, Das A. Grad-cam: visual explanations from deep networks via gradient-based localization[J]. International Journal of Computer Vision, 2020, 128: 336−359. doi: 10.1007/s11263-019-01228-7
[28]	Woo S, Park J, Lee J Y, et al. CBAM: convolutional block attention module[C]//Vittorio F, Martial H, Cristian S, et al. Proceedings of the European conference on computer vision. Munich: ECCV, 2018: 3−19.
[29]	Misra D, Nalamada T, Arasanipalai A U, et al. Rotate to attend: convolutional triplet attention module[C]//Vinay P N, Luc V G, Theo G, et al. Winter conference on applications of computer vision. Waikoloa: IEEE, 2021: 3138−3147.