基于改进YOLOv5木板材表面缺陷的定量识别

贾浩男; 徐华东; 王立海; 张金生; 褚晓辉; 唐旭

doi:10.12171/j.1000-1522.20220419

基于改进YOLOv5木板材表面缺陷的定量识别

东北林业大学机电工程学院，黑龙江哈尔滨 150040

基金项目: 中央高校基本科研业务费专项（2572022BL03），国家自然科学基金面上项目（31870537）

详细信息

作者简介:
贾浩男。主要研究方向：木材无损检测。Email：jhn@nefu.edu.cn　地址：150040黑龙江省哈尔滨市香坊区和兴路26号东北林业大学机电工程学院

责任作者:
徐华东，教授，博士生导师。主要研究方向：活立木质量检测。Email：xhd-8215@163.com　地址：同上

中图分类号: S784；TP391.4
计量
- 文章访问数: 1295
- HTML全文浏览量: 414
- PDF下载量: 171
出版历程
- 收稿日期: 2022-10-19
- 修回日期: 2023-03-15
- 录用日期: 2023-03-18
- 网络出版日期: 2023-03-20
- 发布日期: 2023-04-24

Quantitative identification of surface defects in wood paneling based on improved YOLOv5

College of Mechanical and Electrical Engineering, Northeast Forestry University, Harbin 150040, Heilongjiang, China

摘要

摘要:
目的为解决人工及传统数字图像处理方法对木板材表面缺陷识别效果差、效率低等问题，提高木材利用率。以深度学习模型为基础，构建木板材表面缺陷检测系统，旨在拓展深度学习模型在木板材缺陷检测领域的应用。
方法基于“Wood Defect Database”公开数据集中的839张木板材缺陷图像，使用Imgaug数据增强库对数据集进行扩充；通过在主干特征网络部分引入SE注意力机制，使用focus、FPN + PAN结构构建YOLOv5木板材表面缺陷目标检测框架，进而采用迁移学习思想改进训练方式，将训练过程分为两个阶段（冻结阶段和解冻阶段）。然后将构建的模型与当前主流深度学习目标检测模型进行对比，最后利用混淆矩阵、Loss值变化曲线、模型大小、检测时间以及均值平均精确率等指标评价模型。
结果提出了一种基于YOLOv5模型对木板材表面缺陷中活节、死节、裂缝、孔洞的检测方法。模型对死节、活节、裂缝、孔洞识别结果的均值平均精确率分别约为98.66%、99.06%、98.10%和96.53%，并与当前主流检测模型进行比较，改进的模型具有更好的精确率、召回率和均值平均精确率，分别为97.48%、96.53%和98.22%。模型单幅图像平均检测时间为10.3 ms，最大检测耗时20.5 ms，检测效果与泛化特性较好，模型所占内存仅13.7 MB，易于移植。
结论实验表明改进的YOLOv5模型可用于检测木板材表面主要缺陷。且模型对木板材表面缺陷的识别效果优于其他5种主流检测模型。在维持原有检测精度的基础上，提高了小目标缺陷的识别能力，减少了木板材缺陷漏检的情况，实现了在复杂场景下的快速检测。
- 木板材 /
- 表面缺陷 /
- YOLOv5 /
- 实时检测 /
- 深度学习 /
- 定量识别
Abstract:
Objective This paper aims to solve the problems of poor recognition and low efficiency of surface defects of wood panel lumber by manual and traditional digital image processing methods, and to improve the utilization rate of wood. Based on the deep learning model, we constructed a wood panel surface defect detection system, aiming to expand the application of deep learning model in the field of wood panel defect detection.
Method Based on 839 wood panel defect images in the public dataset “Wood Defect Database”, the dataset was expanded using Imgaug data enhancement library; by introducing SE attention mechanism in the backbone feature network part, the YOLOv5 wood panel surface defect target detection framework was constructed using focus, FPN + PAN structure, and then the transfer learning idea to improve the training method and divide the training process into two phases (freezing phase and unfreezing phase). Then the constructed model was compared with the current mainstream deep learning target detection models, and finally the model was evaluated using confusion matrix, loss value change curve, model size, detection time, and mean average accuracy.
Result A detection method based on YOLOv5 model for live knots, dead knots, cracks and holes in wood panel surface defects was proposed. The mean average accuracy of the model for dead knots, live knots, cracks, and hole identification results were about 98.66%, 99.06%, 98.10% and 96.53, respectively, and compared with the current mainstream detection models, the improved model had better accuracy, recall, and mean average accuracy of 97.48%, 96.53% and 98.22%, respectively. The average detection time of the model for a single image was 10.3 ms, and the maximum detection time was 20.5 ms. The detection effect and generalization characteristics were good, and the model only occupied 13.7 MB of memory, making it easy to transplant.
Conclusion The experiments indicate that the improved YOLOv5 model can be used to detect the main defects on the surface of wood paneling. The model is better than the other five mainstream inspection models in identifying surface defects. On the basis of maintaining the original detection accuracy, it improves the recognition of small target defects, reduces the situation of missing wood panel defects, and realizes fast detection in complex scenes.
- wood plate /
- surface defect /
- YOLOv5 /
- real-time detection /
- deep learning /
- quantitative identification

HTML全文

图 1 通过Imgaug数据增强后图片与原图片的对比

Figure 1. Comparison between the image enhanced by Imgaug data and the original image

下载: 全尺寸图片幻灯片

图 2 YOLOv5s网络结构图

i为输入通道数，o为输出通道数，k为卷积核大小，s为步长，n为卷积数量。i is the number of input channels, o is the number of output channels, k is the convolution kernel size, s is the step size, and n is the number of convolutions.

Figure 2. Network structure diagram of YOLOv5s

下载: 全尺寸图片幻灯片

图 3 SE-Net结构示意图

${X}_{1}$ 指输入，U是主干网络每一层卷积层的输出，c、w、h、C、W、H均为特征向量， ${X}_{2}$ 表示结合了权重之后最终的输出。 ${F}_{\mathrm{t}\mathrm{r}}$ 为卷积操作，运算 ${F}_{\mathrm{s}\mathrm{q}}$ 为挤压操作， ${F}_{\mathrm{e}\mathrm{x}}$ 表示激励操作， ${F}_{\mathrm{s}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{e}}$ 指代缩放操作。 ${X}_{1}$ refers to input, U refers to the output of each convolution layer of the backbone network, c, w, h, C, W, H are the eigenvector, ${X}_{2}$ represents the final output after combining weights. ${F}_{\mathrm{t}\mathrm{r}}$ is a convolution operation, ${F}_{\mathrm{s}\mathrm{q}}$ is the squeeze operation, ${F}_{\mathrm{e}\mathrm{x}}$ stands for excitation operation, ${F}_{\mathrm{s}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{e}}$ refers to the scale operation.

Figure 3. SE-Net structure diagram

下载: 全尺寸图片幻灯片

图 4 引入SE-Net注意力机制模块

Figure 4. Introducing SE-Net attention mechanism module

下载: 全尺寸图片幻灯片

图 5 引入注意力机制前后识别结果对比

Figure 5. Comparison of recognition results before and after the introduction of attention mechanism

下载: 全尺寸图片幻灯片

图 6 改进模型分析结果总结

mAP@0.5表示在交并比（IoU）设为0.5时，每一个类别下所有图片的均值平均精确率。 mAP@0.5 indicates the average accuracy of all images in each category when the intersection and combination ratio is set to 0.5.

Figure 6. Summary of improved model analysis results

下载: 全尺寸图片幻灯片

图 7 木板材缺陷识别效果图

Figure 7. Effect picture of wood plate defect identification

下载: 全尺寸图片幻灯片

表 1 数据集标注数量

Table 1 Number of dataset annotations

标签 Label	死节 Dead knot	活节 Live knot	孔洞 Hole	裂缝 Crack
数量 Quantity	3 311	2 117	825	2 269

下载: 导出CSV

表 2 实验环境

Table 2 Experimental environment

配置名称 Configuration name	版本参数 Version parameter
系统环境 System environment	Ubuntu18.04
中央处理器 Central processing unit	AMD Ryzen7 4800H with Radeon Graphics@2.90 GHz
图形处理器 Graphics processing unit	NVIDIA GeForce RTX 2060 6 GB
图形处理器加速库 Graphics processing unit acceleration library	CUDA tookit10.1,cuDNN7.5.6
随机存取存储器 Random access memory	16 GB
深度学习框架 Deep learning framework	Pytorch1.8.0

下载: 导出CSV

表 3 改进YOLOv5模型对不同缺陷识别结果对比

Table 3 Comparison of improved YOLOv5 model for different defect identification results %

标签 Label	精确率 Precision rate	召回率 Recall rate	mAP@0.5	mAP@0.5∶0.95
死节 Dead knot	97.35	97.50	98.66	78.74
活节 Live knot	97.92	98.14	99.06	83.19
裂缝 Crack	96.70	91.55	98.10	72.22
孔洞 Hole	95.57	96.73	96.53	80.15
注：mAP@0.5表示在交并比（IoU）设为0.5时，每一个类别下所有图片的均值平均精确率，mAP@0.5∶0.95表示在不同交并比阈值（0.50 ~ 0.95，步长0.05）（0.50、0.55、0.60、0.65、0.70、0.75、0.80、0.85、0.90、0.95）上的均值平均精确率。Notes: mAP@0.5 indicates the average accuracy of all images in each category when the intersection and combination ratio is set to 0.5, mAP@0.5∶0.95 indicates the average accuracy of the mean value on the threshold of different intersection and combination ratios (0.50−0.95, step size 0.05) (0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90, 0.95).

下载: 导出CSV

表 4 不同模型识别结果对比

Table 4 Comparison of recognition results of different models

网络模型 Network model	精确率 Precision rate/%	召回率 Recall rate/%	mAP@0.5/%	检测时间 Detection time/ms	模型大小 Model size/MB
SSD	81.17	91.60	86.12	91.4	92.1
faster-RCNN	89.16	93.50	81.94	178.6	108.0
YOLOv3	96.89	93.59	96.30	32.7	117.0
YOLOv4	81.90	92.37	86.75	120.2	224.0
YOLOv5	97.14	96.12	98.06	22.1	13.7
改进的YOLOv5 Improved YOLOv5	97.48	96.53	98.22	10.3	14.1

下载: 导出CSV

参考文献(22)

[1]	谢永华. 数字图像处理技术在木材表面缺陷检测中的应用研究[D]. 哈尔滨: 东北林业大学, 2013. Xie Y H. The application and research of digital image processing on wood surface texture inspection[D]. Harbin: Northeast Forestry University, 2013.
[2]	Akhyar F, Novamizanti L, Putra T, et al. Lightning YOLOv4 for a surface defect detection system for sawn lumber[C]//Jay K C C, Klara N, Yong R, et al. IEEE 5th International Conference on Multimedia Information Processing and Retrieval (MIPR). New York: IEEE, 2022: 184−189.
[3]	肖雨晴, 杨慧敏, 王柯欣, 等. 卷积神经网络在木材缺陷检测应用中的研究进展[J]. 木材科学与技术, 2021, 35(3): 12−18. doi: 10.12326/j.2096-9694.2020088 Xiao Y Q, Yang H M, Wang K X, et al. Research progress of convolutional neural network in wood defect detection[J]. Wood Science and Technology, 2021, 35(3): 12−18. doi: 10.12326/j.2096-9694.2020088
[4]	Xia B, Luo H, Shi S. Improved faster R-CNN based surface defect detection algorithm for plates[J]. Computational Intelligence and Neuroscience, 2022, 2022: 11−22.
[5]	Mu H, Zhang M, Qi D, et al. Wood defects recognition based on fuzzy bp neural network[J]. International Journal of Smart Home, 2015, 9: 143−152.
[6]	Yang Y, Zhou X, Liu Y, et al. Wood defect detection based on depth extreme learning machine[J]. Applied Sciences, 2020, 10(21): 7488. doi: 10.3390/app10217488
[7]	Mu H, Zhang M, Qi D, et al. The application of RBF neural network in the wood defect detection[J]. International Journal of Hybrid Information Technology, 2015, 8(2): 41−50. doi: 10.14257/ijhit.2015.8.2.04
[8]	李超, 刘思佳, 曹军, 等. 基于PSO优选特征的实木板材缺陷的压缩感知分选方法[J]. 北京林业大学学报, 2015, 37(7): 117−122. Li C, Liu S J, Cao J, et al. The method of wood defectrecognition based on PSO feature selection and compressed sensing[J]. Journal of Beijing Forestry University, 2015, 37(7): 117−122.
[9]	Chen L C, Pardeshi M S, Lo W T, et al. Edge-glued wooden panel defect detection using deep learning[J]. Wood Science and Technology, 2022, 56(2): 477−507. doi: 10.1007/s00226-021-01316-3
[10]	缪伟志, 陆兆纳, 王俊龙, 等. 基于视觉的火灾检测研究[J]. 森林工程, 2022, 38(1): 86−92. doi: 10.3969/j.issn.1006-8023.2022.01.011 Miao W Z, Lu Z N, Wang J L, et al. Fire detection research based on vision[J]. Forest Engineering, 2022, 38(1): 86−92. doi: 10.3969/j.issn.1006-8023.2022.01.011
[11]	高明宇, 倪海明, 张博洋, 等. 一种基于GoogLeNet卷积神经网络的木节缺陷识别方法[J]. 森林工程, 2021, 37(4): 66−70. doi: 10.3969/j.issn.1006-8023.2021.04.009 Gao M Y, Ni H M, Zhang B Y, et al. A method for recognizing wood knots defects based on GoogLeNet convolutional neural network[J]. Forest Engineering, 2021, 37(4): 66−70. doi: 10.3969/j.issn.1006-8023.2021.04.009
[12]	He T, Liu Y, Yu Y, et al. Application of deep convolutional neural network on feature extraction and detection of wood defects[J]. Measurement, 2020, 152: 107357. doi: 10.1016/j.measurement.2019.107357
[13]	Urbonas A, Raudonis V, Maskeliūnas R, et al. Automated identification of wood veneer surface defects using faster region-based convolutional neural network with data augmentation and transfer learning[J]. Applied Sciences, 2019, 9(22): 4898. doi: 10.3390/app9224898
[14]	Shi J, Li Z, Zhu T, et al. Defect detection of industry wood veneer based on NAS and multi-channel mask R-CNN[J]. Sensors, 2020, 20(16): 4398. doi: 10.3390/s20164398
[15]	Sun P A. Wood quality defect detection based on deep learning and multicriteria framework[J]. Mathematical Problems in Engineering, 2022, 2022: 9−16.
[16]	Wang L, Yan W Q. Tree leaves detection based on deep learning[C]//Minh N, Wei Q Y, Harvey H. International Symposium on Geometry and Vision. Auckland: Auckland University of Technology (AUT), 2021: 26−38.
[17]	赵睿, 刘辉, 刘沛霖, 等. 基于改进YOLOv5s的安全帽检测算法[J/OL]. 北京航空航天大学学报, 2023[2023−01−12]. https://doi.org/10.13700/j.bh.1001-5965.2021.0595. Zhao R, Liu H, Liu P L, et al. Research on safety helmet detection algorithm based on improved YOLOv5s[J/OL]. Journal of Beijing University of Aeronautics and Astronautics, 2023[2023−01−12]. https://doi.org/10.13700/j.bh.1001-5965.2021.0595.
[18]	李彦甫, 范习健, 杨绪兵, 等. 基于自注意力卷积网络的遥感图像分类[J]. 北京林业大学学报, 2021, 43(10): 81−88. doi: 10.12171/j.1000-1522.20210196 Li Y F, Fan X J, Yang X B, et al. Remote sensing image classification framework based on self-attention convolutional neural network[J]. Journal of Beijing Forestry University, 2021, 43(10): 81−88. doi: 10.12171/j.1000-1522.20210196
[19]	邹梓吟, 盖绍彦, 达飞鹏, 等. 基于注意力机制的遮挡行人检测算法[J]. 光学学报, 2021, 41(15): 157−165. Zou Z Y, Gai S Y, Da F P, et al. Occluded pedestrian detection algorithm basedon attention mechanism[J]. Acta Optica Sinica, 2021, 41(15): 157−165.
[20]	Gao M, Wang F, Liu J, et al. Estimation of the convolutional neural network with attention mechanism and transfer learning on wood knot defect classification[J]. Journal of Applied Physics, 2022, 131(23): 233101. doi: 10.1063/5.0087060
[21]	Du J. Understanding of object detection based on CNN family and YOLO[J] . Journal of Physics Conference, 2018, 1004: 012029.
[22]	Gao M, Song P, Wang F, et al. A novel deep convolutional neural network based on ResNet-18 and transfer learning for detection of wood knot defects[J]. Journal of Sensors, 2021, 2021: 16−27.

施引文献(44)

期刊类型引用(24)

1.	张秀芸，伍文慧，梁英梅. 落叶松枯梢病在中国的适生性. 生态学报. 2024(07): 3027-3037 . 百度学术
2.	葛婉婷，刘莹，赵智佳，张珅，李洁，杨桂娟，曲冠证，王军辉，麻文俊. 不同气候情景下黄心梓木在我国的潜在适生区预测. 林业科学. 2024(11): 63-74 . 百度学术
3.	汤思琦，武扬，梁定东，郭恺. 未来气候变化下栎树猝死病菌在中国的适生性分析. 生态学报. 2023(01): 388-397 . 百度学术
4.	刘璐璐，赵亮，蔺诗颖，冯建龙. 基于MaxEnt和GARP的阿蒙森海域南极磷虾(EUPHAUSIA SUPERBA)的分布区预测. 海洋与湖沼. 2023(02): 399-411 . 百度学术
5.	唐雨薇，张晓龙，张雪云，吕佩锋，罗乐. 基于GIS与AHP分析法的单叶蔷薇生态适宜性评价. 绿色科技. 2023(13): 205-208+213 . 百度学术
6.	王广平，李成，王书砚，刘超，杨君珑. 宁夏罗山青海云杉林空间分布特征研究. 农业科学研究. 2023(03): 10-15+23 . 百度学术
7.	张惠惠，孟祥霄，林余霖，陈士林，黄林芳. 基于GMPGIS系统和MaxEnt模型预测人参全球潜在生长区域. 中国中药杂志. 2023(18): 4959-4966 . 百度学术
8.	李盼畔，何旭诺，吴海荣，陈萍，刘明航，武目涛，王亚锋. 多年生豚草在中国的潜在分布预测. 植物检疫. 2022(04): 57-62 . 百度学术
9.	李盼畔，何旭诺，左然玲，吕文刚，吴海荣. 4种蒺藜草属杂草在中国的潜在适生性预测. 杂草学报. 2022(02): 15-23 . 百度学术
10.	林姗，陆兴利，王茹琳，李庆，王明田，郭翔，文刚. RCP8.5情景下气候变化对四川省猕猴桃溃疡病病菌地理分布的影响. 江苏农业科学. 2020(03): 124-129 . 百度学术
11.	王华辰，朱弘，李涌福，伊贤贵，李蒙，南程慧，王贤荣. 中国特有植物雪落樱桃潜在分布及其生态特征. 热带亚热带植物学报. 2020(02): 136-144 . 百度学术
12.	赵金鹏，王茹琳，刘原，陆兴利，王庆，郭翔，文刚，李庆. RCP4.5情景下四川省猕猴桃溃疡病菌适生性分析. 沙漠与绿洲气象. 2020(02): 137-143 . 百度学术
13.	段义忠，王佳豪，王驰，王海涛，杜忠毓. 未来气候变化下西北干旱区4种扁桃亚属植物潜在适生区分析. 生态学杂志. 2020(07): 2193-2204 . 百度学术
14.	陈爱莉，赵志华，龚伟，孔芬，张克亮. 气候变化背景下紫楠在中国的适宜分布区模拟. 热带亚热带植物学报. 2020(05): 435-444 . 百度学术
15.	文雪梅，艾科拜尔·木哈塔尔，木巴来克·阿布都许科尔，阿不都拉·阿巴斯. 基于MaxEnt模型的新疆微孢衣属地衣生境适宜性评价. 武汉大学学报(理学版). 2019(01): 77-84 . 百度学术
16.	常红，刘彤，王大伟，纪孝儒. 气候变化下中国西北干旱区梭梭(Haloxylon ammodendron)潜在分布. 中国沙漠. 2019(01): 110-118 . 百度学术
17.	吕汝丹，何健，刘慧杰，姚敏，程瑾，谢磊. 羽叶铁线莲的分布区与生态位模型分析. 北京林业大学学报. 2019(02): 70-79 . 本站查看
18.	陆兴利，罗伟，李庆，林姗，王茹琳，游超，郭翔，王明田. RCP2.6情景下四川省猕猴桃溃疡病菌潜在分布预测. 湖北农业科学. 2019(18): 49-54 . 百度学术
19.	王蕾，罗磊，刘平，侯晓臣，邱琴，高亚琪，李曦光. 基于MaxEnt模型分析新疆特色林果区春尺蠖发生风险. 新疆农业科学. 2019(09): 1691-1700 . 百度学术
20.	王茹琳，郭翔，李庆，王明田，游超. 四川省猕猴桃溃疡病潜在分布预测及适生区域划分. 应用生态学报. 2019(12): 4222-4230 . 百度学术
21.	赵健，李志鹏，张华纬，陈宏，翁启勇. 基于MaxEnt模型和GIS技术的烟粉虱适生区预测. 植物保护学报. 2019(06): 1292-1300 . 百度学术
22.	王奕晨，郑鹏，潘文斌. 运用GARP生态位模型预测福寿螺在中国的潜在适生区. 福建农林大学学报(自然科学版). 2018(01): 21-25 . 百度学术
23.	邱靖，朱弘，陈昕，汤庚国. 基于DIVA-GIS的水榆花楸适生区模拟及生态特征. 北京林业大学学报. 2018(09): 25-32 . 本站查看
24.	王野，陈磊，白云，张俊娥，刘红霞，田呈明. 云杉矮槲寄生遗传多样性的ISSR分析. 西北植物学报. 2017(11): 2153-2162 . 百度学术