UAV remote sensing image extraction of single tree crown of Chinese fir based on Mask R-CNN and transfer learning

Xie Yunhong; Sun Zhao; Ding Zhidan; Luo Mi; Li Yun; Sun Yujun

doi:10.12171/j.1000-1522.20210343

Journal of Beijing Forestry University > 2024 > 46(3): 153-166. > DOI: 10.12171/j.1000-1522.20210343

Xie Yunhong, Sun Zhao, Ding Zhidan, Luo Mi, Li Yun, Sun Yujun. UAV remote sensing image extraction of single tree crown of Chinese fir based on Mask R-CNN and transfer learning[J]. Journal of Beijing Forestry University, 2024, 46(3): 153-166. DOI: 10.12171/j.1000-1522.20210343

Citation:

PDF (5254 KB)

UAV remote sensing image extraction of single tree crown of Chinese fir based on Mask R-CNN and transfer learning

National Forestry and Grassland Administration Key Laboratory of Forest Resources & Environmental Management, Beijing Forestry University, Beijing 100083, China

More Information

Received Date: September 02, 2021
Revised Date: October 19, 2023
Accepted Date: January 02, 2024
Available Online: January 07, 2024

Graphical Abstract

Abstract

Abstract

Objective
The UAV image automatically extracts canopy for precise information.
Method
In this study, single tree crown extraction method from UAV images based on Mask R-CNN and transfer learning was proposed. Firstly, the optimized Mask R-CNN instance segmentation model based on faster R-CNN was selected, and the optimal feature extraction network was chosen between ResNet50 and ResNet101. Secondly, transfer learning was introduced to train with Mask R-CNN together, combined with the guiding role of transfer learning to reduce training time and improve training accuracy.
Result
The results showed that the overall accuracy of the Mask R-CNN model was 93.59%, the user accuracy, F1 score and mean average precision were 65.46%, 76.05% and 0.31, respectively. After adding transfer learning, the user accuracy of Mask R-CNN model was increased by 29.53%, the F1 score was increased by 19.63%, and the average accuracy was increased by 0.21. In the Mask R-CNN model with ResNet50 and ResNet101 as feature extraction networks, the average values of overall accuracy, user accuracy, F1 score and mean average precision of the ResNet50 + Mask R-CNN model were 96.94%, 95.57%, 96.17% and 0.54, respectively. The average values of overall accuracy, user accuracy, F1 score and mean average precision of the ResNet101 + Mask R-CNN model were 96.20%, 94.41%, 95.19% and 0.49, respectively. The R² of ResNet50 + Mask R-CNN model loaded with transfer learning in predicting east-west crown width, north-south crown width, crown area and quadrat canopy density were 0.87, 0.84, 0.93 and 0.83, respectively.
Conclusion
The method based on Mask R-CNN and transfer learning proposed in this study obtains more accurate results of tree crown parameters, which provides a fast and efficient solution for tree resource assessment in UAV images.
- UAV,
- remote sensing image,
- deep learning,
- Mask R-CNN,
- transfer learning,
- crown extraction

FullText(HTML)

References (33)

References

[1]	覃先林, 李增元, 易浩若. 高空间分辨率卫星遥感影像树冠信息提取方法研究[J]. 遥感技术与应用, 2005(2): 228−232. Qin X L, Li Z Y, Yi H R. Extraction method of tree crown using high-resolution satellite image[J]. Remote Sensing Technology and Application, 2005(2): 228−232.
[2]	孙振峰, 张晓丽, 李霓雯. 机载与星载高分遥感影像单木树冠分割方法和适宜性对比[J]. 北京林业大学学报, 2019, 41(11): 66−75. Sun Z F, Zhang X L, Li N W. Comparison of individual tree crown extraction method and suitability of airborne and spaceborne high-resolution remote sensing images[J]. Journal of Beijing Forestry University, 2019, 41(11): 66−75.
[3]	Santos A A, Marcato J J, Araújo M S, et al. Assessment of CNN-based methods for individual tree detection on images captured by RGB cameras attached to UAVs[J]. Sensors, 2019, 19(16): 3595. doi: 10.3390/s19163595
[4]	于东海, 冯仲科. 基于无人机倾斜航空影像的树冠体积测算方法[J]. 农业工程学报, 2019, 35(1): 90−97. doi: 10.11975/j.issn.1002-6819.2019.01.011 Yu D H, Feng Z K. Tree crown volume measurement method based on oblique aerial images of UAV[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2019, 35(1): 90−97. doi: 10.11975/j.issn.1002-6819.2019.01.011
[5]	汪小钦, 王苗苗, 王绍强, 等. 基于可见光波段无人机遥感的植被信息提取[J]. 农业工程学报, 2015, 31(5): 152−159. doi: 10.3969/j.issn.1002-6819.2015.05.022 Wang X Q, Wang M M, Wang S Q, et al. Extraction of vegetation information from visible unmanned aerial vehicle images[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2015, 31(5): 152−159. doi: 10.3969/j.issn.1002-6819.2015.05.022
[6]	Huang H, Li X, Chen C. Individual tree crown detection and delineation from very-high-resolution UAV images based on bias field and marker-controlled watershed segmentation algorithms[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(7): 2253−2262. doi: 10.1109/JSTARS.2018.2830410
[7]	李丹, 张俊杰, 赵梦溪. 基于FCM和分水岭算法的无人机影像中林分因子提取[J]. 林业科学, 2019, 55(5): 180−187. doi: 10.11707/j.1001-7488.20190520 Li D, Zhang J J, Zhao M X. Extraction of stand factors in UAV image based on FCM and watershed algorithm[J]. Scientia Silvae Sinicae, 2019, 55(5): 180−187. doi: 10.11707/j.1001-7488.20190520
[8]	张凝, 冯跃文, 张晓丽, 等. 结合航空影像纹理和光谱特征的单木冠幅提取[J]. 北京林业大学学报, 2015, 37(3): 13−19. Zhang N, Feng Y W, Zhang X L, et al. Extracting individual tree crown by combining spectral and texture features from aerial images[J]. Journal of Beijing Forestry University, 2015, 37(3): 13−19.
[9]	孙钊, 潘磊, 孙玉军. 基于无人机影像的高郁闭度杉木纯林树冠参数提取[J]. 北京林业大学学报, 2020, 42(10): 20−26. Sun Z, Pan L, Sun Y J. Extraction of tree crown parameters from high-density pure Chinese fir plantations based on UAV images[J]. Journal of Beijing Forestry University, 2020, 42(10): 20−26.
[10]	徐永胜. 基于无人机影像的林木冠幅提取与树种识别研究[D]. 哈尔滨: 东北林业大学, 2021. Xu Y S. Research on tree crown extraction and tree species recognition bases on UAV images[D]. Harbin: Northeast Forestry University, 2021.
[11]	Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation[C]//International Conference on Medical image computing and computer-assisted intervention. Munich: Technische Universität München, 2015.
[12]	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. Boston: Computer Vision, 2015.
[13]	Li W J, Fu H h, Yu L, et al. Deep learning based oil palm tree detection and counting for high-resolution remote sensing images[J]. Remote Sensing, 2017, 9(1): 22.
[14]	田萱, 王亮, 丁琪. 基于深度学习的图像语义分割方法综述[J]. 软件学报, 2019, 30(2): 440−468. Tian X, Wang L, Ding Q. Review of image semantic segmentation based on deep learning[J]. Journal of Software, 2019, 30(2): 440−468.
[15]	He K, Gkioxari G, Dollár P, et al. Mask R-CNN[C]//Proceedings of the IEEE international conference on computer vision. Venice: Computer Vision, 2017.
[16]	Li Q, Arnab A, Torr P H S. Weakly-and semi-supervised panoptic segmentation[C]//Proceedings of the European conference on computer vision (ECCV). Munich: Computer Vision, 2018.
[17]	高云, 郭继亮, 黎煊, 等. 基于深度学习的群猪图像实例分割方法[J]. 农业机械学报, 2019, 50(4): 179−187. Gao Y, Guo J L, Li X, et al. Instance-level segmentation method for group pig images based on deep learning[J]. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50(4): 179−187.
[18]	Dai J, He K, Sun J. Instance-aware semantic segmentation via multi-task network cascades[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. Las Vegas: Computer Vision, 2016.
[19]	Bai M, Urtasun R. Deep watershed transform for instance segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Hawaii: Computer Vision, 2017.
[20]	Tan C, Sun F, Kong T, et al. A survey on deep transfer learning[C]//International conference on artificial neural networks. Springer: European Neural Network Society, 2018.
[21]	Gu S, Ding L. A complex-valued vgg network based deep learning algorithm for image recognition[C]//2018 Ninth International Conference on Intelligent Control and Information Processing (ICICIP). Wanzhou: International Conference on Intelligent Control and Information Processing (ICICIP), 2018.
[22]	李博, 江朝晖, 谢军, 等. 基于迁移学习的园艺作物叶部病害识别及应用[J]. 中国农学通报, 2021, 37(7): 138−143. Li B, Jiang C H, Xie J, et al. Leaf disease recognition of horticultural crops based on transfer learning and its application[J]. Chinese Agricultural Science Bulletin, 2021, 37(7): 138−143.
[23]	查志华, 周文静, 吴杰. 基于迁移学习Faster R-CNN模型田间红提葡萄果穗的识别[J]. 石河子大学学报(自然科学版), 2021, 39(1): 26−31. Zha Z H, Zhou W J, Wu J. Identification of red globe grape cluster in grapery with Faster R-CNN model based on transfer learning[J]. Journal of Shihezi University (Natural Science Edition), 2021, 39(1): 26−31.
[24]	Ocer N E, Kaplan G, Erdem F, et al. Tree extraction from multi-scale UAV images using Mask R-CNN with FPN[J]. Remote Sensing Letters, 2020, 11(9): 847−856. doi: 10.1080/2150704X.2020.1784491
[25]	Safonova A, Guirado E, Maglinets Y, et al. Olive tree biovolume from uav multi-resolution image segmentation with mask r-cnn[J]. Sensors, 2021, 21(5): 1617. doi: 10.3390/s21051617
[26]	Hao Z, Lin L, Post C J, et al. Automated tree-crown and height detection in a young forest plantation using mask region-based convolutional neural network (Mask R-CNN)[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 178: 112−123. doi: 10.1016/j.isprsjprs.2021.06.003
[27]	Ren S, He K, Girshick R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. Advances in Neural Information Processing Systems, 2015, 28: 91−99.
[28]	Sun W, Wang R. Fully convolutional networks for semantic segmentation of very high resolution remotely sensed images combined with DSM[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(3): 474−478.
[29]	Sharma N, Jain V, Mishra A. An analysis of convolutional neural networks for image classification[J]. Procedia Computer Science, 2018, 132: 377−384. doi: 10.1016/j.procs.2018.05.198
[30]	Torrey L, Shavlik J. Transfer learning[M]//Handbook of research on machine learning applications and trends: algorithms, methods, and techniques. Hershey: IGI Global, 2010.
[31]	梁泽, 黄姣, 韦飞黎, 等. 基于夜光遥感影像与百度POI数据的中国城市群空间范围识别方法[J]. 地理研究, 2020, 39(1): 92−102. Liang Z, Huang J, Wei F L, et al. Identifying the spatial range of urban agglomerations in China based on night light remote sensing and POI data[J]. Geographical Research, 2020, 39(1): 92−102.
[32]	伍艳莲, 赵力, 姜海燕, 等. 基于改进均值漂移算法的绿色作物图像分割方法[J]. 农业工程学报, 2014, 30(24): 161−167. Wu Y L, Zhao L, Jiang H Y, et al. Image segmentation method for green crops using improved mean shift[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2014, 30(24): 161−167.
[33]	Özcan A H, Hisar D, Sayar Y, et al. Tree crown detection and delineation in satellite images using probabilistic voting[J]. Remote Sensing Letters, 2017, 8(8): 761−770. doi: 10.1080/2150704X.2017.1322733

[1]	Zhang Minghui, Yin Yunzhou, Wang Ke, Wang Shuli. Effects of spatial structure characteristics of Fraxinus mandshurica plantation on soil nutrient content[J]. Journal of Beijing Forestry University, 2023, 45(9): 73-82. DOI: 10.12171/j.1000-1522.20220476
[2]	Hui Gangying, Zhao Zhonghua, Hu Yanbo, Zhang Ganggang, Zhang Gongqiao, Cheng Shiping, Lu Yanlei. Research on the measurement method of forest spatial structure diversity based on 4 neighborhood tree relationship[J]. Journal of Beijing Forestry University, 2023, 45(7): 18-26. DOI: 10.12171/j.1000-1522.20220282
[3]	Wang Lina, Wu Junwen, Dong Qiong, Shi Zhuogong, Hu Haocheng, Wu Danzi, Li Luping. Effects of tending and thinning on non-structural carbon and stoichiometric characteristics of Pinus yunnanensis[J]. Journal of Beijing Forestry University, 2021, 43(8): 70-82. DOI: 10.12171/j.1000-1522.20210115
[4]	Hu Xuefan, Zhang Huiru, Zhou Chaofan, Zhang Xiaohong. Effects of different thinning patterns on the spatial structure of Quercus mongolica secondary forests[J]. Journal of Beijing Forestry University, 2019, 41(5): 137-147. DOI: 10.13332/j.1000-1522.20190037
[5]	LI Jian, PENG Peng, HE Huai-jiang, TAN Ling-zhao, ZHANG Xin-na, WU Xiang-ju, LIU Zhao-gang. Effects of thinning intensity on spatial structure of multi-species temperate forest at Jiaohe in Jilin Province, northeastern China[J]. Journal of Beijing Forestry University, 2017, 39(9): 48-57. DOI: 10.13332/j.1000-1522.20170220
[6]	LI Ji-ping, FENG Yao, ZHAO Chun-yan, ZHANG Cai-cai. Quantitative analysis of stand spatial structure of Cunninghamia lanceolata non-commercial forest based on Voronoi diagram.[J]. Journal of Beijing Forestry University, 2014, 36(4): 1-7. DOI: 10.13332/j.cnki.jbfu.2014.04.005
[7]	WANG Ping, JIA Li-ming, WEI Song-po, WANG Qi-feng. Analysis of stand spatial structure of Platycladus orientalis recreational forest based on Voronoi diagram method[J]. Journal of Beijing Forestry University, 2013, 35(2): 39-44.
[8]	DONG Ling-bo, LIU Zhao-gang, MA Yan, NI Bao-long, LI Yuan. A new composite index of stand spatial structure for natural forest.[J]. Journal of Beijing Forestry University, 2013, 35(1): 16-22.
[9]	DUAN Chang-sheng, WANG Jun-hui, MA Jian-wei, YUAN Shi-yun, DU Yan-chang. Evaluation of Quercus aliena var. acuteserrata forest at the western segment of Qinling Mountain，northwestern China[J]. Journal of Beijing Forestry University, 2009, 31(5): 61-66.
[10]	LIU Yan, YU Xin-xiao, YUE Yong-jie, GAN Jing, WANG Xiao-ping, LI Jin-hai. Spatial structure of Robinia pseudoacacia plantation in Miyun Reservoir Watershed of Beijing.[J]. Journal of Beijing Forestry University, 2009, 31(5): 25-28.

Cited By

Cited by

Periodical cited type(8)

1.	张慧，燕怡帆，朱雅，陈玉婷，王菁华，崔志鹏，杨迪，任学敏. 林分密度对伏牛山南麓山茱萸人工林林下草本植物多样性和土壤性质的影响. 西南林业大学学报(自然科学). 2025(01): 96-105 .
2.	赵金同，马俊. 刺槐扦插育苗技术与精细抚育要点. 现代园艺. 2024(08): 49-51 .
3.	何欢，康必均，尹婧，李菲，彭栋，李桂静，查同刚. 不同营林措施对川东华蓥山杉木林土壤团聚体稳定性及细根分布的影响. 土壤通报. 2024(02): 351-359 .
4.	史小鹏，苟贺然，何淑勤，刘柏廷，冉兰芳，杨琪琳，扎西拉姆，陈雨馨，骆紫藤. 成都市温江区两种绿地土壤抗蚀抗冲性及其影响因素. 水土保持通报. 2024(04): 117-125 .
5.	刘忆南，申振宏，都都，张知然，林勇明. 蒋家沟泥石流堆积扇不同植被类型区土壤抗蚀性评价. 应用与环境生物学报. 2024(05): 886-893 .
6.	王依瑞，王彦辉，段文标，李平平，于澎涛，甄理，李志鑫，尚会军. 黄土高原刺槐人工林郁闭度对林下植物多样性特征的影响. 应用生态学报. 2023(02): 305-314 .
7.	赵云鹤，钟鹏，高晗，付玉. 土地利用类型对典型黑土团聚体稳定性和抗蚀性的影响. 东北林业大学学报. 2023(09): 112-119 .
8.	胡亚伟，施政乐，刘畅，徐勤涛，张建军. 晋西黄土区刺槐林密度对林下植物多样性及土壤理化性质的影响. 生态学杂志. 2023(09): 2072-2080 .

Other cited types(9)

Get Citation

PDF

XML

Article views (667) PDF downloads (93) Cited by(17)

Turn off MathJax

Article Contents

Abstract

References

UAV remote sensing image extraction of single tree crown of Chinese fir based on Mask R-CNN and transfer learning