小样本条件下基于数据扩充和ResNeSt的雪豹识别

张毓; 高雅月; 常峰源; 谢将剑; 张军国

doi:10.12171/j.1000-1522.20210185

小样本条件下基于数据扩充和ResNeSt的雪豹识别

doi: 10.12171/j.1000-1522.20210185

张毓^1,,
高雅月¹,
常峰源²,
谢将剑^2, ,,
张军国^2, ,

1.
青海省祁连山自然保护区管理局，青海省北藏族自治州 810400
2.
北京林业大学工学院，北京 100083

基金项目: 北京市自然科学基金项目面上项目（6192019），北京市自然科学基金项目青年项目（6214040）

详细信息

作者简介:
张毓，高级工程师。主要研究方向：野生动物保护管理。Email：369927234@foxmail.com　地址：810400 青海省海北州祁连县八宝镇青海省祁连山自然保护区管理局

责任作者:
谢将剑，博士，副教授。主要研究方向：林业信息监测、信号处理、模式识别。Email：shyneforce@bjfu.edu.cn　地址：100083 北京市海淀区清华东路35号北京林业大学工学院

张军国，博士，教授。主要研究方向：物联网与无线传感器网络、图像处理以及深度学习。Email：zhangjunguo@bjfu.edu.cn　地址：100083 北京市海淀区清华东路35号北京林业大学工学院

中图分类号: TP391.41
计量
- 文章访问数: 70
- HTML全文浏览量: 48
- PDF下载量: 12
- 被引次数: 0
出版历程
- 收稿日期: 2021-05-14
- 修回日期: 2021-06-09
- 网络出版日期: 2021-08-04

Panthera unica recognition based on data expansion and ResNeSt with few samples

Zhang Yu^1
,,
Gao Yue¹,
Chang Fengyuan²,
Xie Jiangjian^{2
, ,},
Zhang Junguo^{2
, ,}

1.
Qinghai Qilian Mountain Nature Reserve Administration, North Tibetan Autonomous Prefecture of Qinghai Province, 810400, China
2.
Beijing Forestry University School of Technology, Beijing, 100083, China

摘要

摘要: 目的红外触发相机采集的雪豹监测图像质量参差不齐，且数量有限，为了提升小样本下雪豹的识别准确率，本研究提出一种雪豹监测图像自动识别方法。方法该方法基于具备注意力机制的ResNeSt50模型，使用祁连山国家公园的雪豹监测图像作为原始数据集，红外触发相机拍摄的非雪豹陆生野生动物图像作为扩充负样本，网络雪豹图像作为扩充正样本，生成3种数据集并依次进行对比实验，选择合适的扩充方式引导模型逐步关注到雪豹个体关键特征，使用梯度类激活热力图可视化进一步验证数据扩充后的有效性。结果使用原始数据集+扩充负样本+扩充正样本训练的模型识别效果最好，热力图可视化显示模型正确关注到雪豹个体花纹与斑点特征，对比基于Vgg16和ResNet50的识别模型，ResNeSt50的识别效果最好，测试集识别准确率达到97.70%，精确率97.26%，召回率97.59%。结论采用本研究提出的原始数据集+扩充负样本+扩充正样本数据扩充方法训练的模型，可以区分背景与前景，且对雪豹本身特征具有较强的判别能力，泛化能力最好。
- 雪豹 /
- 监测图像 /
- 小样本 /
- 数据扩充 /
- 卷积神经网络
Abstract: Objective The quality of snow leopard monitoring images collected by infrared trigger cameras is uneven and the number is limited. An automatic recognition method of snow leopard monitoring images based on deep learning data expansion was proposed to improve the recognition accuracy of the snow leopard under limited samples. Method Improve the ResNeSt50 model with attention mechanism, the snow leopard monitoring images of Qilian Mountain National Park were used as the original data set, the non-snow leopard terrestrial wildlife images taken by the infrared trigger camera were used as the extended negative sample, and the network snow leopard images were used as the extended positive sample. Comparative experiments were conducted in turn based on the above three data sets. The model was gradually guided to focus on the key characteristics of individual snow leopards by choosing an appropriate expansion method, and the effectiveness of the data expansion was verified by using Gradient-weighted Class Activation Map. Result Results showed that the model trained with the original data set+expanded negative samples+expanded positive samples had the best recognition effect. The Grad-CAM showed that the model correctly focused on the individual pattern and spot characteristics of the snow leopard. Compared with the recognition model based on Vgg16 and ResNet50, ResNeSt50 achieved the best recognition effect, the test set recognition accuracy rate reached 97.70%, the precision rate reached 97.26%, and the recall rate reached 97.59%. Conclusion The model trained by the original data set+extended negative sample+extended positive sample data expansion method proposed in this paper can distinguish the background from the foreground, and has a strong ability to discriminate the characteristics of the snow leopard itself, and the generalization ability is the best.
- Panthera unica /
- monitoring image /
- few sample /
- data expansion /
- convolutional neural network

HTML全文

图 1 雪豹监测图像的常见情形

Figure 1. Snow leopard monitoring images of common situations

下载: 全尺寸图片幻灯片

图 2 ResNeSt的基本单元

(h,w,c)为输入特征图的（高，宽，通道数）；Cardinal K为第K个分支；Split S为第S个子组；C为中间卷积层的特征图通道数；Concatenate代表通道拼接；Split-Attention表示分割注意力模块。(h,w,c), The (height, width and number of channels) of the input feature graph; Cardinal K, The Kth cardinal group; Split S, The Rth split; C, The number of feature graph channels in the middle convolutional layer; Concatenate, the concatenate between channels; Split-Attention, split attention block.

Figure 2. The basic unit of ResNeSt

下载: 全尺寸图片幻灯片

图 3 Split-Attention的具体结构

$ {U}_{j} $为Split-Attention模块中第j个输入特征；$ {\widehat{U}}^{k} $为第k个分支的组合特征；Global pooling为全局池化层；Dense C/K为全连接层；BN为批量归一化层；ReLU为激活函数；S-Softmax为分类器；H, W, C/K表示中间特征层的高、宽、通道数。$ {U}_{j} $, The j th input feature in Split-Attention block; $ {\widehat{U}}^{k} $, The combinatorial feature of the kth cardinal group; Global pooling, Global pooling layer; Dense C/K, fully connected layer; BN, Batch Normalization layer; ReLU, activation function; R-Softmax, classifier; H, W, C/K, the height, width and number of channels of middle feature layer.

Figure 3. The structure of the Split-Attention

下载: 全尺寸图片幻灯片

图 4 识别准确率随迭代次数的变化

Figure 4. The variation of recognition accuracy with the number of iterations

下载: 全尺寸图片幻灯片

图 5 模型1对测试集1样本的识别热力图

Figure 5. Heat maps of test set 1 samples recognized by model 1

下载: 全尺寸图片幻灯片

图 6 模型1对测试集2中负样本的识别热力图

Figure 6. Heat maps of negative samples in test set 2 recognized by model 1

下载: 全尺寸图片幻灯片

图 7 训练集2、测试集2、测试集3的识别准确率曲线

Figure 7. Recognition accuracy curve of training set 2, test set 2 and test set 3

下载: 全尺寸图片幻灯片

图 8 模型2对测试集2上的识别热力图

Figure 8. Heat maps of test set 2 samples recognized by model 2

下载: 全尺寸图片幻灯片

图 9 模型2对测试集3中正样本的识别热力图

Figure 9. Heat maps of positive samples in test set 3 recognized by model 2

下载: 全尺寸图片幻灯片

图 10 训练集3、测试集3的识别准确率曲线

Figure 10. Recognition accuracy curve of training set 3 and test set 3

下载: 全尺寸图片幻灯片

图 11 模型3对测试集3样本的识别热力图

Figure 11. Heat maps of test set 3 samples recognized by model 3

下载: 全尺寸图片幻灯片

图 12 不同模型在3个数据集上训练的识别结果

1_1，模型1在测试集1上的识别结果；2_2，模型2在测试集2上的识别结果；3_3，模型3在测试集3上的识别结果；3_1，模型3在测试集1上的识别结果。1_1, Recognition results of test set 1 by model 1; 2_2, Recognition results of test set 2 by model 2; 3_3, Recognition results of test set 3 by model 3; 3_1, Recognition results of test set 1 by model 3.

Figure 12. Recognition results of three datasets by different models

下载: 全尺寸图片幻灯片

图 13 模型1得到的特征散点图

Figure 13. Characteristic scatter diagram obtained by model 1

下载: 全尺寸图片幻灯片

图 14 模型2得到的特征散点图

Figure 14. Characteristic scatter diagram obtained by model 2

下载: 全尺寸图片幻灯片

图 15 模型3得到的特征散点图

Figure 15. Characteristic scatter diagram obtained by model 3

下载: 全尺寸图片幻灯片

表 1 数据集分布情况

Table 1. Dataset distribution

数据集 Datasets	正样本数 Number of the positive sample	负样本数 Number of the negative sample
原始数据集 Original dataset	1324	1110
扩充数据集 Extended dataset	310	524
总数据集 Whole dataset	1634	1634

下载: 导出CSV

表 2 训练使用的扩充数据集

Table 2. Extended datasets for training use

数据集来源 Dataset sources	数据集 Dataset	训练集 Train set	测试集 Test set
原始数据集 The original dataset	数据集1 Dataset 1	训练集1 Train set 1	测试集1 Test set 1
原始数据集+扩充负样本 The original dataset + extended negative sample	数据集2 Dataset 2	训练集2 Train set 2	测试集2 Test set 2
原始数据集+扩充负样本+扩充正样本 The original dataset + extended negative sample + extended positive sample	数据集3 Dataset 3	训练集3 Train set 3	测试集3 Test set 3

下载: 导出CSV

表 3 模型1在测试集1、2上的识别结果

Table 3. Recognition results of test sets 1 and 2 by model 1

测试集 Test set	准确率 Accuracy rate/%	精确率 Precision rate/%	召回率 Recall rate/%
测试集1 Test set 1	96.30	94.14	98.25
测试集2 Test set 2	88.85	78.39	98.25

下载: 导出CSV

表 4 模型2对测试集2、3上的识别结果

Table 4. Recognition results of test sets 2 and 3 by model 2

测试集 Test set	准确率 Accuracy rate/%	精确率 Precision rate/%	召回率 Recall rate/%
测试集2 Test set 2	97.29	94.56	98.68
测试集3 Test set 3	94.03	95.32	91.06

下载: 导出CSV

表 5 模型3在测试集3上的识别结果

Table 5. Recognition results of test sets 3 by model 3

测试集 Test set	准确率 Accuracy rate/%	精确率 Precision rate/%	召回率 Recall rate/%
测试集3 Test set 3	97.70	97.26	97.59

下载: 导出CSV

参考文献(24)

[1]	洪洋, 张晋东, 王玉君. 雪豹生态与保护研究现状探讨[J]. 四川动物, 2020, 39(6):711−720. doi: 10.11984/j.issn.1000-7083.20190438 Hong Y, Zhang J D, Wang Y J. Progress in the ecology and conservation research on Panthera unica[J]. Sichuan Journal of Zoology, 2020, 39(6): 711−720. doi: 10.11984/j.issn.1000-7083.20190438
[2]	Bracciale L, Catini A, Gentile G, et al. Delay tolerant wireless sensor network for animal monitoring: the pink iguana case[C]//Alessandro D G. Proceedings of international conference on applications in electronics pervading industry, environment and society. Rome, Italy, 2016: 18−26.
[3]	徐峰. 新疆雪豹研究简史[J]. 人与生物圈, 2020(增刊 1):77−79. Xu F. A brief history of snow leopard in Xinjiang[J]. Man & Biosphere, 2020(Suppl. 1): 77−79.
[4]	马鸣, 徐峰, 吴逸群, 等. 新疆雪豹种群密度监测方法探讨[J]. 生态与农村环境学报, 2011, 27(1):79−83. doi: 10.3969/j.issn.1673-4831.2011.01.016 Ma M, Xu F, Wu Y Q, et al. Monitoring of population density of snow leopard in xinjiang[J]. Journal of Ecology and Rural Environment, 2011, 27(1): 79−83. doi: 10.3969/j.issn.1673-4831.2011.01.016
[5]	汪六三, 黄子良, 王儒敬. 基于近红外光谱和机器学习的大豆种皮裂纹识别研究[J]. 农业机械学报, 2021, 52(6):361−368. doi: 10.6041/j.issn.1000-1298.2021.06.038 Wang L S, Huang Z L, Wang R J. Identification of soybean seed coat crack using near infrared spectroscopy and machine learning[J]. Transactions of the Chinese Society for Agricultural Machinery, 2021, 52(6): 361−368. doi: 10.6041/j.issn.1000-1298.2021.06.038
[6]	杨晓花, 高海云. 基于改进贝叶斯的书目自动分类算法[J]. 计算机科学, 2018, 45(8):203−207. Yang X H, Gao H Y. Improved bayesian algorithm based automatic classification method for bibliography[J]. Computer Science, 2018, 45(8): 203−207.
[7]	Majdar R S, Ghassemian H. A probabilistic svm approach for hyperspectral image classification using spectral and texture features[J]. International Journal of Remote Sensing, 2017, 38(15): 4265−4284. doi: 10.1080/01431161.2017.1317941
[8]	Le Cun Y, Boser B, Denker J S, et al. Handwritten digit recognition with a back-propagation network[C]// Touretzky D S. Advances in neural information processing systems. San Francisco, CA, 1990: 396−404.
[9]	Le Cun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278−2324. doi: 10.1109/5.726791
[10]	Okafor E, Pawara P, Karaaba F, et al. Comparative study between deep learning and bag of visual words for wild-animal recognition[C]//2016 IEEE symposium series on computational intelligence (SSCI). Athens: IEEE, 2017: 1−9.
[11]	向秋敏. 野生动物监测图像显著性检测算法及应用[D]. 北京: 北京林业大学. 2017. Xiang Q M. Saliency detection and application in wildlife monitoring images[D]. Beijing: Beijing Forestry University, 2017.
[12]	Horn G V, Aodha O M, Song Y, et al. The iNaturalist species classification and detection dataset[C]// Mortensen E. 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR). Salt Lake City, Utah., 2018(3): 132−139.
[13]	Timm M, Maji S, Fuller T. Large-scale ecological analyses of animals in the wild using computer vision[C]//Proceedings of the IEEE conference on computer vision and pattern recognition workshops. IEEE, 2018: 1896−1898.
[14]	王柯力, 袁红春. 基于迁移学习的水产动物图像识别方法[J]. 计算机应用, 2018, 38(5):1304−1308, 1326. Wang K L, Yuan H C. Aquatic animal image classification method based on transfer learning[J]. Journal of Computer Applications, 2018, 38(5): 1304−1308, 1326.
[15]	Willi M, Pitman R T, Cardoso A W, et al. Identifying animal species in camera trap images using deep learning and citizen science[J]. Methods in Ecology and Evolution, 2019, 10(1): 80−91. doi: 10.1111/2041-210X.13099
[16]	陈争涛, 黄灿, 杨波等. 基于迁移学习的并行卷积神经网络牦牛脸识别算法[J]. 计算机应用, 2021, 41(5):1332−1336. Chen Z T, Huang C, Yang B, et al. Parallel convolutional neural network yak face recognition algorithm based on transfer learning[J]. Computer Application, 2021, 41(5): 1332−1336.
[17]	赵歆. 基于ResNet网络的奶山羊行为识别方法研究[D]. 西安: 西北农林科技大学, 2020. Zhao X. Research on dairy goat behavior recognition method based on resnet network[D]. Xi’an: Northwest A&F University, 2020.
[18]	程浙安. 基于深度卷积神经网络的内蒙古地区陆生野生动物自动识别[D]. 北京: 北京林业大学, 2019. Cheng Z A. Automatic recognition of terrestrial wildlife in inner mongolia based on deep convolutional neural network[D]. Beijing: Beijing Forestry University, 2019.
[19]	李安琪. 基于卷积神经网络的野生动物监测图像自动识别方法研究[D]. 北京: 北京林业大学, 2020. Li A Q. Research on automatic recognition method of wildlife monitoring images based on convolutional neural network[D]. Beijing: Beijing Forestry University, 2020.
[20]	Xie S, Girshick R, Dollár P, et al. Aggregated residual transformations for deep neural networks[C]// 2017 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, 2016: 5987−5995.
[21]	赵凯琳, 靳小龙, 王元卓. 小样本学习研究综述[J]. 软件学报, 2021, 32(2):349−369. Zhao K L, Jin X L, Wang Y Z. Survey on few-shot learning[J]. Journal of Software, 2021, 32(2): 349−369.
[22]	Royle J A, Dorazio R M, Link W A. Analysis of multinomial models with unknown index using data augmentation[J]. Journal of Computational and Graphical Statistics, 2007, 16(1): 67−85. doi: 10.1198/106186007X181425
[23]	Zhang H, Wu C, Zhang Z, et al. ResNeSt: split-attention networks[J/OL]. arXiv, 2020 [2021-05-25] https://arxiv.org/abs/2004.08955
[24]	Li X, Wang W, Hu X, et al. Selective kernel networks[C]// Brendel W. 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). Long Beach, CA, 2019: 510−519.