高级检索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

小样本条件下基于数据扩充和ResNeSt的雪豹识别

张毓 高雅月 常峰源 谢将剑 张军国

张毓, 高雅月, 常峰源, 谢将剑, 张军国. 小样本条件下基于数据扩充和ResNeSt的雪豹识别[J]. 北京林业大学学报, 2021, 43(10): 89-99. doi: 10.12171/j.1000-1522.20210185
引用本文: 张毓, 高雅月, 常峰源, 谢将剑, 张军国. 小样本条件下基于数据扩充和ResNeSt的雪豹识别[J]. 北京林业大学学报, 2021, 43(10): 89-99. doi: 10.12171/j.1000-1522.20210185
Zhang Yu, Gao Yayue, Chang Fengyuan, Xie Jiangjian, Zhang Junguo. Panthera unica recognition based on data expansion and ResNeSt with few samples[J]. Journal of Beijing Forestry University, 2021, 43(10): 89-99. doi: 10.12171/j.1000-1522.20210185
Citation: Zhang Yu, Gao Yayue, Chang Fengyuan, Xie Jiangjian, Zhang Junguo. Panthera unica recognition based on data expansion and ResNeSt with few samples[J]. Journal of Beijing Forestry University, 2021, 43(10): 89-99. doi: 10.12171/j.1000-1522.20210185

小样本条件下基于数据扩充和ResNeSt的雪豹识别

doi: 10.12171/j.1000-1522.20210185
基金项目: 北京市自然科学基金项目面上项目(6192019),北京市自然科学基金项目青年项目(6214040)
详细信息
    作者简介:

    张毓,高级工程师。主要研究方向:野生动物保护管理。Email:369927234@foxmail.com 地址:810400 青海省海北州祁连县八宝镇青海省祁连山自然保护区管理局

    责任作者:

    谢将剑,博士,副教授。主要研究方向:林业信息监测、信号处理、模式识别。Email:shyneforce@bjfu.edu.cn 地址:100083 北京市海淀区清华东路35号北京林业大学工学院

    张军国,博士,教授。主要研究方向:物联网与无线传感器网络、图像处理以及深度学习。Email:zhangjunguo@bjfu.edu.cn 地址:100083 北京市海淀区清华东路35号北京林业大学工学院

  • 中图分类号: TP391.41

Panthera unica recognition based on data expansion and ResNeSt with few samples

  • 摘要:   目的  红外触发相机采集的雪豹监测图像质量参差不齐,且数量有限,为了提升小样本下雪豹的识别准确率,本研究提出一种雪豹监测图像自动识别方法。  方法  该方法基于具备注意力机制的ResNeSt50模型,使用祁连山国家公园的雪豹监测图像作为原始数据集,红外触发相机拍摄的非雪豹陆生野生动物图像作为扩充负样本,网络雪豹图像作为扩充正样本,生成3种数据集并依次进行对比实验,选择合适的扩充方式引导模型逐步关注到雪豹个体关键特征,使用梯度类激活热力图可视化进一步验证数据扩充后的有效性。  结果  使用原始数据集+扩充负样本+扩充正样本训练的模型识别效果最好,热力图可视化显示模型正确关注到雪豹个体花纹与斑点特征,对比基于Vgg16和ResNet50的识别模型,ResNeSt50的识别效果最好,测试集识别准确率达到97.70%,精确率97.26%,召回率97.59%。  结论  采用本研究提出的原始数据集+扩充负样本+扩充正样本数据扩充方法训练的模型,可以区分背景与前景,且对雪豹本身特征具有较强的判别能力,泛化能力最好。

     

  • 图  1  雪豹监测图像的常见情形

    Figure  1.  Snow leopard monitoring images of common situations

    图  2  ResNeSt的基本单元

    (h,w,c)为输入特征图的(高,宽,通道数);Cardinal K为第K个分支;Split S为第S个子组;C为中间卷积层的特征图通道数;Concatenate代表通道拼接;Split-Attention表示分割注意力模块。(h,w,c), height, width and number of channels of the input feature graph; Cardinal K, he Kth cardinal group; Split S, the Rth split; C, number of feature graph channels in the middle convolutional layer; Concatenate, concatenate between channels; Split-Attention, split attention block.

    Figure  2.  Basic unit of ResNeSt

    图  3  Split-Attention的具体结构

    $ {U}_{j} $为Split-Attention模块中第j个输入特征;$ {\widehat{U}}^{k} $为第k个分支的组合特征;Global pooling为全局池化层;Dense C/K为全连接层;BN为批量归一化层;ReLU为激活函数;S-Softmax为分类器;H, W, C/K表示中间特征层的高、宽、通道数。$ {U}_{j} $, the jth input feature in Split-Attention block; $ {\widehat{U}}^{k} $, combinatorial feature of the kth cardinal group; global pooling, global pooling layer; dense C/K, fully connected layer; BN, batch normalization layer; ReLU, activation function; R-Softmax, classifier; H, W, C/K, height, width and number of channels of middle feature layer.

    Figure  3.  Structure of Split-Attention

    图  4  识别准确率随迭代次数的变化

    Figure  4.  Variation of recognition accuracy with the number of iterations

    图  5  模型1对测试集1样本的识别热力图

    Figure  5.  Heat maps of test set 1 samples recognized by model 1

    图  6  模型1对测试集2中负样本的识别热力图

    Figure  6.  Heat maps of negative samples in test set 2 recognized by model 1

    图  7  训练集2、测试集2、测试集3的识别准确率曲线

    Figure  7.  Recognition accuracy curves of training set 2, test set 2 and test set 3

    图  8  模型2对测试集2样本的识别热力图

    Figure  8.  Heat maps of test set 2 samples recognized by model 2

    图  9  模型2对测试集3中正样本的识别热力图

    Figure  9.  Heat maps of positive samples in test set 3 recognized by model 2

    图  10  训练集3、测试集3的识别准确率曲线

    Figure  10.  Recognition accuracy curves of training set 3 and test set 3

    图  11  模型3对测试集3样本的识别热力图

    Figure  11.  Heat maps of test set 3 samples recognized by model 3

    图  12  不同模型在3个数据集上训练的识别结果

    1_1,模型1在测试集1上的识别结果;2_2,模型2在测试集2上的识别结果;3_3,模型3在测试集3上的识别结果;3_1,模型3在测试集1上的识别结果。1_1, recognition results of test set 1 by model 1; 2_2, recognition results of test set 2 by model 2; 3_3, recognition results of test set 3 by model 3; 3_1, recognition results of test set 1 by model 3.

    Figure  12.  Recognition results of three datasets by different models

    图  13  模型1得到的特征散点图

    红色n代表负样本图像的二维特征;蓝色p代表正样本图像的二维特征。下同。Red n represents two-dimensional features of negative sample images; blue p represents two-dimensional features of positive sample images. The same below.

    Figure  13.  Characteristic scatter diagram obtained by model 1

    图  14  模型2得到的特征散点图

    Figure  14.  Characteristic scatter diagram obtained by model 2

    图  15  模型3得到的特征散点图

    Figure  15.  Characteristic scatter diagram obtained by model 3

    表  1  数据集分布情况

    Table  1.   Dataset distribution

    数据集
    Dataset
    正样本数
    Number of
    positive sample
    负样本数
    Number of
    negative sample
    原始数据集 Original dataset 1 324 1 110
    扩充数据集 Extended dataset 310 524
    总数据集 Whole dataset 1 634 1 634
    下载: 导出CSV

    表  2  训练使用的扩充数据集

    Table  2.   Extended datasets for training use

    数据集来源 Dataset source数据集 Dataset训练集 Train set测试集 Test set
    原始数据集 Original dataset 数据集1 Dataset 1 训练集1 Train set 1 测试集1 Test set 1
    原始数据集 + 扩充负样本 Original dataset + extended negative sample 数据集2 Dataset 2 训练集2 Train set 2 测试集2 Test set 2
    原始数据集 + 扩充负样本+扩充正样本
    Original dataset + extended negative sample + extended positive sample
    数据集3 Dataset 3 训练集3 Train set 3 测试集3 Test set 3
    下载: 导出CSV

    表  3  模型1在测试集1、2上的识别结果

    Table  3.   Recognition results of test sets 1 and 2 by model 1

    测试集
    Test set
    准确率
    Accuracy rate/%
    精确率
    Precision rate/%
    召回率
    Recall rate/%
    测试集1 Test set 1 96.30 94.14 98.25
    测试集2 Test set 2 88.85 78.39 98.25
    下载: 导出CSV

    表  4  模型2对测试集2、3上的识别结果

    Table  4.   Recognition results of test sets 2 and 3 by model 2

    测试集
    Test set
    准确率
    Accuracy rate/%
    精确率
    Precision rate/%
    召回率
    Recall rate/%
    测试集2 Test set 297.2994.5698.68
    测试集3 Test set 394.0395.3291.06
    下载: 导出CSV

    表  5  模型3在测试集3上的识别结果

    Table  5.   Recognition results of test set 3 by model 3

    测试集
    Test set
    准确率
    Accuracy rate/%
    精确率
    Precision rate/%
    召回率
    Recall rate/%
    测试集3 Test set 397.7097.2697.59
    下载: 导出CSV
  • [1] 洪洋, 张晋东, 王玉君. 雪豹生态与保护研究现状探讨[J]. 四川动物, 2020, 39(6):711−720. doi: 10.11984/j.issn.1000-7083.20190438

    Hong Y, Zhang J D, Wang Y J. Progress in the ecology and conservation research on Panthera unica[J]. Sichuan Journal of Zoology, 2020, 39(6): 711−720. doi: 10.11984/j.issn.1000-7083.20190438
    [2] Bracciale L, Catini A, Gentile G, et al. Delay tolerant wireless sensor network for animal monitoring: the pink iguana case[C]//Alessandro D G. Proceedings of International Conference on Applications in Electronics Pervading Industry, Environment and Society. Cham, Switzerland: Springer , 2016: 18−26.
    [3] 徐峰. 新疆雪豹研究简史[J]. 人与生物圈, 2020(增刊 1):77−79.

    Xu F. A brief history of snow leopard in Xinjiang[J]. Man & Biosphere, 2020(Suppl. 1): 77−79.
    [4] 马鸣, 徐峰, 吴逸群, 等. 新疆雪豹种群密度监测方法探讨[J]. 生态与农村环境学报, 2011, 27(1):79−83. doi: 10.3969/j.issn.1673-4831.2011.01.016

    Ma M, Xu F, Wu Y Q, et al. Monitoring of population density of snow leopard in Xinjiang[J]. Journal of Ecology and Rural Environment, 2011, 27(1): 79−83. doi: 10.3969/j.issn.1673-4831.2011.01.016
    [5] 汪六三, 黄子良, 王儒敬. 基于近红外光谱和机器学习的大豆种皮裂纹识别研究[J]. 农业机械学报, 2021, 52(6):361−368. doi: 10.6041/j.issn.1000-1298.2021.06.038

    Wang L S, Huang Z L, Wang R J. Identification of soybean seed coat crack using near infrared spectroscopy and machine learning[J]. Transactions of the Chinese Society for Agricultural Machinery, 2021, 52(6): 361−368. doi: 10.6041/j.issn.1000-1298.2021.06.038
    [6] 杨晓花, 高海云. 基于改进贝叶斯的书目自动分类算法[J]. 计算机科学, 2018, 45(8):203−207.

    Yang X H, Gao H Y. Improved bayesian algorithm based automatic classification method for bibliography[J]. Computer Science, 2018, 45(8): 203−207.
    [7] Majdar R S, Ghassemian H. A probabilistic svm approach for hyperspectral image classification using spectral and texture features[J]. International Journal of Remote Sensing, 2017, 38(15): 4265−4284. doi: 10.1080/01431161.2017.1317941
    [8] Le Cun Y, Boser B, Denker J S, et al. Handwritten digit recognition with a back-propagation network[C]// Touretzky D S. Advances in neural information processing systems. San Francisco: Morgan Kaufmann, 1990: 396−404.
    [9] Le Cun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278−2324. doi: 10.1109/5.726791
    [10] Okafor E, Pawara P, Karaaba F, et al. Comparative study between deep learning and bag of visual words for wild-animal recognition[C]//2016 IEEE symposium series on computational intelligence (SSCI). Athens: IEEE, 2017: 1−9.
    [11] 向秋敏. 野生动物监测图像显著性检测算法及应用[D]. 北京: 北京林业大学, 2017.

    Xiang Q M. Saliency detection and application in wildlife monitoring images[D]. Beijing: Beijing Forestry University, 2017.
    [12] Horn G V, Aodha O M, Song Y, et al. The iNaturalist species classification and detection dataset[C]// Mortensen E. 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR). Salt Lake City: Utah, 2018(3): 132−139.
    [13] Timm M, Maji S, Fuller T. Large-scale ecological analyses of animals in the wild using computer vision[C]//Proceedings of the IEEE conference on computer vision and pattern recognition workshops. Salt Lake City: IEEE, 2018: 1896−1898.
    [14] 王柯力, 袁红春. 基于迁移学习的水产动物图像识别方法[J]. 计算机应用, 2018, 38(5):1304−1308, 1326.

    Wang K L, Yuan H C. Aquatic animal image classification method based on transfer learning[J]. Journal of Computer Applications, 2018, 38(5): 1304−1308, 1326.
    [15] Willi M, Pitman R T, Cardoso A W, et al. Identifying animal species in camera trap images using deep learning and citizen science[J]. Methods in Ecology and Evolution, 2019, 10(1): 80−91. doi: 10.1111/2041-210X.13099
    [16] 陈争涛, 黄灿, 杨波,等. 基于迁移学习的并行卷积神经网络牦牛脸识别算法[J]. 计算机应用, 2021, 41(5):1332−1336.

    Chen Z T, Huang C, Yang B, et al. Parallel convolutional neural network yak face recognition algorithm based on transfer learning[J]. Computer Application, 2021, 41(5): 1332−1336.
    [17] 赵歆. 基于ResNet网络的奶山羊行为识别方法研究[D]. 西安: 西北农林科技大学, 2020.

    Zhao X. Research on dairy goat behavior recognition method based on resnet network[D]. Xi’an: Northwest A&F University, 2020.
    [18] 程浙安. 基于深度卷积神经网络的内蒙古地区陆生野生动物自动识别[D]. 北京: 北京林业大学, 2019.

    Cheng Z A. Automatic recognition of terrestrial wildlife in Inner Mongolia based on deep convolutional neural network[D]. Beijing: Beijing Forestry University, 2019.
    [19] 李安琪. 基于卷积神经网络的野生动物监测图像自动识别方法研究[D]. 北京: 北京林业大学, 2020.

    Li A Q. Research on automatic recognition method of wildlife monitoring images based on convolutional neural network[D]. Beijing: Beijing Forestry University, 2020.
    [20] Xie S, Girshick R, Dollár P, et al. Aggregated residual transformations for deep neural networks[C]// 2017 IEEE conference on computer vision and pattern recognition (CVPR). Honolulu: IEEE, 2016: 5987−5995.
    [21] 赵凯琳, 靳小龙, 王元卓. 小样本学习研究综述[J]. 软件学报, 2021, 32(2):349−369.

    Zhao K L, Jin X L, Wang Y Z. Survey on few-shot learning[J]. Journal of Software, 2021, 32(2): 349−369.
    [22] Royle J A, Dorazio R M, Link W A. Analysis of multinomial models with unknown index using data augmentation[J]. Journal of Computational and Graphical Statistics, 2007, 16(1): 67−85. doi: 10.1198/106186007X181425
    [23] Zhang H, Wu C, Zhang Z, et al. ResNeSt: split-attention networks[J/OL]. arXiv, 2020 [2021−05−25]. https://arxiv.org/abs/2004.08955.
    [24] Li X, Wang W, Hu X, et al. Selective kernel networks[C]// Brendel W. 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). Long Beach: IEEE, 2019: 510−519.
  • 加载中
图(15) / 表(5)
计量
  • 文章访问数:  454
  • HTML全文浏览量:  243
  • PDF下载量:  52
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-05-14
  • 修回日期:  2021-06-09
  • 网络出版日期:  2021-08-04
  • 刊出日期:  2021-10-30

目录

    /

    返回文章
    返回