低空林地航拍图像拼接的改进缝合线算法

张帆; 付慧; 杨刚

doi:10.13332/j.1000-1522.20170372

低空林地航拍图像拼接的改进缝合线算法

北京林业大学信息学院，北京 100083

基金项目:

教育部高等学校优秀青年教师研究基金项目 2015ZCQ-XX

详细信息

作者简介:
张帆。主要研究方向：图像拼接、计算机视觉。Email:zhangfan_0125@163.com 地址:100083 北京市海淀区清华东路35号北京林业大学信息学院

责任作者:
付慧，博士，副教授。主要研究方向：图像拼接、图像识别。Email:fuhuir@bjfu.edu.cn 地址:同上

中图分类号: S771.8
计量
- 文章访问数: 1926
- HTML全文浏览量: 543
- PDF下载量: 77
出版历程
- 收稿日期: 2017-10-15
- 修回日期: 2018-01-03
- 发布日期: 2018-04-30

Improved optimal seamless-line algorithm of aerial image mosaic in low altitude woodland

School of Information Science & Technology, Beijing Forestry University, Beijing 100083, China

摘要

摘要:
目的图像拼接效果的优劣主要取决于图像配准和图像融合两个步骤。图像配准误差导致的错切以及图像序列间的视差导致的鬼影、重影问题可通过图像融合算法减小或消除。目前图像融合算法中最佳缝合线算法的综合效果较好, 但没有考虑到拼接图的正射效果, 并且无人机低空飞行时树高相对飞行高度比值较大, 这在获取正射影像时是不可忽略的干扰因素。传统的数字正射影像(DOM)是基于数字高程模型(DEM)对单张影像进行数字微分纠正进而拼接成整个区域的正射影像图。但是, 地形高程数据和植物的高度数据获取困难, 而恢复出来的地形和植物高度与实际数据存在误差, 造成DOM在局部边缘出现扭曲、模糊问题。本文提出一种不需要DEM数据, 仅用图像信息使拼接结果图保留正射投影的改进缝合线算法。
方法首先对SURF特征检测、匹配与筛选, 用RANSAC算法求得的单应性矩阵确定相邻图像重叠区域; 然后基于重叠区中像素点位置与相邻两图像中心点位置的距离差可以反映正射效果的思想, 将距离差引入能量函数, 同时设计了动态权值参数用来平衡颜色、结构和距离三者的重要程度, 利用动态规划思想搜索得到最佳缝合线; 最后在缝合线两侧进行多频带融合生成类似正射影像的无缝拼接图。
结果实验图像来源于不同样地、不同飞行高度, 在相邻两幅图像以及同一条带航线图像上将本文的改进缝合线算法与其他3种缝合线算法以及Pix4D生成的数字正射影像进行对比。实验结果表明, 本文提出的缝合线改进算法能保留正射投影, 视觉效果优于现有的缝合线算法, 在城镇建筑图像的实验中局部效果优于Pix4D。
结论本文针对无人机低空林地航拍图像拼接重影问题和拼接结果由于视角不同而产生非正射影像区域的问题, 实现了一种不需要DEM数据进行数字微分纠正但能生成类似正射影像效果的改进缝合线算法。实验结果显示, 本文算法优于目前的最佳缝合线算法, 能够保留正射投影, 效果类似DOM, 并且在保证物体边缘清晰方面优于目前商用软件生成的DOM。这有利于更准确地计算林地的郁闭度, 估算林地植被覆盖面积, 对跟踪识别地表动植物也具有一定的帮助。除林地图像之外, 本方法也可以推广到其他需要保留正射投影的低空航拍拼接应用领域, 如城镇航拍图像等。
- 无人机 /
- 林地 /
- 图像拼接 /
- 最佳缝合线 /
- 正射投影
Abstract:
ObjectiveWhether image stitching algorithm is effective or not depends on image registration and image blending. In the stitching of forestry aerial images, misalignment caused by registration error and ghost caused by disparity could be diminished or eliminated. Optimal seamless-line algorithm's general effect is better than other blending algorithm. But it doesn't involve orthographic effect into stitching result. When UAV flies in a very low altitude, the ratio of tree height to flight height is larger, which impedes to obtain orthographic panorama. To get orthophoto of image stitching, traditional methods always use DEM data to compute digital differential rectification. However, DEM data and plant height are variable and difficult to measure. Besides, the recovered terrain as well as plant height have error with ground truth, which leads DOM to generate distortion and obscure in some local areas. Therefore, we proposed a novel improved optimal seamless-line blending algorithm using image information instead of DEM data, which can reserve orthographic projection in the final panorama.
MethodFirstly, SURF was extracted, matched and filtered. RANSAC algorithm was used to compute homographic transform matrix in order to obtain the overlap area. Secondly, we constructed energy function, including distance difference between each pixel in overlap area and its image center, based on the ideology that distance difference can reflect the orthographic effective. Each item's dynamic parameter was set to weigh color, structure and distance. Dynamic programming was applied to seek optimal seamless-line. In the end, multi-band blending was used to generate seamless panorama.
ResultThe image datasets used in experiment varied from land region and UAV altitude. The stitching result of two neighbor images and airline image series showed that our method can reserve orthographic projection whose visual effect was better than current optimal seamless-line algorithm. In the town panorama test, our method was better than DOM generated by Pix4D in local regions.
ConclusionWe proposed a novel improved optimal seamless-line algorithm using image information instead of DEM data. The algorithm can diminish misalignment and ghost, and can reserve orthographic projection in the final panorama, whose visual effect is similar to DOM. Besides, the method does better in retaining sharp edge of object than DOM generated by commercial software. The method helps to compute forest canopy density as well as vegetation coverage area more accurately, thus contributes to tracking and recognizing animals and plants more precisely. In addition to forestry, the algorithm can be extended to other types of aerial images, such as town images, and orthographic projection should be reserved in the panorama.
- UAV /
- forestland /
- image stitching /
- optimal seamless-line /
- orthographic projection

HTML全文

图 1 多视点成像原理

Figure 1. Multi-view image-forming principle

下载: 全尺寸图片幻灯片

图 2 飞行高度为100m的两幅相邻图像

Figure 2. Two neighbor images of 100m altitude

下载: 全尺寸图片幻灯片

图 3 图 2a和图 2b中利用不同缝合线算法的拼接结果

Figure 3. Mosaic results of Fig. 2a and Fig. 2b using various seamless-line algorithm

下载: 全尺寸图片幻灯片

图 4 距离差示意图

Figure 4. Distance difference diagram

下载: 全尺寸图片幻灯片

图 5 重叠区在相邻两幅图像的掩码

Figure 5. Mask of overlap region in neighbor images

下载: 全尺寸图片幻灯片

图 6 本文算法进行多频段融合的前后对比

Figure 6. Seamless-line algorithm with/without multi-band blending

下载: 全尺寸图片幻灯片

图 7 飞行高度为150m的相邻两幅图像

Figure 7. Two neighbor images of 150m altitude

下载: 全尺寸图片幻灯片

图 8 银杏林样地中3种缝合线算法与本文算法拼接结果对比

Figure 8. Mosaic comparison of three seamless-line algorithms and the algorithm in this paper for gingko forest

下载: 全尺寸图片幻灯片

图 9 飞行高度为200m的相邻两幅图像

Figure 9. Two neighbor images of 200m altitude

下载: 全尺寸图片幻灯片

图 10 红枫样地中本文算法与3种缝合线算法拼接结果对比

Figure 10. Mosaic comparison of three seamless-line algorithms and the algorithm used in this paper for Acer rubrum forest

下载: 全尺寸图片幻灯片

图 11 序列图像拼接中现有算法与文本算法的效果对比

Figure 11. Comparison of current algorithm and the algorithm used in this paper for image series mosaic

下载: 全尺寸图片幻灯片

图 12 图 11中的3块区域在4种算法中的放大效果对比

Figure 12. Comparison enlarge results of three blocks in Fig. 11 among 4 algorithms

下载: 全尺寸图片幻灯片

图 13 建筑区样地中传统缝合线算法与本文算法的拼接效果对比

Figure 13. Mosaic comparison of current seamless-line algorithm and the algorithm used in this paper in building sample area

下载: 全尺寸图片幻灯片

表 1 图像块在缝合线算法中是否保留正射投影(第2组)

Table 1 Whether image blocks reserve orthophoto in seamless-line algorithm (group 2)

图像块 Image block	Voronoi图 Voronoi graph	图割 Graph cut	动态规划 Dynamic programming	本文算法 Algorithm used in this paper
1	×	×	×	√
2	√	×	√	√
3	√	√	√	√
4	×	×	×	√
保留率Reservation rate/%	50	25	50	100
注：√代表图像块在指定算法中能保留正射投影，×代表图像块在指定算法中不能保留正射投影。下同。Notes：√ represents the image block can reserve orthophoto in the designated algorithm，× represents the image block can't reserve orthophoto in the designated algorithm. The same below.

下载: 导出CSV

表 2 图像块在缝合线算法中是否保留正射投影(第3组)

Table 2 Whether image blocks reserve orthophoto in seamless-line algorithm (group 3)

图像块 Image block	Voronoi图 Voronoigraph	图割 Graph cut	动态规划 Dynamic programming	本文算法 Algorithm used in this paper
a	×	×	√	√
b	×	×	×	√
c	×	×	×	√
保留率Reservation rate/%	0	0	33.3	100

下载: 导出CSV

参考文献(12)

[1]	郑悦, 程红, 孙文邦, 等.基于动态三角形改正范围的结构拼接缝消除方法[J].地理与地理信息科学, 2013, 29(4):113-116. http://d.old.wanfangdata.com.cn/Periodical/dlxygtyj201304025 Zheng Y, Cheng H, Sun W B, et al.A novel method for structural seamline removal based on dynamic triangle correction area[J]. Geography and Geo-Information Science, 2013, 29(4):113-116. http://d.old.wanfangdata.com.cn/Periodical/dlxygtyj201304025
[2]	Vaish V. Image stitching and panoramas[EB/OL].[2017-09-10]. https://wenku.baidu.com/view/3e944f105f0e7cd184253689.html.
[3]	张剑清, 潘励, 王树根.摄影测量学[M]. 2版.武汉:武汉大学出版社, 2010. Zhang J Q, Pan L, Wang S G. Photogrammetry[M]. 2nd ed. Wuhan: Wuhan University Press, 2010.
[4]	Davis J. Mosaics of scenes with moving objects[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Santa Barnara: IEEE, 1998 : 354-360.
[5]	Duplaquet M L. Building large image mosaics with invisible seam line building large image mosaics with invisible seam lines[C]//Proceedings of the 7th Visual Information Processing. Orlando: Society of Photo-Optical Instrumentation Engineers, 1998 : 369-377.
[6]	方贤勇, 潘志庚, 徐丹.图像拼接的改进算法[J].计算机辅助设计与图形学学报, 2003, 15(11):1362-1365. doi: 10.3321/j.issn:1003-9775.2003.11.006 Fang X Y, Pan Z G, Xu D. An improved algorithm for image mosaics[J]. Journal of Computer-Aided Design and Computer Graphics, 2003, 15(11):1362-1365. doi: 10.3321/j.issn:1003-9775.2003.11.006
[7]	Pan J, Wang M, Li D R, et al. Automatic generation of seamline network using area voronoi diagrams with overlap[J]. IEEE Transactions on Geoscience and Remote Sensing, 2009, 47(6):1737-1744. doi: 10.1109/TGRS.2008.2009880
[8]	Kwatra V, Schodl A, Essa I, et al, Graphcut textures: image and video synthesis using graph cuts[J]. ACM Transactions on Graphics, 2003, 22(3):277-286. doi: 10.1145/882262
[9]	张剑清, 孙明伟, 张祖勋.基于蚁群算法的正射影像镶嵌线自动选择[J].武汉大学学报(信息科学版), 2009, 34(6): 675-678. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=whchkjdxxb200906011 Zhang J Q, Sun M W, Zhang Z X. Automated seamline detection for orthophoto mosaicking based on ant colony algorithm[J]. Geomatics & Information Science of Wuhan University, 2009, 34(6):675-678. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=whchkjdxxb200906011
[10]	孙明伟.正射影像全自动快速制作关键技术研究[D].武汉: 武汉大学, 2009. Sun M W. Research on key technology of automatic and fast DOM generation[D]. Wuhan: Wuhan University, 2009.
[11]	Bay H, Ess A, Tuytelaars T, et al. Speeded-Up robust feature[J]. Computer Vision and Image Understanding, 2008, 110(3):346-359. http://d.old.wanfangdata.com.cn/Periodical/xtgcydzjs200911039
[12]	Allene C, Pons J P, Keriven R. Seamless image-based texture atlases using multi-band blending[C]//Proceedings of the 19th International Conference on Pattern Recognition. Tampa: IEEE, 2008: 1-4.

施引文献(12)

期刊类型引用(6)

1.	刘树超，邵全琴，杨帆，郭兴健，王东亮，黄海波，汪阳春，刘纪远，樊江文，李愈哲. 黄河源区放牧家畜数量及空间分布无人机遥感调查. 地球信息科学学报. 2021(07): 1286-1295 . 百度学术
2.	郑硕，薛兴盛，白杨，吴艳兰. 徐淮地区冬季AOD时空特征及气象因子研究. 环境科学与技术. 2020(01): 78-85 . 百度学术
3.	柴曼，周滨，徐威杰，陈晨，刘文权. 基于多尺度分割GF-1影像地表覆盖信息提取研究. 内蒙古煤炭经济. 2020(06): 1-4 . 百度学术
4.	贾亮亮，汪小钦，苏华，王峰. 台湾岛高分一号卫星WFV数据气溶胶反演与验证. 环境科学学报. 2018(03): 1117-1127 . 百度学术
5.	李裕冬，罗艳，赵海涛，王程亮，杜杰. 基于MaxEnt模型的九寨沟国家级自然保护区川金丝猴适宜生境研究. 四川动物. 2018(05): 585-591 . 百度学术
6.	贾亮亮，汪小钦，王峰. 基于波段运算和纹理特征的高分一号多光谱数据云检测. 遥感信息. 2018(05): 62-68 . 百度学术