RecStitchNet:Learning to stitch images with rectangular boundaries  

作　　者：Yun Zhang Yu-Kun Lai Lang Nie Fang-Lue Zhang Lin Xu 

机构地区：[1]Key Lab of Film and TV Media Technology of Zhejiang Province,College of Media Engineering,Communication University of Zhejiang,Hangzhou 310018,China [2]School of Computer Science and Informatics,Cardiff University,Cardiff CF244AG,UK [3]Institute of Information Science,Beijing Jiaotong University,Beijing 100091,China [4]School of Engineering and Computer Science,Victoria University of Wellington,Wellington 6012,New Zealand [5]STEM,University of South Australia,Adelaide 5095,Australia

出　　处：《Computational Visual Media》2024年第4期687-703,共17页计算可视媒体（英文版）

基　　金：supported by the Zhejiang Province Basic Public Welfare Research Program(No.LGG22F020009);Key Lab of Film and TV Media Technology of Zhejiang Province(No.2020E10015);Marsden Fund Council managed by the Royal Society of New Zealand(No.MFP-20-VUW-180).

摘　　要：Irregular boundaries in image stitching naturally occur due to freely moving cameras.To deal with this problem,existing methods focus on optimizing mesh warping to make boundaries regular using the traditional explicit solution.However,previous methods always depend on hand-crafted features(e.g.,keypoints and line segments).Thus,failures often happen in overlapping regions without distinctive features.In this paper,we address this problem by proposing RecStitchNet,a reasonable and effective network for image stitching with rectangular boundaries.Considering that both stitching and imposing rectangularity are non-trivial tasks in the learning-based framework,we propose a three-step progressive learning based strategy,which not only simplifies this task,but gradually achieves a good balance between stitching and imposing rectangularity.In the first step,we perform initial stitching by a pre-trained state-of-the-art image stitching model,to produce initially warped stitching results without considering the boundary constraint.Then,we use a regression network with a comprehensive objective regarding mesh,perception,and shape to further encourage the stitched meshes to have rectangular boundaries with high content fidelity.Finally,we propose an unsupervised instance-wise optimization strategy to refine the stitched meshes iteratively,which can effectively improve the stitching results in terms of feature alignment,as well as boundary and structure preservation.Due to the lack of stitching datasets and the difficulty of label generation,we propose to generate a stitching dataset with rectangular stitched images as pseudo-ground-truth labels,and the performance upper bound induced from the it can be broken by our unsupervised refinement.Qualitative and quantitative results and evaluations demonstrate the advantages of our method over the state-of-the-art.

关 键 词：image stitching boundaries convolutional neural network 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]