Sequential selection and calibration of video frames for 3D outdoor scene reconstruction  

作　　者：Weilin Sun Manyi Li Peng Li Xiao Cao Xiangxu Meng Lei Meng 

机构地区：[1]Shandong University,Jinan,Shandong,China [2]Beijing HQYJ Technology Development Co.,LTD,Jinan,Shandong,China [3]National University of Singapore,Singapore,Singapore [4]Shandong Research Institute of Industrial Technology,Jinan,Shandong,China

出　　处：《CAAI Transactions on Intelligence Technology》2024年第6期1500-1514,共15页智能技术学报(英文)

基　　金：National Key R&D Program of China,Grant/Award Number:2021YFC3300203;TaiShan Scholars Program,Grant/Award Number:tsqn202211289;Oversea Innovation Team Project of the“20 Regulations for New Universities”funding program of Jinan,Grant/Award Number:2021GXRC073;Excellent Youth Scholars Program of Shandong Province,Grant/Award Number:2022HWYQ-048。

摘　　要：3D scene understanding and reconstruction aims to obtain a concise scene representation from images and reconstruct the complete scene,including the scene layout,objects bounding boxes and shapes.Existing holistic scene understanding methods primarily recover scenes from single images,with a focus on indoor scenes.Due to the complexity of real-world,the information provided by a single image is limited,resulting in issues such as object occlusion and omission.Furthermore,captured data from outdoor scenes exhibits characteristics of sparsity,strong temporal dependencies and a lack of annotations.Consequently,the task of understanding and reconstructing outdoor scenes is highly challenging.The authors propose a sparse multi-view images-based 3D scene reconstruction framework(SMSR).It divides the scene reconstruction task into three stages:initial prediction,refinement,and fusion stage.The first two stages extract 3D scene representations from each viewpoint,while the final stage involves selection,calibration and fusion of object positions and orientations across different viewpoints.SMSR effectively address the issue of object omission by utilizing small-scale sequential scene information.Experimental results on the general outdoor scene dataset UrbanScene3D-Art Sci and our proprietary dataset Software College Aerial Time-series Images,demonstrate that SMSR achieves superior performance in the scene understanding and reconstruction.

关 键 词：3D outdoor scene reconstruction 3D scene understanding multi-view fusion 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]