基于双流加权Gabor卷积网络融合的室内RGB-D图像语义分割  

Indoor RGB-D Image Semantic Segmentation Based on Dual-Stream Weighted Gabor Convolutional Network Fusion

作　　者：王旭初[1,2] 刘辉煌 牛彦敏[3] Wang Xuchu;Liu Huihuang;Niu Yanmin(Key Laboratory of Optoelectronic Tech nology and Systems of Ministry of Elucution,Chonggqing University.Chongqing 400040,China;College of Optoelectromic Engineering.Chongging University,Chongqing 400040,China;College of Com puter and Information Science,Chongqing Normal Uiversity,Chongqing 401331,China)

机构地区：[1]重庆大学光电技术及系统教育部重点实验室,重庆400040 [2]重庆大学光电工程学院,重庆400040 [3]重庆师范大学计算机与信息科学学院,重庆401331

出　　处：《光学学报》2020年第19期44-56,共13页Acta Optica Sinica

基　　金：重庆市基础与前沿研究计划(cstc2016jcyjA0317)。

摘　　要：针对室内场景下光照变化、物体相互遮挡以及类别复杂等问题,提出了一种基于双流加权Gabor卷积网络融合的彩色-深度(RGB-D)图像语义分割方法。为了获得方向和尺度不变性特征,设计了一种加权Gabor方向滤波器用于构建深度卷积网络(DCN),提取对方向和尺度变化具有适应性的特征信息。为了构建轻量级特征提取网络,采用宽残差-加权Gabor卷积网络分别提取彩色和深度双流图像特征,并利用金字塔池化模块对提取的深度特征进行多尺度融合以丰富图像上下文信息。对所提语义分割方法在NYUDv2数据集上进行实验,分别设置不同的对比方法。结果表明所提方法具有合理性和有效性,并在分割效果上具有一定的竞争性。To handle the problems of illumination change,mutual occlusion of objects,and complicated semantic categories in indoor scenes,a color-depth(RGB-D)image semantic segmentation method based on the dual-stream weighted Gabor convolutional network fusion is proposed in this work.In order to obtain direction and scale invariant features,a weighted Gabor direction filter is designed to construct a deep convolution network(DCN)to extract feature information that is adaptive to direction and scale changes.In order to build a lightweight feature extraction network,a wide residual weighted Gabor convolutional network module is used to extract color and depth dual-stream image features,and a pyramid pooling module is used to fuse the extracted depth features to enrich the image context information.The proposed semantic segmentation method is tested on NYUDv2 dataset,and different comparison methods are set up.The results show that the proposed method is reasonable and effective,and the segmentation effect is competitive.

关 键 词：图像处理 语义分割 加权Gabor卷积网络 宽残差模块 多尺度特征融合 室内RGBD图像 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]