面向跨模态通信的信息恢复技术  

Information Recovery Technology for Cross-Modal Communications

作　　者：徐建博 魏昕[1,2] 周亮[1,2] XU Jian-bo;WEI Xin;ZHOU Liang(School of Telecommunications&Information Engineering,Nanjing University of Posts and Telecommunications,Nanjing,Jiangsu 210003,China;Key Laboratory of Broadband Wireless Communication and Sensor Network Technology(Ministry of Education),Nanjing University of Posts and Telecommunications,Nanjing,Jiangsu 210003,China)

机构地区：[1]南京邮电大学通信与信息工程学院,江苏南京210003 [2]南京邮电大学宽带无线通信与传感网技术教育部重点实验室,江苏南京210003

出　　处：《电子学报》2022年第7期1631-1642,共12页Acta Electronica Sinica

基　　金：国家自然科学基金(No.62071254);江苏高校优势学科建设工程项目;南京邮电大学宽带无线通信与传感网技术教育部重点实验室开放课题(No.JZNY202111)。

摘　　要：针对多模态数据在传输过程中丢失、受到无线信道噪声污染而严重影响跨模态通信质量的问题,提出了一种面向跨模态通信的信息恢复技术,通过充分利用接收端已有数据,采用同模态一对一检索、跨模态一对一检索、跨模态一对多检索等方式,在接收端进行信息恢复.所提方法在公共数据集以及实际跨模态通信平台上进行验证,实验表明,该方法可以实现精准的信息恢复,有效提升了跨模态通信质量.Aiming at the issues of multi-modal data loss and data pollution by noise of wireless channel during the transmission,which seriously affect the cross-modal communication quality,an information recovery technology for crossmodal communications is proposed.In this scheme,by making full use of the existing data at the receiving end,the information is recovered at the receiving end by means of one-to-one intra-modal retrieval,one-to-one cross-modal retrieval,one-tomany cross-modal retrieval,etc.Moreover,the proposed scheme is validated on an open data set and the practical crossmodal communication platform.Experimental results show that the scheme can achieve accurate multi-modal information recovery and effectively improve the quality of cross-modal communications.

关 键 词：跨模态通信 信息恢复 语义融合 多模态业务 

分 类 号：TP302[自动化与计算机技术—计算机系统结构]