多模态语义通信研究综述  

Survey of research on multimodal semantic communication

作　　者：秦志金 赵菼菼 李凡[2] 陶晓明[1] QIN Zhijin;ZHAO Tantan;LI Fan;TAO Xiaoming(Department of Electronic Engineering,Tsinghua University,Beijing 100084,China;School of Information and Communication Engineering,Xi’an Jiaotong University,Xi’an 710049,China)

机构地区：[1]清华大学电子工程系,北京100084 [2]西安交通大学信息与通信工程学院,陕西西安710049

出　　处：《通信学报》2023年第5期28-41,共14页Journal on Communications

基　　金：国家自然科学基金资助项目(No.61925105);清华大学-中国移动联合研究院基金资助项目。

摘　　要：随着人工智能与通信的交叉融合,文本、图像、音频、视频等多模态数据处理技术蓬勃发展,模态语义的共享维度被深度挖掘,多模态语义信息的高度抽象、智能简约等特性被充分利用,为语义通信带来了全新的思路和手段。首先,介绍了语义通信的基础理论和分类,分别针对文本、图像、音频、视频综述了单模态语义通信的研究现状;然后,综述了多模态语义通信的研究现状,介绍了多模态数据融合技术和安全语义通信的研究;最后,总结了多模态语义通信面临的挑战。With the cross-integration of artificial intelligence and communications,technologies for processing multimodal data such as text,image,audio,and video are booming,the shared dimension of modal semantics is deeply excavated,and the characteristics of multimodal semantic information such as high abstraction,intelligence and simplicity are being fully utilized,which brings new ideas and means to semantic communications.First,the fundamental theories and classifications of semantic communication were introduced,and the research status of single-modal semantic communication was reviewed for text,image,audio,and video respectively.Then,the research status of multimodal semantic communication was reviewed,and multimodal data fusion technology and secure semantic communication were introduced.Finally,the challenges faced by multimodal semantic communication were summarized.

关 键 词：语义通信 多模态数据融合 多模态语义通信 

分 类 号：TN919.8[电子电信—通信与信息系统]