Variational Learned Talking-Head Semantic Coded Transmission System  

作　　者：Yue Weijie Si Zhongwei 

机构地区：[1]Key Laboratory of Universal Wireless Communications,Ministry of Education,Beijing University of Posts and Telecommunications,Beijing 100876,China

出　　处：《China Communications》2024年第7期37-49,共13页中国通信（英文版）

基　　金：supported by the National Natural Science Foundation of China(No.61971062);BUPT Excellent Ph.D.Students Foundation(CX2022153)。

摘　　要：Video transmission requires considerable bandwidth,and current widely employed schemes prove inadequate when confronted with scenes featuring prominently.Motivated by the strides in talkinghead generative technology,the paper introduces a semantic transmission system tailored for talking-head videos.The system captures semantic information from talking-head video and faithfully reconstructs source video at the receiver,only one-shot reference frame and compact semantic features are required for the entire transmission.Specifically,we analyze video semantics in the pixel domain frame-by-frame and jointly process multi-frame semantic information to seamlessly incorporate spatial and temporal information.Variational modeling is utilized to evaluate the diversity of importance among group semantics,thereby guiding bandwidth resource allocation for semantics to enhance system efficiency.The whole endto-end system is modeled as an optimization problem and equivalent to acquiring optimal rate-distortion performance.We evaluate our system on both reference frame and video transmission,experimental results demonstrate that our system can improve the efficiency and robustness of communications.Compared to the classical approaches,our system can save over 90%of bandwidth when user perception is close.

关 键 词：semantic communications source-channel coding talking-head transmission variational modeling 

分 类 号：TN919.8[电子电信—通信与信息系统]