基于双层码本的语音驱动视觉语音合成系统  

Bi-level Codebook Based Speech-driven Visual-speech Synthesis System

作　　者：贾熹滨[1] 尹宝才[1] 孙艳丰[1] 

机构地区：[1]北京工业大学多媒体与智能软件技术北京市重点实验室,北京100124

出　　处：《计算机科学》2014年第1期100-104,共5页Computer Science

基　　金：国家自然科学基金(61070117);北京市自然科学基金(4122004)资助

摘　　要：提出了一种基于双层码本的语音驱动视觉语音合成系统,该系统以矢量量化的思想为基础,建立语音特征空间到视觉语音特征空间的粗耦合映射关系。为加强语音和视觉语音的关联性,系统分别根据语音特征与视觉语音特征的相似性两次对样本数据进行自动聚类,构造同时反映语音之间与视觉语音之间相似性的双层映射码本。数据预处理阶段,提出一种能反映视觉语音几何形状特征与牙齿可见度的联合特征模型,并在语音特征LPCC及MFCC基础上采用遗传算法提取视觉语音相关的语音特征模型。合成的视频中图像数据与原始视频中图像数据的比较结果表明,合成结果能在一定程度上逼近原始数据,取得了很好的效果。The paper proposed a hi-level codebook based speech-driven visual-speech synthesis system. The system uses the vector quantization principle to establish a coarse-coupling mapping relationship from the speech feature space to the visual speech feature space, In order to enhance the relationship between the speech and the visual speech, the system makes the unsupervising-clustering on the sample data according to the similarity of both the acoustic speech and the visual speech and constructs the hi-level mapping codebook reflecting the similarity of both the acoustic speech and the visual speech. At the stage of preproeessing, the paper proposed a joint feature model, which reflects the geometric character and the visibility of teeth. The paper also proposed an approach to extract the visual speech correlative speech feature from the speech features of LPCC and MFCC on the basis of genetic algorithm. The comparison results between the synthesis image sequences with the original one show that the synthesis one can approximate the original one and the result is good. In the future research, the restriction between the visual speech contexts should be considered to im- prove the smoothness of the synthesis results.

关 键 词：双层码本 视觉语音合成 视觉语音特征 语音特征 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]