基于平移不变字典的语音唇动一致性判决方法  

Lip motion and voice consistency analysis algorithm based on shift-invariant dictionary

作　　者：贺前华[1] 朱铮宇[1] 奉小慧[1] 

机构地区：[1]华南理工大学电子与信息学院,广东广州510641

出　　处：《华中科技大学学报（自然科学版）》2015年第10期69-74,共6页Journal of Huazhong University of Science and Technology(Natural Science Edition)

基　　金：国家自然科学基金资助项目(61401161;61571192);中央高校基本科研业务费专项资金资助项目(D2154950)

摘　　要：针对传统语音唇动分析模型容易忽略唇动帧间时变信息从而影响一致性判别结果的问题,提出一种基于平移不变学习字典的一致性判决方法.该方法将平移不变稀疏表示引入语音唇动一致性分析,通过音视频联合字典学习算法训练出时空平移不变的音视频字典,并采用新的数据映射方式对学习算法中的稀疏编码部分进行改进;利用字典中的音视频联合原子作为描述不同音节或词语发音时音频与唇形同步变化关系的模板,最后根据这种模板制定出语音唇动一致性评分判决准则.对四类音视频不一致数据的实验结果表明:本方法与传统统计类方法相比,对于少音节语料,总体等错误率(EER)平均从23.6%下降到11.3%;对于多音节语句,总体EER平均从22.1%下降到15.9%.In order to solve the issue of ignoring the successive and dynamic lip motion information in traditional audiovisual speech synchrony analysis models,a novel method based on shift-invariant learned dictionary was presented.In this method,sparse representation with shift-invariant dictionary was introduced to analysis the bimodal structure of articulation.Firstly,a learning dictionary was obtained based on the audiovisual coherence dictionary learning algorithm,and the sparse coding stage of the learning algorithm was modified by using a new data projection step.Secondly,the dynamic correlation between voice and lip motion of diverse syllable or word was represented as a pattern by this audiovisual coherence atom.Finally,an original audiovisual synchronization score measuring scheme was proposed according to these utterance patterns.The results of the experiment on four different inconsistent data show the good performance of the method.For long sentence,the equal error rate（EER）is reduced from 23.6% to 11.3%,and for short numeric string dataset,the EER is reduced from 22.1%to 15.9%.

关 键 词：音视频处理 稀疏表示 平移不变 一致性分析 字典学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]