基于迁移学习双阶段训练的情感语音克隆技术  

Emotional voice cloning technology based on two-stage training of transfer learning

作　　者：李囡 郭浩 相洁 LI Nan;GUO Hao;XIANG Jie(College of Information and Computer,Taiyuan University of Technology,Jinzhong 030600,China)

机构地区：[1]太原理工大学信息与计算机学院,山西晋中030600

出　　处：《计算机工程与设计》2024年第5期1533-1540,共8页Computer Engineering and Design

基　　金：国家自然科学基金项目(61876124、61873178);山西省科技厅基础研究基金项目(20210302123129、20210302124166、20210302123099)。

摘　　要：为解决传统基于SV2TTS架构的语音克隆系统合成语音缺乏丰富表达能力的问题,提出一种基于迁移学习双阶段训练说话人编码器的情感语音克隆方法。在说话人识别技术基础上,利用迁移学习,对说话人编码器进行音色克隆训练;在情感克隆训练阶段,采用情感语音数据集对其参数进行微调,提取具有情感信息的说话人特征。将此特征作为合成器的输入对梅尔谱的生成过程进行调节,通过声码器将此梅尔谱转换为具有目标说话人信息的情感语音。主客观分析结果表明,该方法所生成的克隆语音的情感相似度更高,音色相似度也有一定改善。To solve the problem that the traditional voice cloning system based on SV2TTS lacks rich expression ability,an emotional voice cloning method based on transfer learning two-stage training speaker encoder was proposed.Based on the speaker recognition technology,the transfer learning was used to clone the voice of the speaker encoder.In the emotion cloning training phase,the emotion speech data set was used to fine-tune the encoder parameters,so that it could extract speaker features with emotion information.This feature was used as the input of the synthesizer to adjust the generation process of the Mel spectrum.The Mel spectrum was converted into emotional speech with target speaker information through a vocoder.The results show that the emotional similarity of cloned speech generated using this method is higher,and the speaker similarity is also improved in both objective and subjective evaluation.

关 键 词：迁移学习 情感语音克隆 情感语音合成 双阶段训练 说话人识别 说话人编码 参数微调 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]