基于双向长短时记忆和卷积Transformer的声学词嵌入模型  

Acoustic word embedding model based on Bi-LSTM and convolutional-Transformer

作　　者：高芸芸 赵腊生 张强[1] GAO Yunyun;ZHAO Lasheng;ZHANG Qiang(Key Laboratory of Advanced Design and Intelligent Computing,Ministry of Education(Dalian University),Dalian Liaoning 116622,China)

机构地区：[1]先进设计与智能计算省部共建教育部重点实验室(大连大学),辽宁大连116622

出　　处：《计算机应用》2024年第1期123-128,共6页journal of Computer Applications

基　　金：辽宁省教育厅基本科研项目(LJKMZ20221838)。

摘　　要：示例查询语音关键词检测中,卷积神经网络(CNN)或者循环神经网络(RNN)提取到的声学词嵌入语音信息有限,为更好地表示语音内容以及改善模型的性能,提出一种基于双向长短时记忆(Bi-LSTM)和卷积Transformer的声学词嵌入模型。首先,使用Bi-LSTM提取特征、对语音序列进行建模,并通过叠加方式来提高模型的学习能力;其次,为了能在捕获全局信息的同时学习到局部信息,将CNN和Transformer编码器并联连接组成卷积Transformer,充分利用它在特征提取上的优势,聚合更多有效的信息,提高嵌入的区分性。在对比损失约束下,所提模型平均精度达到了94.36%,与基于注意力的Bi-LSTM模型相比,平均精度提高了1.76%。实验结果表明,所提模型可以有效改善模型性能,更好地实现示例查询语音关键词检测。In Query-by-Example Spoken Term Detection(QbE-STD),the Acoustic Word Embedding(AWE)speech information extracted by Convolutional Neural Network(CNN)or Recurrent Neural Network(RNN)is limited.To better represent speech content and improve model performance,an acoustic word embedding model based on Bi-directional Long Short-Term Memory(Bi-LSTM)and convolutional-Transformer was proposed.Firstly,Bi-LSTM was utilized for extracting features,modeling speech sequences and improving the model learning ability by superposition.Secondly,to learn local information while capturing global information,CNN and Transformer encoder were connected in parallel to form convolutional-Transformer,which taking full advantages in feature extraction to aggregate more efficient information and improving the discrimination of embeddings.Under the constraint of contrast loss,the Average Precision(AP)of the proposed model reaches 94.36%,which is 1.76%higher than that of the Bi-LSTM model based on attention.The experimental results show that the proposed model can effectively improve model performance and better perform QbE-STD.

关 键 词：卷积神经网络 声学词嵌入 语音信息 示例查询语音关键词检测 循环神经网络 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]