基于时频感知双路径Conformer的语音增强  

Speech Enhancement Based on Time-frequency-aware Dual-path Conformer

作　　者：芮阳 高勇[1] RUI Yang;GAO Yong(Sichuan University,Chengdu Sichuan 610065,China)

机构地区：[1]四川大学,四川成都610065

出　　处：《通信技术》2024年第4期338-346,共9页Communications Technology

摘　　要：近年来,Conformer在语音领域的应用表现较为突出。该模块通过结合多头自注意力机制和卷积神经网络,能够同时关注短时和长时序列信息,从而在语音处理任务中表现出卓越的性能。在此基础上提出了一种基于时频感知双路径Conformer的语音增强网络(TFDPCNet)。首先,该网络将改进的Conformer结构作为核心,采用双路径结构,构成时频感知的双路径Conformer模块(TFDP-Conformer),增强了整体网络的时频提取能力;同时,为了减小时频特征融合的难度,提出了注意力门控交叉融合模块(AGCF),通过额外的注意力门进一步增强了网络训练过程中时频特征的交互,提高了时频特征的利用率;最后,引用度量鉴别器,并对其进行适当剪枝,使得增强后的音频和原始音频在量化评价指标上保持更高的一致性。实验结果表明,相比于TSTNN算法,TFDPCNet在主观和客观指标上都有明显提高。In recent years,Conformer performs more prominently in the field of speech.By combining a multi-head self-attention mechanism and a convolutional neural network,the module is able to focus on both short-and long-time sequence information,thus showing excellent performance in speech processing tasks.Based on this,a time-frequency aware dual-path Conformer based speech enhancement network(TFDPCNet)is proposed.First,the network takes the improved Conformer structure as the core and adopts a dual-path structure to form a time-frequency-aware dual-path Conformer module(TFDP-Conformer),which enhances the time-frequency extraction ability of the overall network.Then,in order to reduce the difficulty of timefrequency feature fusion,the AGCF(Attention Gated Cross Fusion)module is proposed,which further enhances the interaction of time-frequency features during network training through additional attention gates,and improves the utilization of time-frequency features.Finally,a metric discriminator is introduced and appropriately pruned,leading to higher consistency between the enhanced and original audio in terms of quantitative evaluation metrics.The experimental results indicate that compared with the TSTNN algorithm,

关 键 词：语音增强 双路径Conformer 时频域 注意力门控交叉融合 度量鉴别器 

分 类 号：TN912.35[电子电信—通信与信息系统]