基于BERT预训练与混合神经网络的中文语义识别算法设计  

Design of Chinese semantic recognition algorithm based on BERT pre training and hybrid neural network

作　　者：蓝天虹 陈丹霏 郑源 徐正一 LAN Tianhong;CHEN Danfei;ZHENG Yuan;XU Zhengyi(Wenzhou Power Supply Company,State Grid Zhejiang Electric Power Co.,Ltd.,Wenzhou 325000,China;Center of Mass Entrepreneurship and Innovation,State Grid Zhejiang Electric Power Co.,Ltd.,Hangzhou 310000,China;Ruian Power Supply Company,State Grid Zhejiang Electric Power Co.,Ltd.,Ruian 325200,China)

机构地区：[1]国网浙江省电力有限公司温州供电公司,浙江温州325000 [2]国网浙江省电力有限公司双创中心,浙江杭州310000 [3]国网浙江省电力有限公司瑞安市供电公司,浙江瑞安325200

出　　处：《电子设计工程》2024年第12期91-95,共5页Electronic Design Engineering

基　　金：国网浙江省电力有限公司双创中心资助项目(B711JZ210007)。

摘　　要：针对现阶段电力智能客服沟通效率低且语义理解能力不佳的问题,文中基于BERT预训练模型和混合神经网络提出了一种中文语义识别算法。该算法使用BERT模型进行词嵌入表示,在得到深度编码信息的同时还可以获取上下文联系信息。通过将Bi-GRU、注意力机制以及CRF模型进行融合,使其能够处理基于上下文的词向量。同时构建的混合神经网络也可以捕获词向量的多维特征信息,进而全面提升模型的意图识别及中文语义理解能力。在实验测试中,所提算法的意图识别准确率与F1值相较于基线算法分别提升了11.3%和6.6%,表明对语料的预训练可以有效提升模型语义识别的能力。Aiming at the shortcomings of low communication efficiency and poor semantic understanding ability of electric power intelligent customer service at present,this paper proposes a Chinese semantic recognition algorithm based on BERT pre training model and hybrid neural network.The algorithm uses BERT model for word embedding representation,which can obtain both depth coding information and context contact information.By fusing Bi⁃GRU,attention mechanism and CRF model,it can process context based word vectors.The constructed hybrid neural network can also capture multi⁃dimensional feature information of word vectors,thus comprehensively improving the ability of model intention recognition and Chinese semantic understanding.In the experimental test,the accuracy of intention recognition of the algorithm proposed in this paper is 11.3%higher than that of the baseline algorithm,and the F1 value is 6.6%higher,indicating that the pre training of corpus can effectively improve the ability of model semantic recognition.

关 键 词：BERT预训练 循环神经网络 条件随机场 注意力机制 语义识别 自然语言处理 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]