基于Hadoop的并行化命名实体识别模型研究  

Research on parallel named entity recognition model based on Hadoop

作　　者：蔡伊娜 包先雨 林燕奎 彭锦学 彭智彬 林泳奇 李俊霖 郭云 CAI Yina;BAO Xianyu;LIN Yankui;PENG Jinxue;PENG Zhibin;LIN Yongqi;LI Junlin;GUO Yun(China Shenzhen Academy of Inspection and Quarantine,Shenzhen 518045,China;Shenzhen Customs Food Inspection and Quarantine Center,Shenzhen 518045,China;Shenzhen Customs Information Center,Shenzhen 518045,China)

机构地区：[1]深圳市检验检疫科学研究院,广东深圳518045 [2]深圳海关食品检验检疫技术中心,广东深圳518045 [3]深圳海关信息中心,广东深圳518045

出　　处：《实验技术与管理》2022年第2期7-12,39,共7页Experimental Technology and Management

基　　金：国家重点研发计划课题(2017YFC1601001,2018YFC1603601)。

摘　　要：针对现有命名实体识别存在数据处理效率低的问题,该文提出了一种并行化Block-BAC模型。提出了前处理中的数据分块优化算法,并基于Hadoop实现并行化的运作机制;采用局部注意力优化机制,有效减少模型的隐层节点。与已有的BERT-BAC模型相比,在确保较高F;值(精确率和召回率的调和平均数)的情况下,该模型训练时间和实体识别时间分别缩短60.36%、39.43%,具有更广泛的实用性。A parallel Block-BAC model is proposed to solve the problem of inefficient data processing in the current named entity recognition in this paper. The data block optimization algorithm in preprocessing is proposed,and the parallel operation mechanism is realized based on Hadoop. Local attention optimization mechanism is adopted to reduce the hidden nodes of the model. Compared with the existing BERT-BAC model, the training time and the entity recognition time of the model are reduced by 60.36% and 39.43% respectively, with a higher F;(harmonic average of accuracy rate and recall rate), which has wider practicability.

关 键 词：命名实体识别 数据分块 HADOOP 并行化 局部注意力 

分 类 号：TP301.6[自动化与计算机技术—计算机系统结构]