基于对抗迁移学习的军事科技领域命名实体识别  

Named entity recognition in military technology field based on adversarial transfer learning

作　　者：连尧 冯俊池 丁皓 LIAN Yao;FENG Junchi;DING Hao(Institute of Logistics Science and Technology,Academy of System Engineering,Academy of Military Science,Beijing 100071,China)

机构地区：[1]军事科学院系统工程研究院后勤科学与技术研究所,北京100071

出　　处：《电子设计工程》2022年第20期121-127,共7页Electronic Design Engineering

摘　　要：当前通用领域命名实体识别模型可移植性差,在军事科技领域不具备普遍性和适应性,实际效果不佳。针对军事科技领域的独特性、标注语料规模小、实体识别任务多样等特点,将迁移学习的方法应用于军事科技领域命名实体识别,并进行了领域适配与任务适配。通过预训练掩码语言模型的方法对BERT预训练模型进行了领域适配,通过对抗迁移学习的方法对BiLSTM-CRF模型行了任务适配。模型中加入了虚拟对抗训练,通过训练减少虚拟对抗损失以提高模型的鲁棒性。在军事科技领域文本上验证了该方法,实验结果表明,领域适配与任务适配对提高识别效果都有显著的积极作用。Named entity recognition possesses good recognition results in the general field,but nevertheless poor portability among different fields,due to traditional and deep learning methods’ serious dependence on a large number of labeled training data with the same distribution. Therefore,such method is deficient in universality and adaptability in the field of military technology,leading to its unsatisfactory practical effect. In the field of military technology,feature information to be selected need rich professional knowledge. The annotation corpus is difficult to obtain,resulting in the small scale of existing data,and named entity recognition tasks are diverse, including rules and categories. According to the characteristics mentioned above,the transfer learning method was applied to the task of named entity recognition on military science and technology field,and field adaptation and task adaptation were carried out respectively. The method of pre-training mask language model was used to carry out field adaptation of the BERT pre-training model,and meanwhile,the confrontation transfer learning method was used to carry out task adaptation of the BiLSTM-CRF model. Virtual confrontation training was added to the model,which is a regularization method based on virtual confrontation loss. The virtual confrontation loss is generated by the posterior distribution of the model relative to the local disturbance around each input data point. The robustness of the model can be improved by reducing the loss through training. The method was verified on the text of military science and technology field. The experimental results show that domain adaptation and task adaptation have a significant positive influence on improving the recognition effect.

关 键 词：军事科技 命名实体识别 对抗学习 迁移学习 掩码语言模型 虚拟对抗训练 

分 类 号：TN919.5[电子电信—通信与信息系统] TP391.1[电子电信—信息与通信工程]