融合继续预训练和分部池化的司法事件检测模型  

Judicial Event Detection Model Based on Continuous Pre-training and Segment Pooling

作　　者：张家诚 孙媛媛[1] 李志廷 杨亮[1] 林鸿飞[1] ZHANG Jiacheng;SUN Yuanyuan;LI Zhiting;YANG Liang;LIN Hongfei(School of Computer Science,Dalian University of Technology,Dalian,Liaoning 116024,China;Procuratorial Technology and Information Research Center,Supreme People s Procuratorate,Beijing 100726,China)

机构地区：[1]大连理工大学计算机科学与技术学院,辽宁大连116024 [2]最高人民检察院检察技术信息研究中心,北京100726

出　　处：《中文信息学报》2024年第4期99-107,共9页Journal of Chinese Information Processing

基　　金：国家重点研究与发展计划(2022YFC3301801);中央高校基本科研业务费项目(DUT22ZD205)。

摘　　要：事件检测任务在自然语言处理领域中具有重要的研究价值,其主要目标是从文本中识别并分类触发词,实现高级文本分析与语义理解。随着智慧司法建设的推进,自然语言处理模型与司法领域的结合日益紧密。然而,由于司法领域数据较为稀缺且一个句子大多包含多个触发词等问题,该研究在预训练阶段,通过收集的十二万条司法犯罪数据对BERT进行继续预训练,以提高预训练模型对司法领域知识的理解能力;在微调阶段提出了一种分部池化结构并融合PGD对抗训练的方法,以捕获触发词上下文和句子整体的语义特征。该模型在法研杯CAIL 2022事件检测赛道上取得了明显的性能提升,比基于BERT的基线模型平均F 1值提高了3.0%。The task of event detection as a Natural Language Processing(NLP)task aims to identify and classify trigger words from the text,enabling advanced text analysis and semantic understanding.Due to the scarcity of data in the judicial field and the fact that a sentence often contains multiple trigger words,our research continues to pre-train BERT with 120000 pieces of collected judicial crime data during the pre-training phase to enhance the understanding of judicial knowledge.During the fine-tuning phase,we propose a partitioned pooling structure combined with PGD adversarial training to capture the semantic features of the trigger word context and the overall sentence.This model achieved notable performance improvement in the CAIL 2022 event detection track,with an average 3.0%improvement of F 1-score than that of the BERT-based baseline model.

关 键 词：事件检测 司法领域 预训练模型 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]