基于小句复合体的句子边界自动识别研究  

Automatic Recognition of Sentence Boundary Based on Clause Complex

作　　者：何晓文 罗智勇[1] 胡紫娟 王瑞琦 HE Xiaowen;LUO Zhiyong;HU Zijuan;WANG Ruiqi(School of Computer Science,Beijing Language and Culture University,Beijing 100083,China)

机构地区：[1]北京语言大学信息科学学院,北京100083

出　　处：《中文信息学报》2021年第5期1-8,共8页Journal of Chinese Information Processing

基　　金：北京语言大学研究生创新基金(中央高校基本科研业务费专项资金)(19YCX124);国家自然科学基金(62076037)。

摘　　要：自然语言文本的语法结构层次包括语素、词语、短语、小句、小句复合体、语篇等。其中,语素、词、短语等相关处理技术已经相对成熟,而句子的概念至今未有公认的、适用于语言信息处理的界定。该文重新审视了语言学中句子的定义和自然语言处理中句子的切分问题,提出了中文句子切分的任务;基于小句复合体理论将句子定义为最小的话头自足的标点句序列,也就是自足的话题结构,并设计和实现了基于BERT的边界识别模型。实验结果表明,该模型对句子边界自动识别正确率、F_(1)值分别达到88.37%、83.73%,识别效果优于按照不同的标点符号机械分割的效果。The grammatical structure of natural language text consists of words, phrases, sentences, clause complexes and texts. This paper re-examines the definition of sentences in linguistics and the segmentation of sentences in natural language processing, and puts forward the task of Chinese sentence segmentation. Based on the theory of clause complex, the sentence is defined as the smallest topic self-sufficient punctuation sequence, and a sentence boundary recognition model based on BERT is designed and implemented. The experimental results show that the accuracy and F_(1) value of the model are 88.37% and 83.73%, respectively, much better than that of mechanical segmentation according to punctuation marks.

关 键 词：句子 小句复合体 句子边界识别 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]