基于Re-Perceptron-CRF的规范类文本分词研究  

Research on Word Segmentation of Normative Text based on Re-Perceptron-CRF

作　　者：李宝林 刘宇韬 LI Baolin;LIU Yutao(College of Logistics,Chengdu University of Information Technology,Chengdu 610103,China)

机构地区：[1]成都信息工程大学物流学院,四川成都610103

出　　处：《成都信息工程大学学报》2023年第3期298-305,共8页Journal of Chengdu University of Information Technology

基　　金：四川省科技服务业示范资助项目(2021GFW015);四川省电子商务与现代物流研究中心重点资助项目(DSWL21-3)。

摘　　要：通过Re-Perceptron-CRF组合方法,利用规范类文档特点,对关键词进行切分。分别采取Viterbi、Perceptron、CRF和Re-Perceptron-CRF 4种算法分别对规范类文本进行分词研究。具体为基于句法分析对规范类文本使用正则表达式进行标准化处理,得到适合分析的预处理文本,并通过Perceptron与CRF的双重算法返回各自的最优结果。实验表明,Re-Perceptron-CRF算法明显提高分词效果,在准确率和召回率上均有良好表现,其准确率和召回率分别达到94.36%和97.02%。该方法为规范类文本中文分词相关工作提供一定的研究思路,为后续应用提供好的数据支撑。但由于数据量较小,该方法仅适用于特定领域,如建筑检测领域。The Re-Perceptron-CRF combination method was used to segment key words by using the characteristics of specification documents.In this paper,four algorithms including Viterbi,Perceptron,CRF and Reerceptron-CRF are used to split the canonical text into words.Specificallyas follows:regular expressions are used to standardize the canonical text based on syntactic analysis,and the preprocessed text suitable for analysis is obtained.The optimal results are returned by the dual Perceptron and CRF algorithms.The experiment showed that the Re-Perceptron-CRF algorithm has good performance in the accuracy and recall rates of 94.36%and 97.02%respectively.This method provides some ideas for Chinese word segmentation related to standardized text,and provides good data support for subsequent applications.However,due to the small amount of data set,this method is only applicable to specific fields,such as building inspection.

关 键 词：管理科学与工程 文本分析 中文分词 Re-Perceptron-CRF 词性标注 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]