基于动态文本窗口和权重动态分配的中文文本纠错方法  

Chinese Text Error Correction Method Based on Dynamic Text Window and Weighted Dynamic Allocation

作　　者：黄改娟[1,2] 王匆匆 张仰森 HUANG Gaijuan;WANG Congcong;ZHANG Yangsen(Institute of Intelligent Information Processing, Beijing Information Science and Technology University, Beijing 100101, China;Beijing Key Laboratory of Internet Culture Digital Dissemination Research, Beijing 100101, China)

机构地区：[1]北京信息科技大学智能信息处理研究所,北京100101 [2]网络文化与数字传播北京市重点实验室,北京100101

出　　处：《郑州大学学报（理学版）》2020年第3期9-14,共6页Journal of Zhengzhou University:Natural Science Edition

基　　金：国家自然科学基金项目(61772081);国家重点研发计划项目(2018YFB1402901);科技创新服务能力建设-科研基地建设-北京实验室-国家经济安全预警工程北京实验室项目(PXM2018_014224_000010)。

摘　　要：提出一种基于动态文本窗口的中文文本查错方法,依靠窗口的不断滑动来检测文本错误。当中文文本有疑似错误时,采用聚类词集平滑数据稀疏问题,然后采用权重动态分配的纠错词集进行纠错,若纠错结果仍不符合检错规则,则用缩小文本窗口法和拓展窗口法来检查具体错误。构建纠错词集则采用基于最小编辑距离和权重动态分配的方法。实验结果表明,基于动态文本窗口查错方法的F值达到了77.9%;再结合权重动态分配的纠错方法,纠错准确率达到78.1%,相较黑马校对系统和基于平均权重的纠错策略,准确率分别提升了9.7%和15.8%。A Chinese text error checking method based on dynamic text window was proposed,which relied on the continuous sliding window to detect errors in text.When the text was suspected to be wrong,the data sparse problem was smoothed by the clustering word set,and the error correction would be carried out by using the word set assigned dynamically.If the error correction results still could not conform to the error detection rules,the reduced window method and the extended window method would be used to check the specific errors.The error correction word set was constructed by a method which based on the minimum edit distance and the weighted dynamic allocation.The experimental results showed that the F-score of the dynamic text window error checking method was 77.9%.Combined with the error correction method of the weighted dynamic allocation,the error correction accuracy was 78.1%,which was 9.7%and 15.8%higher than black horse proofreading system and average weighted error correction strategy,respectively.

关 键 词：语义搭配 数据稀疏 动态文本窗口 权重动态分配 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]