基于规则与统计相结合的藏文文本自动查错方法研究  

Automatic Tibetan Text Error Checking Based on Rules and Statistics

作　　者：完么扎西[1] 尼玛扎西 Pema Tashi;Nima Tashi(Minority Normal College,Qinghai Normal University,Xining,Qinghai 810008,China;School of Information Science and Technology,Tibet University,Lhasa,Tibet 850000,China)

机构地区：[1]青海师范大学民族师范学院,青海西宁810008 [2]西藏大学信息科学技术学院,西藏拉萨850000

出　　处：《中文信息学报》2022年第2期69-75,共7页Journal of Chinese Information Processing

基　　金：国家社会科学基金(19XYY021)。

摘　　要：针对目前藏文文本自动查错方法的不足,该文提出了一种基于规则和统计相结合的自动查错方法。首先以藏文拼写文法为基础,结合形式语言与自动机理论,构造37种确定型有限自动机识别现代藏文字;然后利用查找字典的方法识别梵音藏文字;最后利用互信息和t-测试差等统计方法查找藏语词语搭配错误和语法错误等真字词错误,实现藏文文本的自动查错。实验测试集由100篇新闻类语料构成,共包含49处错误。实验表明,该文方法能有效发现非字错误和真字词错误,该方法的查错召回率达到83.7%,查错准确率达到70.7%,F值达到76.7%。An automatic error checking method based on rules and statistics is proposed for automatic Tibetan text error checking. Firstly, based on the Tibetan spelling grammar, 37 types of deterministic finite automata are constructed to recognize modern Tibetan characters. Then a dictionary is employed to identify Sanskrit Tibetan. Finally, mutual information and t-test difference are used to identify true word errors including word collocation errors and grammatical errors in Tibetan texts. The test set consists of 100 news articles with 49 errors. Experiments show that the method proposed in this paper can effectively find non-character errors and true word errors, with 83.7% in recall, 70.7% in detection accuracy and 76.7% in F-measure.

关 键 词：藏文文本自动查错 非字错误 真字词错误 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]