基于神经网络的藏文正字检错法  

Tibetan Character Error Detection Based on Neural Network

作　　者：色差甲 慈祯嘉措 才让加[1,2] 华果才让 SECHA Jia;CIZHEN Jiacuo;CAIRANG Jia;HUAGUO Cairang(School of Computer Science,Qinghai Normal University,Xining,Qinhai 810008,China;Tibetan Information Processing and Machine Translation Key Laboratory of Qinghai Province,Xining,Qinhai 810008,China)

机构地区：[1]青海师范大学计算机学院,青海西宁810008 [2]青海省藏文信息处理与机器翻译重点实验室,青海西宁810008

出　　处：《中文信息学报》2020年第12期48-53,64,共7页Journal of Chinese Information Processing

基　　金：国家重点研发计划项目(2017YFB1402200);国家自然科学基金(61063033,61662061);国家社会科学基金(14BYY132);青海省科技厅项目(2020-ZJ-704,2019-SF-129)。

摘　　要：在缺乏标注数据的条件下,该文将藏文正字检错任务视为一个分类问题:首先从语言学知识中构建音节混淆子集并给每个原句加噪,然后建立深层双向表征的BERT作为分类模型,最后为了证明该方法的有效性,构建两个基线模型和三种不同领域的测试集,实验结果表明,该方法的结果优于两个基线模型。该文方法在相同领域测试集上句子分类的正确率达到93.74%,不同领域测试集上也能达到83.6%。对错误音节的识别率为74.53%,同时对无错误音节的误判率只有2.30%。This paper puts the Tibetan character error detection task as a classification problem.First of all,the syllable confusion subset is built according to the language knowledge and each Tibetan sentence is add with noise.Then a deep bi-direction representation based BERT is applied in the classification model.Two baseline model and test sets of different domains are then constructed.The experimental results show that this method is superior to the two baseline models.The accuracy of sentence classification in the same method can reach 93.74%,and achieve 83.6%in test from different fields.In the syllable level,the performance of true negative s is 74.53%,and false negative is 2.30%.

关 键 词：藏文正字检错 加噪算法 BERT模型 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]