藏文文本校对评测集构建  

Construction of Testset for Tibetan Text Proofreading

作　　者：才智杰[1,2] 三毛措 才让卓玛 CAI Zhijie;SAN Maocuo;CAIRANG Zhuoma(College of Computer,Qinghai Normal University,Xining,Qinghai 810016,China;Key Laboratory of Tibetan Intelligent Information Processing and Application,Xining,Qinghai 810008,China;Sichuan Province Weizhou Ethnic Normal School,Aba,Sichuan 624000,China;School of Computer Science and Technology,Southwest Minzu University,Chengdu,Sichuan610041,China)

机构地区：[1]青海师范大学计算机学院,青海西宁810016 [2]省部共建藏语智能信息处理及应用国家重点实验室,青海西宁810008 [3]四川省威州民族师范学校,四川阿坝624000 [4]西南民族大学计算机科学与技术学院,四川成都610041

出　　处：《中文信息学报》2023年第11期15-22,共8页Journal of Chinese Information Processing

基　　金：国家自然科学基金(61866032,61966031);青海省科技厅资助项目(2019-SF-129);“长江学者和创新团队发展计划”创新团队资助项目(IRT1068);青海省重点实验室项目(2013-Z-Y17,2014-Z-Y32,2015-Z-Y03);藏文信息处理与机器翻译重点实验室项目(2013-Y-17,2020-ZJ-Y05)。

摘　　要：文本校对评测集是拼写检查研究的基础,包括传统文本校对评测集和标准文本校对评测集。传统文本校对评测集是对正确的数据集通过主观经验人工伪造而得到的评测集,标准文本校对评测集是通过选择研究对象获取可信度强的真实数据集而得到的评测集。该文在分析英、汉文文本校对评测集构建方法的基础上,结合藏文的特点研究了藏文文本校对评测集的构建方法,构建了用于评价藏文文本校对性能的标准文本校对评测集,并统计分析了评测集中的错误类型及分布,验证了构建的标准文本校对评测集的有效性和可用性。Testset for text proofreading evaluation is the basis of spell checking research,including traditional and standard text proofreading testset.The traditional testset for text proofreading is obtained by artificially forging the correct data through subjective experience.The standard testset for text proofreading is obtained from the real dataset with strong reliability.Based on the analysis of the construction methods of English and Chinese text proofreading testsets,combined with the characteristics of Tibetan language,this paper studies the testset construction for Tibetan text proofreading,and completes a standard text proofreading testset with statistical analysis of the types and distribution of errors.The validity and usability of the testset are verified.

关 键 词：自然语言处理 藏文 文本校对 评测集 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]