字符序列标注的维吾尔语词干提取方法  

Method of Uyghur stemming based on character sequence labeling

作　　者：古丽尼格尔·阿不都外力 买合木提·买买提[1,2] 吐尔根·依布拉音 早克热·卡德尔[1,2] 西热艾力·海如拉 王路路[1,2] ABUDUWAILI Gulinigeer;MAIMAITI Maihemuti;YIBULAYIN Tuergen;KADEER Zaokere;HAIRULA Xireaili;WANG Lulu(School of Information Science and Engineering,Xinjiang University,Urumqi 830046,China;Xinjiang Laboratory of Multi⁃Language Information Technology,Urumqi 830046,China)

机构地区：[1]新疆大学信息科学与工程学院,新疆乌鲁木齐830046 [2]新疆多语种信息技术实验室,新疆乌鲁木齐830046

出　　处：《现代电子技术》2020年第12期151-154,160,共5页Modern Electronics Technique

基　　金：新疆多语种信息技术实验室开放课题(2016D03023);国家自然科学基金资助项目(61762084);国家自然科学基金资助项目(61662077);国家语委科研项目(ZDI135⁃54)。

摘　　要：词干提取是形态丰富语言信息处理中的基础任务,对其他自然语言处理任务有着重要的影响。该文将词干提取任务看作序列标注问题,以字符为切分粒度来表征维吾尔语单词的构成机制,结合条件随机场模型,实现基于字符序列标注的维吾尔语词干提取方法。首先使用词典查询方法进行词干提取,然后结合字符的弱化发音特征、音类特征以及语音特征,针对受限数据和非受限数据采用条件随机场训练模型及预测结果。实验结果表明,该方法在非受限数据集上效果较佳,且能广泛应用到其他语言。Stemming is a fundamental task in the morphologically rich language information processing and has an important influence on other natural language processing(NLP)tasks. Taking the task for stemming as the process of character sequence labeling in this paper,and using the character as the segmentation unit to represent the formation mechanism of Uyghur word,the method of Uyghur stemming based on character sequence labeling is realized in combination with the conditional random field model. The dictionary query method is used to stemming,and then the model training with the conditional random field and the results′prediction are performed for the restricted data and unrestricted data by combining with the weak pronunciation features,sound class features and speech features of characters. The experimental results this method has well effect on unrestricted datasets and can be widely used in other languages.

关 键 词：词干提取 序列标注 条件随机场 特征提取 模型训练 预测结果 

分 类 号：TN912.34-34[电子电信—通信与信息系统] TP391[电子电信—信息与通信工程]