基于词频学习和动态词频更新的藏文自动分词系统设计  

DESIGN OF AUTOMATIC TIBETAN WORD SEGMENTATION SYSTEM BASED ON WORD FREQUENCY LEARNING AND DYNAMIC WORD FREQUENCY UPDATING

作　　者：项炜[1] 金澎[2] 

机构地区：[1]乐山师范学院智能信息处理及应用实验室,四川乐山614000 [2]乐山师范学院计算机科学学院,四川乐山614000

出　　处：《计算机应用与软件》2014年第5期106-109,共4页Computer Applications and Software

摘　　要：藏文自动分词问题是藏文自然语言处理的基本问题之一。针对藏文自动分词中的重点难点,例如:格助词的识别、歧义切分、未登录词识别技术设计一个新的藏文自动分词系统。该系统采用动态词频更新和基于上下文词频的歧义处理和未登录词识别技术。在歧义字段分词准确性、未登录词识别率和分词速度上,该系统具有较优的性能。Automatic Tibetan word segmentation is one of the basic problems in natural language processing of Tibetan. In this paper,we design a new automatic Tibetan word segmentation system in light of the keys and difficulties in it,for example： the technologies of identification of caseauxiliary word,the ambiguity segmentation,and the unknown words recognition. The system uses the techniques of the dynamic word frequency update and the ambiguity treatment and unknown words recognition which are based on the word frequency of the context. The presented system has relatively high performance in terms of the recognition accuracy of ambiguities,the recognition rate of unknown word and the segmentation speed.

关 键 词：藏文自动分词 自然语言处理 格助词 动态词频更新 歧义处理 未登录词识别 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]