基于排序集成的哈萨克语固定短语抽取  

Rank aggregation-based Kazakh fixed phrases extraction

作　　者：桑海岩[1,2] 古丽拉·阿东别克[1,2] 孙瑞娜[3] 陈莉[1,2] 

机构地区：[1]新疆大学信息科学与工程学院,乌鲁木齐830046 [2]国家语言资源监测与研究中心少数民族语言中心哈萨克和柯尔克孜语文基地,乌鲁木齐830046 [3]新疆财经大学统计信息学院,乌鲁木齐830046

出　　处：《计算机工程与应用》2014年第21期205-209,223,共6页Computer Engineering and Applications

基　　金：国家自然科学基金(No.61063025);新疆多语种信息技术重点实验室开放项目(No.049807)

摘　　要：短语抽取是文本自动分类、主题提取及专利检索分析等文本信息理解等工作中都要应用到的一项关键技术。固定短语抽取作为短语研究的一部分,对短语标注、辞典编撰等自然语言处理任务都具有重要的现实意义。哈萨克语是黏着语,词形变化丰富,这些特点给哈语固定短语的抽取带来了一定的困难。提出一个总体的固定短语抽取算法,把固定短语抽取看作一个排序问题,使用C-value、互信息和log-likelihood进行抽取排序,并设计了一个新的排序集成方法对抽取的结果进行集成。实验分析结果表明,与单独的抽取算法比较,该算法达到了更高的准确率。Phrase extraction plays a key role in text information understanding, such as automatic text classification, topic extraction, and analysis of patent search, etc. As the part of phrase research, the fixed phrase extraction has important prac-tical significance on natural language processing tasks including the lexicographer. The Kazakh is agglutinative language, rich in inflections. These characteristics of the Kazakh bring certain difficulties to fixed phrase extraction. This paper proposes a general fixed phrase extraction algorithm. The algorithm considers the fixed phrase extraction as a scheduling problem, uses C-value, mutual information and log-likelihood statistics to extract and schedule, and presents a new rank aggregation method to obtain a scheduling result set. The experimental results indicate that the algorithm gets higher accuracy compared with popular signal extraction algorithms.

关 键 词：自然语言处理 固定短语 排序集成 互信息 似然比 C-value算法 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]