基于串频统计的汉语和孟加拉语专有名词识别  

Chinese and Bengali Proper Noun Recognition Based on String Frequency Statistics Model

作　　者：柯修[1] 王惠临[2] 于薇[2] 

机构地区：[1]北京大学信息管理系,北京100871 [2]中国科学技术信息研究所,北京100038

出　　处：《现代图书情报技术》2011年第12期31-38,共8页New Technology of Library and Information Service

基　　金：中国科学技术信息研究所学科建设项目"自然语言处理"(项目编号:XK2011-6)的研究成果之一

摘　　要：基于Nagao串频统计算法实现汉语和孟加拉语专有名词的识别。提取未经过词性标注的中文和孟加拉语语料中的的n元串,使用改进的SSR算法过滤多余子串,利用字串的相邻字信息计算所有n元串成为专有名词的概率,并据此筛选专有名词。最后,实现基于串频统计的跨语言专有名词识别系统。实验表明,系统能够从输入的生语料中有效地识别出人名、地名、团体机构名等。This paper implements String Frequency Statistics Algorithm proposed by Nagao to build Proper Noun Recognition （PNR） system for Chinese and Bengali languages. First, n - grams are extracted from untagged input corpus, then they are filtered to get rid of redundant sub - strings, using SSR algorithm. Finally, this multilingual PNR system assigns each n - gram a probability of being a proper noun based on the information of their neighboring words and outputs results according to their probability score. The test results show that this system can effectively recognize name of people, places, organizations or institutions from the input text.

关 键 词：专有名词识别 串频统计 Nagao算法 SSR算法 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]