融合词频特性及邻接变化数的微博新词识别  

Weibo newword recognition combining frequency characteristic and accessor variety

作　　者：周超[1] 严馨[1] 余正涛[1] 洪旭东[1] 线岩团[1] 

机构地区：[1]昆明理工大学信息工程与自动化学院计算机系、云南省计算机技术应用重点实验室,云南昆明650500

出　　处：《山东大学学报（理学版）》2015年第3期6-10,共5页Journal of Shandong University(Natural Science)

基　　金：国家自然科学基金资助项目(61462055,61462054,61175068,61363044)

摘　　要：大量的新词伴随着微博的快速发展而产生,这些新词具有传播速度快及与其他词组合方式灵活的特点,而且在进行分词处理时容易被切分为不同的字符串。提出了一种融合词频特性及邻接变化数的微博新词识别方法。该方法首先对大规模的微博语料进行分词,然后将在两停用词间的相邻字串两两组合,根据组合后的字串频率统计取得新词候选串,再通过组合成词规则进行筛选获得候选新词,最后通过词的邻接域变化特性去除垃圾串获得新词。利用该方法在COAE 2014评测任务上进行了新词的发现实验,准确率达到36.5%,取得了较好的成绩。Along with the rapid development of Weibo,a lot of newwords have appeared. These words have characteristic that spread fast and flexible combination with other words. They are easy to be cut apart into different string in segmentation processing. Therefore a newword recognition method that combines word frequency characteristics and accessor variety was proposed. The first step was to segment the large scale Weibo sentences into words,and then combine the two adjacent strings between stop words. The newword candidate strings could be obtained according to the string frequency of the combination. After the filtration through the word formation rules,the candidate newwords would be found. Finally,through the characteristics of the word accessor variety,the garbage string was removed to get the newwords. Experiments of newword recognition on COAE 2014 task 3 showthat the accuracy can reach 36. 5% and this method has a good performance.

关 键 词：邻接变化数 微博新词 字串频率统计 成词规则 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]