基于混乱度的自动初建语义词典评价方法研究  

Research on Evaluation Method for Automated Construction of Chinese Thesaurus Based on Perplexity

作　　者：陈涛[1,2] 林杰[1] 罗贤春[2] 

机构地区：[1]同济大学经济与管理学院,上海200092 [2]宁波大学信息管理系,宁波315211

出　　处：《情报学报》2010年第2期240-245,共6页Journal of the China Society for Scientific and Technical Information

基　　金：国家社会科学基金资助项目（07CTQ011）.

摘　　要：语义词典在语言学和自然语言处理研究中占有相当关键的位置。利用聚类方法自动初步构造词典可以克服人工构造的主观缺陷，但对聚类后的结果比较难于评价。本文借鉴语言模型中的词混乱度概念对已经构建的语义词典进行自动评价，并计算比较SOM学习过程中不同阶段和不同输出网格的混乱度值；最后，与人工主观评价标准相比较，验证了混乱度方法对语义词典的评价与人工主观评价一致性；同时进一步分析了利用混乱度在真实语料库上对初建语义词典评价的客观性。Thesaurus is a major and pivotal problem in the areas of linguistic and nature language processing. The method of automatic clustering constructing thesaurus overcomes subjective shortcomings, but the result of clustering is hard to evaluate. This paper evaluates the effectiveness of clustering automatically by way of perplexity in language models, calculates and compares the change of perplexities of words during clustering at different lattices in SOM process. Finally, compared with artificial subjective evaluation criterion, the perplexity method made consistency and possessed objective property.

关 键 词：混乱度 词典 评价 

分 类 号：H08[语言文字—语言学]