中文文本分类相关算法的研究与实现  

Research and Implementation of Related Algorithm of Chinese Text Categorization

作　　者：徐沛娟[1] 李雄飞[1] 惠玥[1] 张桂林[1] 

机构地区：[1]吉林大学计算机科学与技术学院,长春130012

出　　处：《吉林大学学报（理学版）》2009年第4期790-794,共5页Journal of Jilin University:Science Edition

基　　金：国家自然科学基金(批准号:60275026);"十一五"国家科技支撑计划重大项目基金(批准号:2006BAK01A33)

摘　　要：通过对分词歧义处理情况的分析,提出一种基于上下文的双向扫描分词算法,对分词词典进行改进,将词组短语的固定搭配引入词典中.讨论了特征项的选择及权重的设定,并引进2χ统计量参与项的权值计算,解决了目前通用TF-IDF加权法的不足,同时提出了项打分分类算法,提高了特征项对于文本分类的有效性.实验结果表明,改进后的权重计算方法性能更优越.On the basis of the analysis of the process of dealing with the Chinese word segmentation ambiguity, this paper covers bidirectional scan word segmentation algorithm based on the context. In order to improve the word segmentation dictionary, the authors put the fixed phrase into the dictionary and discussed the feature selectionand the weighting schema enactment in detail. In order to solve the problem of general TF-IDF weighting schema at present, we took statistics into consideration, and meanwhile put up the item-scoring method which improves the efficiency of the feature item about text categorization. At last we proved the advantage of the improved weighting schema through test.

关 键 词：文本分类 上下文双向扫描 向量空间模型 权重 特征选择 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]