突发事件新闻网页的去重方法研究  

ON DELETION OF DUPLICATED BREAKING NEWS' WEBPAGES

作　　者：罗永莲[1] 罗永秀 张永奎[3] 

机构地区：[1]晋中学院计算机系,山西晋中030600 [2]晋华中学,山西晋中030600 [3]山西大学计算机与信息技术学院,山西太原030006

出　　处：《计算机应用与软件》2008年第8期24-26,共3页Computer Applications and Software

基　　金：国家自然科学基金项目(60475022);山西省高校科技研究开发项目(200613048)

摘　　要：随着人们对突发事件新闻的日益关注,需要对其进行有效地分类、索引、加工、处理。参考传统文本处理技术,结合网页结构特征和特定领域文本特征,提出在提取主题内容的基础上,根据突发事件特有的重复规律实现网页去重。实验结果表明,该方法能有效地提高网页去重准确率。With people＇s increasing attention on breaking news, these news have to be effectively classified, indexed, processed, and dealt with. In this paper we put forward an approach for duplicated webpage deletion according to the peculiar rule in repetition of the sudden events based on picking up the subject contents, in combination with the characteristics of webpage structure and special field text, and in reference to the traditional technology of text treatment. It is shown by the experimental result that this method can effectively improve the accuracy of duplicated webpages deletion.

关 键 词：突发事件新闻 权值计算 网页去重 

分 类 号：TP393.092[自动化与计算机技术—计算机应用技术] TB93[自动化与计算机技术—计算机科学与技术]