基于新闻网页主题要素的网页去重方法研究  

Study on duplicated removal algorithm web pages based on elements of news subject

作　　者：王鹏[1,2] 张永奎[1,2] 张彦[1,2] 刘睿[1,2] 

机构地区：[1]山西大学计算机与信息技术学院,太原030006 [2]计算智能与中文信息处理省部共建教育部重点实验室,太原030006

出　　处：《计算机工程与应用》2007年第28期177-180,共4页Computer Engineering and Applications

基　　金：国家自然科学基金(the National Natural Science Foundation of China under Grant No.60475022);山西省自然科学基金(the Natural Science Foundation of Shanxi Province of China under Grant No.20041041);山西省回国留学人员基金(No.2002004)

摘　　要：网页检索结果中,用户经常会得到内容相同的冗余页面。提出了一种通过新闻主题要素学习新闻内容的新闻网页去重算法。该方法的基本思想是:首先,抽取新闻要素中关于事件发生的时间和地点短语;然后,通过抽取的时间和地点短语抽取新闻的内容;最终,根据学习的新闻内容通过计算它们的相似度来判断新闻网页的重复度。实验结果表明,该方法能够完成针对新闻内容的新闻网页的去重,并得到较高的查全率和查准率。In the homepage retrieval result,the user can obtain the content same redundant page frequently.This article proposes one kind of duplicated news web pages removal algorithm though study news content on elements of news subject.This method basic thought is：First,extracts the time and the place phrase which in the news essential factor the event occurs;Then,through extraction time and place phrase extraction news content;Finally,through calculates their similarity according to the study news content to judge the news homepage the heavy multiplicity.The experimental result indicates that,this method can complete in view of the news content duplicated web pages,and obtains the high recall and the accuracy ratio.

关 键 词：新闻主题要素 模糊匹配 去重算法 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]