基于视觉的网页重要变化检测方法  

Vision Based Important Change Detection Method for Web Pages

作　　者：史存会 俞晓明[1] 刘悦[1] 靳小龙[1,2] 程学旗 SHI Cunhui;YU Xiaoming;LIU Yue;JIN Xiaolong;CHENG Xueqi(Key Laboratory of Network Data Science and Technology,Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100190;School of Computer Science and Technology,University of Chinese Academy of Sciences,Beijing 100049)

机构地区：[1]中国科学院计算技术研究所网络数据科学与技术重点实验室,北京100190 [2]中国科学院大学计算机科学与技术学院,北京100049

出　　处：《模式识别与人工智能》2020年第11期1004-1012,共9页Pattern Recognition and Artificial Intelligence

基　　金：国家重点研究计划项目(No.2017YFC0820404);国家杰出青年科学基金项目(No.61425016);国家自然科学基金重大研究计划项目(No.91746301)资助。

摘　　要：检测网页重要变化,判断页面核心内容是否发生变化,可有效降低数据采集中重复索引的数量,因此,文中提出基于视觉的网页重要变化检测方法,用于检测页面不同语义区域的变化,可将页面压缩表示为一个低维向量.从用户视觉的角度,理解页面不同区块语义重要度的差异.相比现有方法,文中方法独立于基于HTML类基础文档的分析方法,在新媒体,如移动互联网上,也有一定的适用性.实验也验证文中方法的有效性.Duplicate Web indexes of Web crawling can be reduced effectively by detecting important changes and determining changes of essential content in Web pages.Therefore,a vision based detection method is proposed to detect changes in different semantic regions of the page and compress the page into a low dimensional vector representation.The proposed method is utilized to understand the difference of semantic importance in different regions from the perspective of users.Compared with the existing methods,the proposed method is independent of the analysis of HTML,and thus it is suitable for new media,such as mobile Internet.Experiments show the effectiveness of the proposed method.

关 键 词：WEB内容 变化检测 视觉特征 低维向量 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]