基于特征相似度计算的网页包装器自适应  

Web Page Wrapper Adaptation Based on Feature Similarity Calculation

作　　者：陈迎仁 郭莹楠 郭享 倪一涛[1,2] 陈星[1,2] CHEN Ying-ren;GUO Ying-nan;GUO Xiang;NI Yi-tao;CHEN Xing(College of Mathematics and Computer Science,Fuzhou University,Fuzhou 350108,China;Fujian Key Laboratory of Network Computing and Intelligent Information Processing(Fuzhou University),Fuzhou 350108,China)

机构地区：[1]福州大学数学与计算机科学学院,福州350108 [2]福建省网络计算与智能信息处理重点实验室(福州大学),福州350108

出　　处：《计算机科学》2021年第S02期218-224,257,共8页Computer Science

基　　金：国家重点研发计划(2017YFB1002000);福建省自然科学基金杰青项目(2020J06014);福建省自然科学基金项目(2018J07005)。

摘　　要：随着大数据的发展,互联网数据呈现爆炸式的增长。Web作为一种重要的信息载体,包含了各种类型的信息,而包装器的提出就是为了从杂乱的Web信息中提取出目标数据。但是,随着网页更新的频繁,轻微的结构变化都可能导致原有的包装器失效,增加包装器的维护成本。针对包装器的健壮性以及维护成本问题,提出了一种基于特征相似度计算的网页包装器自适应技术。该技术主要通过解析新网页的特征集合和旧包装器所蕴含的特征信息,通过网页相似度计算,重定位旧包装器在新网页中的映射区域和映射数据项,并根据映射关系使旧包装器能够自适应新网页的数据提取。该技术主要针对各类型网站进行实验,其中包括了购物类、新闻类、资讯类、论坛类和服务类,从中选取了250对新旧版本网页,共500个网页,进行包装器自适应实验。实验结果表明,当网页结构改变时,该方法能够有效地自适应新网页的数据提取,且数据提取的平均精确度和平均召回值分别达到82.2%和84.36%。With the development of big data,Internet data has exploded.As an important information carrier,the Web contains various types of information.The wrapper is proposed to extract target data from messy Web information.However,with frequent Web page updates,minor structural changes may cause the original wrapper to fail,leading to increased maintenance costs for the wrapper.Aiming at the robustness and maintenance cost of the wrapper,a Web page wrapper adaptive technology based on feature similarity calculation is proposed.This technology mainly analyzes the feature set of the new Web page and the feature information contained in the old wrapper,and calculates the similarity of the Web page to relocate the mapping area and mapping data items of the old wrapper in the new Web page,and make the old wrapper based on the mapping relationship able to adapt the data extraction of new Web pages.The technology is mainly used for experiments on various types of Websites,including shopping,news,information,forums and services.250 pairs of old and new versions of Web pages,totaling 500 Web pages,are selected for wrapper adaptation experiments.The experimental results show that when the Web page structure changes,the method can effectively adapt to the data extraction of the new Web page,and the average precision and average recall of data extraction reach 82.2%and 84.36%,respectively.

关 键 词：网页数据抽取 自适应 包装器 相似度计算 网页特征 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]