基于元信息的云盘资源检索结果去重  

Deduplicating search results of cloud disk resources using meta-information

作　　者：刘驰[1] 闫宏飞[1] 

机构地区：[1]北京大学网络与信息系统研究所,北京100871

出　　处：《山东大学学报（理学版）》2016年第7期11-17,共7页Journal of Shandong University(Natural Science)

基　　金：国家重点基础研究发展计划(973计划)项目(2014CB340400);国家自然科学基金资助项目(61272340;61472013)

摘　　要：区别于传统计算网页文本相似度的去重方法,以多媒体数据文件为主的云盘资源仅可利用相当有限的元信息进行检索结果去重。针对这一问题,以搭建的面向云盘资源数据的搜索引擎系统为基础,通过对云盘资源元信息特性的分析,发现除名称之外,资源文件后缀名、占用空间大小、资源的用户归属是判定重复记录的有效特征。在此基础上,给出了处理上述特征的归一化方法,进而使用无监督方法进行去重。实验结果表明,该方法能够有效对云盘资源检索结果去重。Different from classical duplicate detection methods which calculating text similarity of web pages,the multimedia cloud disk resources only have limited meta-information to deduplicate search results. The research is based on a newly established cloud disk resources search engine. This paper analyzed the characteristic of cloud disk resource metainformation,finding that besides resource names,extension filename,size and ownership are significant features to detect duplicate records. According to this,this paper proposed a feature normalization method and trained an unsupervised method to capture the task. Experiments proved that this method is able to solve the cloud disk resources search results deduplicating problem effectively.

关 键 词：搜索引擎 去重 云盘资源 元信息 

分 类 号：TP333[自动化与计算机技术—计算机系统结构]