基于LSH和MapReduce的近邻模型推荐算法  

Nearest Neighbor Model Recommendation Algorithm Based on LSH and MapReduce

作　　者：陈佑雄 向阳[1] 张骐[2] 潘涛[2] 

机构地区：[1]同济大学计算机科学与技术系,上海201804 [2]神华和利时信息技术有限公司,北京100009

出　　处：《微电子学与计算机》2013年第12期47-49,53,共4页Microelectronics & Computer

基　　金：国家自然科学基金(71171148);国家科技计划课题(2012BAD35B01);上海市科技创新计划(11DZ1501703);陈家镇智慧社区和智能交通项目(11dz1210600)

摘　　要：传统的近邻模型(k-nearest Neighborhood,KNN)是一种使用广泛的协同过滤模型,但是随着用户和项目的增加,需要计算大量用户或项之间的相似度,其时间复杂度过高.通过结合位置敏感哈希(Locality-Sensitive Hashing,LSH)与MapReduce,提出了一种能够在线性时间复杂度内并行计算用户或项之间相似度的近邻模型推荐算法,降低了时间和空间复杂度.在Tencent Weibo数据集上进行了仿真实验,实验表明提出的模型能有效解决传统近邻模型对于大数据集时间复杂度过高的问题,显著地提高了传统近邻模型的精度和降低传统近邻模型的耗时.Traditional k-nearest neighborhood （KNN） model has been widely used in the recommender systems. However, with the increasing of users and items, the large scale of similarity between users or items need to be calculated and the time complexity is too high. In this paper, a nearest neighbor model recommendation algorithm combined with a locality sensitive hash （Locality--Sensitive Hashing, LSH） and MapReduce is proposed , which is a way to linear time complexity by parallel computing similarity between users or items, reducing the time and space complexity. Simulate experiments in Tencent Weibo datasets show that the proposed model can effectively solve the problem of high time complexity exists in the traditional nearest neighbor model for large data sets and significantly improve the accuracy of the traditional nearest neighbor model and reduce the time--consuming.

关 键 词：协同过滤 K-nearest NEIGHBOR LSH MAPREDUCE 

分 类 号：TP31[自动化与计算机技术—计算机软件与理论]