基于Hadoop的异构网络协同过滤推荐算法设计  

Design of Collaborative Filtering Approach Recommendation Algorithm Based on Hadoop

作　　者：余先荣[1] 樊捷杰 YU Xianrong;FAN Jiejie(Department of Science Technology and Information Security,Jiangxi Police Institute,Nanchang 330100,China;School of Computer and Communication Engineering,University of Science and Technology Beijing,Beijing 100089,China)

机构地区：[1]江西警察学院科技与信息安全系,南昌330100 [2]北京科技大学计算机与通信工程学院,北京100089

出　　处：《信息网络安全》2022年第10期91-97,共7页Netinfo Security

基　　金：江西省教育厅科学技术研究项目[GJJ151198]。

摘　　要：在处理复杂计算任务时,来自不同人群的大量异构数据会在异构网络中产生异常值和噪声,这极易导致推荐算法性能低下。针对此类问题,文章提出了一种基于物品和用户的协同过滤个性化推荐算法,并在Hadoop平台下实现算法的分布式并行化。首先,基于皮尔森相关和余弦相似度的方法,在其相似度计算中引入评分贡献权重函数;其次,构造异构网络,通过对权重函数的设计,计算物品相似性,进而实现了离群点敏感性的钝化;最后,在Hadoop平台下,分别对基于物品和用户的异构网络的协同过滤推荐算法在影视数据集上进行验证。实验结果表明,该算法能有效提高推荐算法的准确率和实时性,改善网络监测质量,延长网络生存时间。While dealing with complex computing tasks,the large number of heterogeneous data from different populations will cause abnormal values and noise in heterogeneous networks,which will lead to low performance of the recommendation algorithm easily.Thus,a personalized recommendation algorithm is proposed for such issue based on the items.Firstly,based on the Pearson correlation and cosine similarity method,the weight function of the item contribution is introduced in the similarity calculation.Secondly,according to the construction of the heterogeneous network,the similarity of two items is calculated by the design of the weight function,the insensitive performance of the outliers is realized.Finally,according to the movie data,we realized the collaborative filtering of the recommendation algorithm based on the Hadoop platform.Experimental results show that the method can effectively improve the accuracy and real-time performance of recommendation algorithm,improve the quality of network monitoring and prolong the network lifetime.

关 键 词：协同过滤 推荐 异构网络 HADOOP 

分 类 号：TP309[自动化与计算机技术—计算机系统结构]