基于图划分的分布式推荐系统  

A Distributed Recommender System Based on Graph Partition

作　　者：杨锦光 熊菲[1] 顾峻瑜 席炜亭 YANG Jinguang;XIONG Fei;GU Junyu;XI Weiting(Beijing Jiaotong University,Beijing 100044,China;Computer Network Information Center,Chinese Academy of Sciences,Beijing 100083,China;University of Chinese Academy of Sciences,Beijing 100049,China;North China Electric Power University,Beijing 100096,China)

机构地区：[1]北京交通大学,北京100044 [2]中国科学院计算机网络信息中心,北京100083 [3]中国科学院大学,北京100049 [4]华北电力大学,北京100096

出　　处：《数据与计算发展前沿（中英文）》2024年第5期102-110,共9页Frontiers of Data & Computing

基　　金：国家自然科学基金(61872033);国家自然科学基金(72004009);国家重点研发计划(2018YFC0832304);北京市科技新星计划(Z201100006820015)。

摘　　要：【目的】设计一个数据处理效率高的推荐系统具有重要的意义。【方法】使用图结构来模拟推荐系统中的用户偏好关系,将其通过图划分算法处理,可以更深层次地挖掘推荐系统中数据的信息价值,并将得到的负载均衡的子图数据作为分布式系统的输入,最终经过一个自适应聚合模块的融合实现了一个分布式推荐系统。【结果】该系统可以提高推荐算法对于大规模数据的处理效率,在预测精度不下降的前提下,算法在一个由16个CPU构成的集群训练相比于单个CPU训练可提高6.4倍的效率。【结论】实验结果证明了该系统于推荐效率方面的有效性。[Objective]It is of great significance to design a recommender system with high data processing efficiency.[Methods]The graph structure is used to simulate the user preference relationship in the recommender system.Through the graph partition algorithm processing,the information value of the data in the recommender system can be further mined,and the obtained subgraph data with load balancing can be used as the input of the distributed system.Finally,a distributed recommender system is implemented through the fusion of an adaptive aggregation module.[Results]The system can improve the processing efficiency of the recommender algorithm for large-scale data.On the premise that the prediction accuracy does not decline,the algorithm can improve the efficiency 6.4 times in a cluster training consisting of 16 CPUs compared with a single CPU training.[Conclusions]The experimental results show that the system is effective in recommendation efficiency.

关 键 词：推荐系统 图划分 负载均衡 分布式系统 

分 类 号：TP391.3[自动化与计算机技术—计算机应用技术]