面向企业数据孤岛的联邦排序学习  

Cross-Silo Federated Learning-to-Rank

作　　者：史鼎元 王晏晟 郑鹏飞[1,2,3] 童咏昕 SHI Ding-Yuan;WANG Yan-Sheng;ZHENG Peng-Fei;TONG Yong-Xin(State Key Laboratory of Software Development Enviroment(Beihang University),Beijing 100191,China;Beijing Advanced Innovation Center for Big Data and Brain Computing(Beihang University),Beijing 100191,China;School of Computer Science and Engineering,Beihang University,Beijing 100191,China)

机构地区：[1]软件开发环境国家重点实验室(北京航空航天大学),北京100191 [2]大数据科学与脑机智能高精尖创新中心(北京航空航天大学),北京100191 [3]北京航空航天大学计算机学院,北京100191

出　　处：《软件学报》2021年第3期669-688,共20页Journal of Software

基　　金：国家重点研发计划(2018AAA0101100);国家自然科学基金(61822201,U1811463);软件开发环境国家重点实验室(北京航空航天大学)开放课题(SKLSDE-2020ZX-15)。

摘　　要：排序学习(learning-to-rank,简称LTR)模型在信息检索领域取得了显著成果,而该模型的传统训练方法需要收集大规模文本数据.然而,随着数据隐私保护日渐受到人们重视,从多个数据拥有者(如企业)手中收集数据训练排序学习模型的方式变得不可行.各企业之间数据被迫独立存储,形成了数据孤岛.由于排序模型训练需要使用查询记录、文档等诸多隐私信息,数据孤岛难以融合打通,这制约了排序学习模型的训练.联邦学习能够让多数据拥有方在隐私保护的前提下联合训练模型,是一种打通数据孤岛的新方法.在其启发下,提出了一种新的框架,即面向企业数据孤岛的联邦排序学习,它同时解决了联邦学习场景下排序学习所面临的两大挑战,即交叉特征生成与缺失标签处理.为了应对多方交叉特征的生成问题,使用了一种基于略图(sketch)数据结构与差分隐私的方法,其相比于传统加密方法具有更高的效率,同时还具有隐私性与结果精度的理论保证.为了应对缺失标签问题,提出了一种新的联邦半监督学习方法.最终,通过在公开数据集上的大量实验,验证了所提方法的有效性.Learning-to-rank(LTR)model has made a remarkable achievement.However,traditional training scheme for LTR model requires large amount of text data.Considering the increasing concerns about privacy protection,it is becoming infeasible to collect text data from multiple data owners as before,and thus data is forced to save separately.The separation turns data owners into data silos,among which the data can hardly exchange,causing LTR training severely compromised.Inspired by the recent progress in federated learning,a novel framework is proposed named cross-silo federated learning-to-rank(CS-F-LTR),which addresses two unique challenges faced by LTR when applied it to federated scenario.In order to deal with the cross-party feature generation problem,CS-F-LTR utilizes a sketch and differential privacy based method,which is much more efficient than encryption-based protocols meanwhile the accuracy loss is still guaranteed.To tackle with the missing label problem,CS-F-LTR relies on a semi-supervised learning mechanism that facilitates fast labeling with mutual labelers.Extensive experiments conducted on public datasets verify the effectiveness of the proposed framework.

关 键 词：排序学习 企业数据孤岛 联邦学习 略图 差分隐私 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]