面向点击率预测的自注意力深度域嵌入因子分解机  

Self-attention Deep Field-embedded Factorization Machine for Click-through Rate Prediction

作　　者：李广丽[1] 叶艺源 许广鑫 张红斌[1] 吴光庭 吕敬钦 LI Guangli;YE Yiyuan;XU Guangxin;ZHANG Hongbin;WU Guangting;LYU Jingqin(School of Info.and Software Eng.,East China Jiaotong Univ.,Nanchang 330013,China)

机构地区：[1]华东交通大学信息与软件工程学院,江西南昌330013

出　　处：《工程科学与技术》2024年第5期287-296,共10页Advanced Engineering Sciences

基　　金：国家自然科学基金项目(62361027,62161011);江西省重点研发计划重点项目(20223BBE51036);教育部人文社会科学研究规划基金项目(23YJA870005);江西省自然科学基金面上项目(20232BAB202004);江西省社会科学研究规划基金项目(22TQ01);江西省高校人文社科基金项目(TQ21203);江西省教育厅科技项目(GJJ2200639,GJJ200628);江西省研究生创新专项资金项目(YC2022-s546)。

摘　　要：点击率(CTR)预测通过预测用户对广告或商品的点击概率,实现数字广告精准推荐。针对现有CTR模型存在原始嵌入向量未精化、特征交互方式偏简单的问题,本文提出自注意力深度域嵌入因子分解机(self-attention deep field-embedded factorization machine,Self-AtDFEFM)模型。首先,通过多头自注意力对原始嵌入向量加权,精化出关键低层特征;其次,构建深度域嵌入因子分解机(FEFM)模块,设计域对对称矩阵以提升不同特征域之间的交互强度,为高阶特征交互优选出低阶特征组合;再次,基于低阶特征组合构建深度神经网络(DNN),完成隐式高阶特征交互;然后,围绕精化后的嵌入向量,联合多头自注意力与残差机制堆叠多个显式高阶特征交互层,通过自注意力捕获同一特征在不同子空间上的互补信息,完成显示高阶特征交互;最后,联合显式与隐式高阶特征交互实现点击率预测。在Criteo和Avazu两大公开数据集上,将Self-AtDFEFM模型与主流基线模型在AUC和LogLoss指标上进行对比实验;为Self-AtDFEFM模型调制显式高阶特征交互层层数、注意力头数量、嵌入层维度及隐式高阶特征交互层层数等参数;对Self-AtDFEFM模型进行消融实验。实验结果表明:在两大数据集上,Self-AtDFEFM模型的AUC、LogLoss均优于主流基线模型;Self-AtDFEFM模型的全部参数已调为最佳;各模块形成合力以促使Self-AtDFEFM模型性能达到最优,其中显示高阶特征交互层的作用最大。Self-AtDFEFM模型各模块即插即用,易于构建和部署,且在性能与复杂度之间取得平衡,具备较高实用性。Objective Click-through rate(CTR)prediction realizes accurate recommendation of digital advertisements by predicting the user's click probability on advertisements or commodities.However,current CTR prediction models have the following key issues.First,the raw embedding vectors have not been fully refined.Second,the corresponding feature interaction method is too simple.As a result,the performance of the models is heavily restricted.To alleviate these issues,a novel CTR model named self-attention deep field-embedded factorization machine(Self-AtDFEFM)is proposed.Methods First,a well-known multi-head self-attention mechanism is employed to capture the implicit information of the raw embedding vectors on different sub-spaces,and the corresponding weight is calculated to further refine the key low-level features.Second,a novel field-embedded factorization machine(FEFM)is designed to strengthen the interaction intensity between different feature fields by the field pair symmetric matrix.The key low-order feature combinations are fully optimized by the FEFM module for the subsequent high-order feature interaction.Third,a deep neural network(DNN)is built based on the low-order feature combinations to complete implicit high-order feature interaction.Finally,both the explicit and implicit feature interactions are combined together to implement CTR prediction.Results and Discussions Extensive experiments have been performed on the two public available datasets,namely Criteo and Avazu.First,the proposed Self-AtDFEFM is compared with numerous state-of-the-art baselines on the AUC(area under curve)and LogLoss metrics.Second,all parameters of Self-AtDFEFM was tuned,and the parameters included the number of the explicit high-order feature interaction layers,the number of the attention heads,the embedding dimension,and the number of the implicit high-order feature interaction layers.Further,ablation experiments of our model were completed.The results of the experiments showed that:the Self-AtDFEFM model outperformed mainstream

关 键 词：点击率预测 多头自注意力 特征交互 域嵌入因子分解机 深度神经网络 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]