多头自注意力在双曲空间下的点击率预测  

Click-Through Rate Prediction of Multi-Head Self-Attention in Hyperbolic Space

作　　者：韩越林 王小玉[1] HAN Yue-lin;WANG Xiao-yu(School of Computer Science and Technology,Harbin University of Science and Technology,Harbin 150080,China)

机构地区：[1]哈尔滨理工大学计算机科学与技术学院,哈尔滨150080

出　　处：《北京邮电大学学报》2021年第5期127-132,共6页Journal of Beijing University of Posts and Telecommunications

基　　金：国家自然科学基金项目(60572153,60972127)。

摘　　要：在推荐系统中,了解用户行为背后的复杂功能交互,对预测用户点击广告或商品的概率至关重要.人们一直努力寻找稀疏和高维原始特征的低维表示形式及有意义的组合.其中深度交叉网络可以显式地在每一层进行特征交叉,但其"一视同仁"地对待所有交叉特征,未考虑不同特征对结果的影响,造成一些有用信息被消除.因此提出了多头自注意力神经网络在双曲空间下的点击率预测模型.在双曲空间下,模型不再使用内积而使用洛伦兹距离违背三角不等式程度来度量特征之间的相似性与相关性,从而避免了维度灾难.实验表明,就模型准确性而言,其在点击率预测数据集上均优于深度交叉网络。In recommendation systems,understanding the complex functional interactions behind user behaviors is crucial to predict the clicking probability of users on advertisements or commodities.Efforts have been made to find low-dimensional representations and meaningful combinations of sparse and highdimensional original features.Among them,the deep&cross network can explicitly cross features at each layer.However it treats all crossing features"equally"and does not consider the influence of different features on the results,which may eliminate some useful information.Therefore,a prediction model of click-through rate of multi-head self-attention neural network in hyperbolic space is proposed.In hyperbolic space,the model uses Lorentzian distance instead of inner product,to measure the similarity and correlation between features,which can avoid dimension disaster.Experimental results show that the model is superior to the deep&cross network on predicting click-through rate data sets in terms of accuracy.

关 键 词：双曲空间 多头自注意力 洛伦兹距离 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]