基于自适应学习的序列生成方法  

Sequence generation method based on adaptive learning

作　　者：张宝奇 赵书良[1,2,3] 张剑[1,2,3] 吕晓锋 Zhang Baoqi;Zhao Shuliang;Zhang Jian;Lyu Xiaofeng(College of Computer&Cyber Security,Hebei Normal University,Shijiazhuang 050024,China;Hebei Provincial Engineering Research Center for Supply Chain Big Data Analytics&Data Security,Hebei Normal University,Shijiazhuang 050024,China;Hebei Provincial Key Laboratory of Network&Information Security,Hebei Normal University,Shijiazhuang 050024,China)

机构地区：[1]河北师范大学计算机与网络空间安全学院,石家庄050024 [2]河北师范大学供应链大数据分析与数据安全河北省工程研究中心,石家庄050024 [3]河北师范大学河北省网络与信息安全重点实验室,石家庄050024

出　　处：《计算机应用研究》2022年第7期2081-2086,共6页Application Research of Computers

基　　金：国家社会科学基金重大资助项目(13&ZD091,18ZDA200);河北省重点研发计划项目(20370301D);河北师范大学重大关键技术攻关项目(L2020K01)。

摘　　要：离散序列生成广泛应用于文本生成、序列推荐等领域。目前的研究工作主要集中在提高序列生成的准确性,却忽略了生成的多样性。针对该现象,提出了一种自适应序列生成方法ECoT,设置两层元控制器,在数据层面,使用元控制器实现自适应可学习采样,自动平衡真实数据与生成数据分布得到混合数据分布;在模型层面,添加多样性约束项,并使用元控制器自适应学习最优更新梯度,提升生成模型生成多样性。此外,进一步提出融合协同训练和对抗学习的方法,提升生成模型生成准确性。与目前的主流模型进行对比实验,结果表明,在生成准确性和多样性上,自适应协同训练序列生成方法具有更均衡的准确性和多样性,同时有效缓解了生成模型的模式崩溃问题。Discrete sequence generation is widely used in text generation,sequence recommendation and other fields.The current research work mainly focuses on improving the accuracy of sequence generation,but ignores the diversity of generation.To address this phenomenon,this paper proposed an adaptive sequence generation method(ECoT),and designed a two-layer meta controller.In the data layer,the function of meta controller was to realize adaptive learning sampling,automatically balance the distribution of real data and generated data,and obtain mixed data distribution.At the model level,this paper added diversity constraints.The function of the meta controller was to adaptively learn the optimal update gradient to improve the generation diversity of the generation model.In addition,in order to improve the accuracy of the generation model,this paper proposed a method combining cooperative training and adversarial learning.Compared with the current mainstream models,the results show that the adaptive cooperative training sequence generation method has more balanced accuracy and diversity in terms of generation accuracy and diversity,and can effectively alleviate the pattern collapse of the generation model.

关 键 词：深度学习 机器学习 序列生成 协同训练 对抗学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]