基于HNP模型的强化学习状态空间表示方法  

STATE SPACE REPRESENTATION OF REINFORCEMENT LEARNING BASED ON HNP MODEL

作　　者：吴宏杰[1,2] 韩佳妍[1] 杨茹 陆卫忠 傅启明 Wu Hongjie;Han Jiayan;Yang Ru;Lu Weizhong;Fu Qiming(School of Electronics and Information Engineering,Suzhou University of Science and Technology,Suzhou 215009,Jiangsu,China;Jiangsu Provincial Key Lab for Information Processing Technologies,Soochow University,Suzhou 215006,Jiangsu,China)

机构地区：[1]苏州科技大学电子与信息工程学院,江苏苏州215009 [2]苏州大学江苏省计算机信息处理技术重点实验室,江苏苏州215006

出　　处：《计算机应用与软件》2021年第12期243-250,279,共9页Computer Applications and Software

基　　金：国家自然科学基金项目(61772357,61902272,61672371,61876217,61902271);苏州市科技项目(SYG201704,SNG201610,SZS201609)。

摘　　要：蛋白质结构预测是生物信息领域中具有挑战性的问题之一。将强化学习运用在HNP晶格模型的最优结构发现中,性能出色,但结构预测所需的状态空间巨大,容易导致维数灾难问题。在全状态空间基础上,进一步提出半状态空间与简单状态空间方法,以达到约减状态空间的目的,同时对奖赏函数与策略进行定量分析。实验结果表明,该方法有效解决全状态空间无法计算长序列的缺点,其中简单状态空间较全状态空间有3条序列预测出更低能量,半状态空间较全状态空间方法全部6条长序列都预测出更低能量,且半状态空间预测的能量平均值较简单状态空间降低了9.83百分点。Protein structure prediction is one of the challenging problems in the field of bioinformatics.When reinforcement learning is applied to the optimal structure discovery of HNP lattice model,the performance is excellent,but the state space needed for structure prediction is huge,which is easy to lead to dimension disaster.On the basis of the full state space,this paper put forward the semi state space and simple state space methods to reduce the state space,and the reward function and strategy are analyzed quantitatively.The experimental results show that this method can effectively solve the problem that the full state space can t calculate the long sequence.Among them,three sequences in simple state space predict lower energy than that in full state space,all six sequences in half state space predict lower energy than that in full state space,and the average energy predicted in half state space is 9.83 percentage points lower than that in simple state space.

关 键 词：强化学习 HNP模型 维数灾难 状态空间 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]