基于Q学习参数辨识的动物学习能力评价方法  

Evaluating Method of Animal Learning Ability Based on Q-learning Parameter Identification

作　　者：冯浩然 尚志刚[1,2] 杨莉芳 靳富丽 马佐豪[1,2] FENG Hao-ran;SHANG Zhi-gang;YANG Li-fang;JIN Fu-li;MA Zuo-hao(School of Electrical Engineering,Zhengzhou University,Zhengzhou 450000,China;Henan Province Key Laboratory of Brain Science and Brain-computer Interface Technology,Zhengzhou 450000,China)

机构地区：[1]郑州大学电气工程学院,郑州450000 [2]河南省脑科学与脑机接口技术重点实验室,郑州450000

出　　处：《科学技术与工程》2022年第27期11842-11849,共8页Science Technology and Engineering

基　　金：国家自然科学基金面上项目(61673353)。

摘　　要：动物在特定环境下对行为决策的学习能力是其生存的重要基础,因此,如何准确地评价动物在马尔科夫决策任务中利用过去经验与重视未来奖励的学习能力,对于动物行为学与心理学研究至关重要。设置了含有状态转移概率的马尔科夫决策任务,训练家鸽在不同状态下从两个选项中做出选择,并考虑未来收益,以最大化累计奖励。实验结束后,对家鸽的行为决策进行Q-learning建模,用学习率α评估其利用过去积累经验做出选择的能力,用折扣因子γ评估其对未来奖励的重视程度。结果表明,家鸽在马尔科夫决策任务中利用过去经验与重视未来奖励的学习能力可以通过Q-learning模型参数进行评价。The learn ability of animals to behavioral decisions in a specific environment is an important basis for their survival, therefore, how to accurately evaluate the learning ability between using past experience and valuing future reward of animals in Markov decision-making tasks is important for animal behavior and psychology. A Markov decision task with state transition probability was designed, pigeons were trained to choose between two options in different states and consider future benefits to maximize cumulative rewards. At the end of the experiment, Q-learning model of pigeons’ behavioral decision-making was modeled, and the learning rate was used to evaluate their ability to make choices based on past experience, and the discount factor was used to evaluate their emphasis on future rewards. The results shows that the learning ability between using past experience and valuing future reward of pigeons in Markov decision-making tasks could be evaluated by Q-learning model parameters.

关 键 词：行为决策 家鸽 Q-LEARNING 模型参数 学习能力 

分 类 号：Q951.4[生物学—动物学]