面向人类智能增强的多模态人机交互  

Multi-modal human-machine interaction for human intelligence augmentation

作　　者：王党校[1] 郑一磊 李腾 彭聪 王丽君 张玉茹[1] Dangxiao WANG;Yilei ZHENG;Teng LI;Cong PENG;Lijun WANG;Yuru ZHANG(State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing 100083, China)

机构地区：[1]北京航空航天大学虚拟现实技术与系统国家重点实验室,北京100083

出　　处：《中国科学：信息科学》2018年第4期449-465,共17页Scientia Sinica(Informationis)

基　　金：国家自然科学基金(编号:61572055);国家重点研发计划(编号:2016YFB1001200)资助项目

摘　　要：人机和谐是人机交互的理想境界和永恒追求.近年来人工智能的蓬勃发展使得机器智能不断进步,引发了对于人类智力造成挑战的担忧.能否利用多模态人机交互技术实现人类智能的训练和强化,是人机交互研究的新命题.神经可塑性研究成果表明,人类的核心认知能力,如注意力控制能力、工作记忆容量等,有望通过视听觉交互任务、触觉交互任务、经颅电磁刺激、脑机接口等技术进行训练和强化.本文通过剖析虚拟现实系统的沉浸感、交互性、想象性的本质优势,结合工程控制论思想,提出面向人类智能增强的视听触觉多模态有机融合的人机交互机制.通过构建可控认知负荷、及时生理反馈、双向体脑交互的新型人机交互系统,探索基于Hebbian learning可塑性机制的智能增强方法.该研究将为揭示和认识人脑神经可塑性机制提供新的研究工具,同时将促进新型人机交互的硬件设备和软件方法研究,并将在个性化教育、神经康复、特种职业认知能力强化等领域产生应用价值.Human-machine symbiosis and harmony is an ultimate goal of human-machine interaction （HMI）. Recently, the rapid development of artificial intelligence has given rise to concern regarding the domination of machine intelligence over human intelligence. Enhancing human intelligence through multi-modal HMI technology is becoming an important research topic. The studies on neural plasticity indicate that the core capabilities of human cognition including attention control ability and working memory capacity could be enhanced and trained through several approaches including visual-auditory games, haptic interaction tasks, transcranial electromag- netic stimulation, and brain-computer interfaces. In this paper, we propose a systematic paradigm of augmenting cognition through HMI tasks by seamlessly integrating haptic-visual-aural multisensory feedback. This paradigm is constructed on analyzing the inherent features of virtual reality including immersion, interaction, and imagination. Based on the concept of cybernetics, cognitive enhancement methods based on Hebbian learning are proposed to achieve several goals including the realization of controllable cognitive loads, immediate physiological feedback, and bidirectional mind-body interaction. The proposed paradigm may provide new tools for revealing the mechanism of neural plasticity and promote the development of novel human-machine interaction devices; it is promising in the generation of practical values in the domain of personalized education, neural rehabilitation, and cognitive training of specialized personnel.

关 键 词：神经可塑性 触觉交互 视听触融合 人机交互 注意力 工作记忆 智能增强 

分 类 号：TP11[自动化与计算机技术—控制理论与控制工程]