基于辅助喂食全过程意图识别的多模态安全交互方法研究  

A multimodal safe interaction method based on intention recognition in the whole process of assisted feeding

作　　者：吕相谊 赵东辉 丁嘉辉 杨俊友[1] 王硕玉[2] Lyu Xiangyi;Zhao Donghui;Ding Jiahui;Yang Junyou;Wang Shuoyu(School of Electrical Engineering,Shenyang University of Technology,Shenyang 110870,China;Department of Intelligent Mechanical Systems Engineering,Kochi University of Engineering,Kochi 7828502,Japan)

机构地区：[1]沈阳工业大学电气工程学院,沈阳110870 [2]日本高知工科大学智能机械电子学院,高知7828502

出　　处：《仪器仪表学报》2025年第1期351-362,共12页Chinese Journal of Scientific Instrument

基　　金：国家自然科学基金项目(92248304);辽宁省人工智能重点研发计划项目(2023JH26/10200018);辽宁省教育厅基本科研项目(LJ212410142073)资助。

摘　　要：为提高无法自主进食的弱机能人群面对被喂食过程中的安全性与灵活性,提出一种面向辅助喂食行为的多模态安全人机交互框架及其全过程意图识别算法。首先,结合用户特征及安全性与灵活性需求,提出一种结合视、触、力、位与语言融合的多模态人机交互框架并构建辅助喂食系统。其次,针对进食全过程包括对进食意图、菜品选择意图、动态喂食点估计与递送位姿计算、咀嚼意图等,提出一种基于视觉为主导的喂食意图全过程识别方法。选取有效刻画进食过程面部动态变化的特征点并设计融合口部纵横比与下颌纵横比识别算法,通过视线矢量分析用户菜品选择意图并基于面部实时位姿设计动态喂食点,从而形成全过程动态意图准确识别。同时,在辅助喂食虚拟映射系统中,结合大语言模型对交互全过程中的模糊意图、临时变更意图进行提问,形成反馈机制进而提升交互安全性。最终通过仿真与综合性实验验证提出的方法,通过多模态交互框架可以有效提升辅助喂食过程的灵活性,同时结合大语言模型形成模糊意图与变更意图有效反馈,最终提升交互过程的安全性,该方法为弱机能人群的日常生活辅助喂食行为提供了新型护理方法。To improve the safety and flexibility of the feeding process for individuals with limited mobility who are unable to feed themselves,this study proposes a multimodal human-robot interaction framework for assisted feeding and a whole-process intention recognition algorithm.First,based on user characteristics and the requirements for safety and flexibility,a multimodal interaction framework integrating vision,touch,force,position,and language fusion is proposed,and an assisted feeding system is developed.Second,a vision-driven whole-process intention recognition method is introduced to address the entire feeding process,including feeding intention,dish selection intention,dynamic feeding point estimation,delivery pose calculation,and chewing intention.Key facial feature points that effectively capture dynamic changes during feeding are selected,and an algorithm combining the aspect ratio of the mouth and the mandible is designed.The user′s dish selection intention is analyzed through gaze vector estimation,and dynamic feeding points are determined based on real-time facial pose tracking,enabling accurate recognition of dynamic intentions throughout the process.Furthermore,in the virtual mapping system for assisted feeding,a feedback mechanism is established by leveraging a large language model to clarify ambiguous intentions and adapt to temporary changes during the interaction,thereby enhancing safety.Finally,the proposed method is validated through simulations and comprehensive experiments.The results demonstrate that the multimodal interaction framework significantly improves the flexibility of the assisted feeding process,while the integration of the large language model provides effective feedback for ambiguous and changing intentions,ultimately enhancing the safety of the interaction.This approach offers a novel care solution for assisting feeding behaviors in the daily lives of individuals with limited mobility.To enhance the safety and flexibility of feeding process for individuals with weak motion capabi

关 键 词：辅助喂食 全过程意图识别 多模态交互框架 安全性与灵活性 

分 类 号：TN911.72[电子电信—通信与信息系统] TH711[电子电信—信息与通信工程] TH9[机械工程—测试计量技术及仪器]