Large language models for robotics:Opportunities,challenges,and perspectives  

作　　者：Jiaqi Wang Enze Shi Huawen Hu Chong Ma Yiheng Liu Xuhui Wang Yincheng Yao Xuan Liu Bao Ge Shu Zhang 

机构地区：[1]School of Computer Science,Northwestern Polytechnical University,Xi’an,710072,China [2]School of Automation,Northwestern Polytechnical University,Xi’an,710072,China [3]School of Physics and Information Technology,Shaanxi Normal University,Xi’an,710119,China

出　　处：《Journal of Automation and Intelligence》2025年第1期52-64,共13页自动化与人工智能(英文)

基　　金：supported by National Natural Science Foundation of China(62376219 and 62006194);Foundational Research Project in Specialized Discipline(Grant No.G2024WD0146);Faculty Construction Project(Grant No.24GH0201148).

摘　　要：Large language models(LLMs)have undergone significant expansion and have been increasingly integrated across various domains.Notably,in the realm of robot task planning,LLMs harness their advanced reasoning and language comprehension capabilities to formulate precise and efficient action plans based on natural language instructions.However,for embodied tasks,where robots interact with complex environments,textonly LLMs often face challenges due to a lack of compatibility with robotic visual perception.This study provides a comprehensive overview of the emerging integration of LLMs and multimodal LLMs into various robotic tasks.Additionally,we propose a framework that utilizes multimodal GPT-4V to enhance embodied task planning through the combination of natural language instructions and robot visual perceptions.Our results,based on diverse datasets,indicate that GPT-4V effectively enhances robot performance in embodied tasks.This extensive survey and evaluation of LLMs and multimodal LLMs across a variety of robotic tasks enriches the understanding of LLM-centric embodied intelligence and provides forward-looking insights towards bridging the gap in Human-Robot-Environment interaction.

关 键 词：Large language models ROBOTICS Generative AI Embodied intelligence 

分 类 号：TP242[自动化与计算机技术—检测技术与自动化装置]