数字人风格化、多模态驱动与交互进展  

Advancements in digital character stylization,multimodal animation,and interaction

作　　者：潘烨 李韶旭 谭帅 韦俊杰 翟广涛 杨小康 Pan Ye;Li Shaoxu;Tan Shuai;Wei Junjie;Zhai Guangtao;Yang Xiaokang(Department of Computer Science and Engineering,Shanghai Jiao Tong University,Shanghai 200240,China)

机构地区：[1]上海交通大学计算机科学与工程系,上海200240

出　　处：《中国图象图形学报》2025年第2期334-360,共27页Journal of Image and Graphics

基　　金：国家自然科学基金项目(62472285,62102255);上海市科技重大专项项目(2021SHZDZX0102)。

摘　　要：风格化数字人是在计算机图形学、视觉艺术和游戏设计等领域中迅速发展的一个领域。数字人物的设计和制作技术取得了显著的进步,使得数字人物能够具有更加逼真的外观和行为,同时也可以更好地适应各种艺术风格和情境。本文围绕风格化数字人任务,围绕数字人的风格化生成、多模态驱动与用户交互3个核心研究方向的发展现状、前沿动态、热点问题等进行系统性综述。针对数字人的风格化生成,从显式三维模型和隐式三维模型两种数字人的三维表达方式对于方法进行分类。显式三维数字人风格化以基于优化的方法、基于生成对抗网络的方法、基于引擎的方法为主要分析对象;隐式三维数字人风格化从通用隐式场景风格化方法以及针对人脸的隐式风格化进行回顾。针对数字人的驱动,根据驱动源的不同,从显式音频驱动、文本驱动和视频驱动3个方面进行回顾。根据驱动实现算法的不同,从基于中间变量、基于编码—解码结构等方面进行回顾。此外,算法还根据中间变量的不同可分为基于关键点、三维人脸和光流的方法。针对数字人的用户交互,目前主流的交互方式是语音交互,本文对语音交互模块从自动语音识别和文本转语音合成两方面进行了回顾,对于数字人的对话系统模块,从自然语言理解和自然语言生成等方面进行了回顾。在此基础上,展望了风格化数字人研究的未来发展趋势,为后续的相关研究提供参考。Stylized digital characters have emerged as a fundamental force in reshaping the landscape of computer graph⁃ics,visual arts,and game design.Their unparalleled ability to mimic human appearance and behavior,coupled with their flexibility in adapting to a wide array of artistic styles and narrative frameworks,underscores their growing importance in crafting immersive and engaging digital experiences.This comprehensive exploration delves deeply into the complex world of stylized digital humans,explores their current development status,identifies the latest trends,and addresses the press⁃ing challenges that lie ahead in three foundational research domains:the creation of stylized digital humans,multimodal driving mechanisms,and user interaction modalities.The first domain,creation of stylized digital humans,examines the innovative methodologies employed in generating lifelike but stylistically diverse characters that can seamlessly integrate into various digital environments.From advancements in 3D modeling and texturing to the integration of artificial intelli⁃gence for dynamic character development,this section provides a thorough analysis of the tools and technologies that are pushing the boundaries of what digital characters can achieve.In the realm of multimodal driving mechanisms,this study investigates evolving techniques for animating and controlling digital humans by using a range of inputs,such as voice,ges⁃ture,and real-time motion capture.This section delves into how these mechanisms not only enhance the realism of charac⁃ter interactions but also open new avenues for creators to involve users in interactive narratives in more meaningful ways.Finally,the discussion of user interaction modalities explores the various ways in which end-users can engage with and influence the behavior of digital humans.From immersive virtual and augmented reality experiences to interactive web and mobile platforms,this segment evaluates the effectiveness of different modalities in creating a two-way interaction th

关 键 词：风格化 数字人 人脸驱动 人机交互 三维建模 深度学习 神经网络 

分 类 号：TP37[自动化与计算机技术—计算机系统结构]