语音驱动说话数字人视频生成方法综述  

A Review on Audio-Driven Digital Human Generation Methods

作　　者：刘颖 李济廷 柴瑞坤 位纪伟 杨阳 LIU Ying;LI Jiting;CHAI Ruikun;WEI Jiwei;YANG Yang(Military Political Work Research Institute of the Academy of Military Sciences,Beijing 100166,China;School of Computer Science and Engineering,University of Electronic Science and Technology of China,Chengdu 611731,China)

机构地区：[1]军事科学院军队政治工作研究院,北京100166 [2]电子科技大学计算机科学与工程学院,成都611731

出　　处：《电子科技大学学报》2024年第6期911-921,共11页Journal of University of Electronic Science and Technology of China

基　　金：国家自然科学基金(62306067)。

摘　　要：近年来,深度学习技术的飞速发展极大地推动了虚拟数字人技术的进步,尤其是在说话数字人视频生成方面。该领域的研究在视频翻译、电影制作和虚拟助手等多个场景中展现出广阔的应用前景。该文对当前语音驱动说话数字人视频生成方法及研究现状进行了梳理与总结,并深入探讨了关键技术、数据集以及评估策略。在关键技术方面,生成对抗模型、扩散模型和神经辐射场等人工智能技术均发挥了重要作用。数据集的规模和多样性对于模型训练至关重要,而评估策略的完善则有助于更加客观地评价生成效果。说话数字人视频生成技术将继续面临众多挑战与机遇,期待该领域能够持续创新与发展,为人类社会带来更多便捷与乐趣。In recent years,the rapid development of deep learning technology has greatly promoted the progress of virtual digital human technology,especially in the area of audio-driven digital human video generation.Research in this field has shown broad application prospects in various scenarios such as video translation,film production,and virtual assistants.The current methods and research status of audio-driven digital human video generation are sorted out and summarized in this paper,focusing on the key technologies,datasets,and evaluation strategies.In terms of key technologies,artificial intelligence technologies such as generative adversarial networks,diffusion models,and neural radiance fields have all played an important role.The scale and diversity of datasets are crucial for model training,and the improvement of evaluation strategies helps to evaluate the generation effect more objectively.The technology of audio-driven digital human video generation will continue to face numerous challenges and opportunities.It is expected that this field can continue to innovate and develop,bringing more convenience and fun to human society.

关 键 词：说话数字人 视频生成 生成对抗模型 扩散模型 神经辐射场 多模态融合 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]