基于具身交互的听觉数字孪生及无人机作战应用  

Embodied interaction-based auditory digital twin and application in UAV

作　　者：靳聪[1] 吴凤鹃 李波[2] 郭子淳 王晶[4] JIN Cong;WU Fengjuan;LI Bo;GUO Zichun;WANG Jing(School of Information and Communication Engineering,Communication University of China,Beijing 100024,China;School of Electronics and Information,Northwestern Polytechnical University,Xi’an 710129,China;Beijing University of Chemical Technology,Beijing 100029,China;School of Information and Electronics,Beijing Institute of Technology,Beijing 100029,China)

机构地区：[1]中国传媒大学信息与通信工程学院,北京100024 [2]西北工业大学电子信息学院,西安710129 [3]北京化工大学艺术与设计系,北京100029 [4]北京理工大学信息与电子学院,北京100029

出　　处：《兵器装备工程学报》2023年第12期256-262,共7页Journal of Ordnance Equipment Engineering

基　　金：国家自然科学基金项目(62207029);国家重点研发计划项目(2018YFB1403903);中国传媒大学中央高校基本科研业务费(CUC220B018)。

摘　　要：听觉数字孪生技术应用在扩展现实、军事虚拟仿真、游戏交互中是必不可少的,以实现身临其境的视听体验。音频的空间化渲染必须与视觉反馈一致,并以最小的延迟响应时间和用户动作进行具身交互。声音的波动性对逼真的具身交互和沉浸式渲染提出了严峻的挑战,并导致巨大的计算成本。从封闭到开放、静态到动态、室内和室外混合区域的虚拟场景到虚实融合的扩展现实体验,带来了运算复杂度的剧增。以一种音频视角来实现“以人为本”的模拟听众和数字孪生体之间的映射关系,通过听觉数字孪生体描述听者和虚拟环境之间的交互行动,建立基于具身交互的面向无人机作战场景的听觉数字孪生系统,实现了扩展现实体验下的空间音频孪生及多感官具身交互,讨论了无人机作战场景下的应用。Auditory digital twin technology applications are essential in extended reality,military virtual simulation,and game interaction to achieve an immersive audio-visual experience.Spatialized rendering of audio must be consistent with visual feedback and minimal latency response time to user actions for embodied interaction.The volatility of sound poses a serious challenge to realistic embodied interaction and immersive rendering,as well as significant computational costs.From closed to open,static to dynamic,virtual scenes in mixed indoor and outdoor areas to an extended reality experience that blends reality with reality,there is a dramatic increase in computational complexity.This paper takes an audio perspective to realize a“human-centered”mapping relationship between a simulated listener and a digital twin,describes the interaction between the listener and the virtual environment through an auditory digital twin,establishes an auditory digital twin system for UAV combat scenarios based on embodied interaction,realizes spatial audio twin and multisensory embodied interaction under extended reality experience,and discusses the application in UAV combat scenarios.

关 键 词：数字孪生 军事虚拟仿真 扩展现实 具身交互 空间音频 

分 类 号：TJ301[兵器科学与技术—火炮、自动武器与弹药工程]