基于动态频域分解的乐队指挥动作生成  

Conducting motion generation based on dynamic frequency domain decomposition

作　　者：贺鑫 刘凡[1] 陈德龙 周睿志 He Xin;Liu Fan;Chen Delong;Zhou Ruizhi(School of Computer&Information,Hohai University,Nanjing 210098,China)

机构地区：[1]河海大学计算机与信息学院,南京210098

出　　处：《计算机应用研究》2024年第3期923-927,955,共6页Application Research of Computers

基　　金：国家自然科学基金资助项目(62372155);装备预研教育部联合基金资助项目;江苏高校“青蓝工程”资助项目。

摘　　要：近年来,音乐与人体动作之间的内在关联一直以来都在被广泛研究。然而,很少有人关注音乐驱动的乐队指挥动作生成这一任务,即以音乐为输入信号,生成与音乐节奏和语义相协调的乐队指挥动作。聚焦于这一任务,针对指挥动作多种语义成分时空重合的特性,提出基于动作动态频域分解(dynamic frequency-domain motion decomposition,DFMD)的指挥动作生成方法。具体地,首先利用节拍信息构建滤波器,将指挥动作分解成高频和低频分量;接着,通过深度卷积神经网络动态地学习这些分量;最后合成最终的指挥动作。在大规模指挥动作数据集ConductorMotion100上进行的实验中,基于DFMD的指挥动作生成方法得到的低频动作分量和高频动作分量的标准差分别达到了4.4579和9.6466,与真实动作十分接近。该方法突破了现有基于时域或空间域动作分解中连贯性与协调性不可兼得的局限,并有效避免了大幅值低频动作对小幅值高频动作的影响。可视化结果证明生成的动作自然、美观、多样,且与音乐信号紧密同步。为音乐与动作之间的关联提供了新的解释,并为音乐表演领域带来了创新的应用前景。In recent years,the intrinsic relationship between music and motions have been widely studied.However,very few efforts have been made to develop music-driven conducting motion generation models,which takes music as input signal to generate conducting motion in harmony with music rhythm and semantics.This paper proposed a music-driven conducting motion generation approach based on DFMD.Specifically,firstly it constructed a filter using the beat information to decompose the command action into high and low frequency components.Then,a deep convolutional neural network dynamically learnt these components,and it synthesized the final command action.Experimental results on the large-scale ConductorMotion100 dataset show that the standard deviation of the generated low-frequency and high-frequency motion components is 4.4579 and 9.6466,which are very close to the real motions.The proposed method breaks through the limitations of coherence and coordination in time-domain or spatial-domain motion decomposition,and effectively avoids the influence of large-value low-frequency motion on small-value high-frequency motion.The visualized results show that the generated movements are natural,beautiful,diverse,and closely synchronize with the music signal.It provides a new understanding of the connection between music and movement,and brings innovative application prospects to the field of musical performance.

关 键 词：跨模态生成 人体动作生成 频域分解 动作分解 音乐驱动生成 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]