Music-stylized hierarchical dance synthesis with user control  

作　　者：Yanbo CHENG Yichen JIANG Yingying WANG 

机构地区：[1]Department of Computing and Software,McMaster University,ON L8S 4L8,Canada

出　　处：《虚拟现实与智能硬件（中英文）》2024年第5期339-357,共19页Virtual Reality & Intelligent Hardware

基　　金：Supported by Startup Fund 20019495,McMaster University。

摘　　要：Background Synthesizing dance motions to match musical inputs is a significant challenge in animation research.Compared to functional human motions,such as locomotion,dance motions are creative and artistic,often influenced by music,and can be independent body language expressions.Dance choreography requires motion content to follow a general dance genre,whereas dance performances under musical influence are infused with diverse impromptu motion styles.Considering the high expressiveness and variations in space and time,providing accessible and effective user control for tuning dance motion styles remains an open problem.Methods In this study,we present a hierarchical framework that decouples the dance synthesis task into independent modules.We use a high-level choreography module built as a Transformer-based sequence model to predict the long-term structure of a dance genre and a low-level realization module that implements dance stylization and synchronization to match the musical input or user preferences.This novel framework allows the individual modules to be trained separately.Because of the decoupling,dance composition can fully utilize existing high-quality dance datasets that do not have musical accompaniments,and the dance implementation can conveniently incorporate user controls and edit motions through a decoder network.Each module is replaceable at runtime,which adds flexibility to the synthesis of dance sequences.Results Synthesized results demonstrate that our framework generates high-quality diverse dance motions that are well adapted to varying musical conditions and user controls.

关 键 词：Deep learning Character animation Motion synthesis Motion stylization Multimodal synchronization User control 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程] J70-05[自动化与计算机技术—控制科学与工程]