Cogeneration of Innovative Audio-visual Content: A New Challenge for Computing Art  

作　　者：Mengting Liu Ying Zhou Yuwei Wu Feng Gao 

机构地区：[1]Advanced Institute of Information Technology,Peking University,Hangzhou,311215,China [2]School of Electronic and Computer Engineering,Shenzhen Graduate School,Peking University,Shenzhen 518000,China [3]Peng Cheng Laboratory,Shenzhen 518000,China [4]School of Arts,Peking University,Beijing 100871,China

出　　处：《Machine Intelligence Research》2024年第1期4-28,共25页机器智能研究（英文版）

基　　金：This work was supported by National Natural Science Foundation of China(No.62176006);the National Key Research and Development Program of China(No.2022YFF0902302).

摘　　要：In recent years,computing art has developed rapidly with the in-depth cross study of artificial intelligence generated con-tent(AIGC)and the main features of artworks.Audio-visual content generation has gradually been applied to various practical tasks,including video or game score,assisting artists in creation,art education and other aspects,which demonstrates a broad application pro-spect.In this paper,we introduce innovative achievements in audio-visual content generation from the perspective of visual art genera-tion and auditory art generation based on artificial intelligence(Al).We outline the development tendency of image and music datasets,visual and auditory content modelling,and related automatic generation systems.The objective and subjective evaluation of generated samples plays an important role in the measurement of algorithm performance.We provide a cogeneration mechanism of audio-visual content in multimodal tasks from image to music and display the construction of specific stylized datasets.There are still many new op-portunities and challenges in the field of audio-visual synesthesia generation,and we provide a comprehensive discussion on them.

关 键 词：Artificial intelligence(AI)art AUDIO-VISUAL artificial intelligence generated content(AIGC) MULTIMODAL artistic evalu-ation 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程] TP39[自动化与计算机技术—控制科学与工程]