Teacher-student learning of generative adversarial network-guided diffractive neural networks for visual tracking and imaging  

作　　者：Hang Su Yanping He Baoli Li Haitao Luan Min Gu Xinyuan Fang 

机构地区：[1]University of Shanghai for Science and Technology,School of Artificial Intelligence Science and Technology,Shanghai,China [2]University of Shanghai for Science and Technology,Institute of Photonic Chips,Shanghai,China

出　　处：《Advanced Photonics Nexus》2024年第6期87-97,共11页先进光子学通讯(英文)

基　　金：supported by the National Natural Science Foundation of China(Grant Nos.62422509 and 62405188);the Shanghai Natural Science Foundation(Grant No.23ZR1443700);the Shuguang Program of Shanghai Education Development Foundation and Shanghai Municipal Education Commission(Grant No.23SG41);the Young Elite Scientist Sponsorship Program by CAST(Grant No.20220042);the Science and Technology Commission of Shanghai Municipality(Grant No.21DZ1100500);the Shanghai Municipal Science and Technology Major Project,and the Shanghai Frontiers Science Center Program(2021-2025 No.20).

摘　　要：Efficiently tracking and imaging interested moving targets is crucial across various applications,from autonomous systems to surveillance.However,persistent challenges remain in various fields,including environmental intricacies,limitations in perceptual technologies,and privacy considerations.We present a teacher-student learning model,the generative adversarial network(GAN)-guided diffractive neural network(DNN),which performs visual tracking and imaging of the interested moving target.The GAN,as a teacher model,empowers efficient acquisition of the skill to differentiate the specific target of interest in the domains of visual tracking and imaging.The DNN-based student model learns to master the skill to differentiate the interested target from the GAN.The process of obtaining a GAN-guided DNN starts with capturing moving objects effectively using an event camera with high temporal resolution and low latency.Then,the generative power of GAN is utilized to generate data with position-tracking capability for the interested moving target,subsequently serving as labels to the training of the DNN.The DNN learns to image the target during training while retaining the target’s positional information.Our experimental demonstration highlights the efficacy of the GAN-guided DNN in visual tracking and imaging of the interested moving target.We expect the GAN-guided DNN can significantly enhance autonomous systems and surveillance.

关 键 词：visual tracking diffractive neural network generative adversarial network teacher-student learning event-based camera optical machine learning 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]