基于音视频多模态数据感知的智能巡检系统设计与应用  

Design and Application of an Intelligent Inspection System Based on Audio and Video Multimodal Data Perception

作　　者：李阳 丘建培 宋坤 LI Yang;QIU Jianpei;SONG Kun(Gansu Company of China Huadian Co.,Ltd.,Lanzhou 730300,China;Fujian Company of China Huadian Co.,Ltd.,Fuzhou 350013,China;Nanjing Nanzi Information Technology Co.,Ltd.,Nanjing 210000,China)

机构地区：[1]中国华电集团有限公司甘肃分公司,甘肃兰州730300 [2]中国华电集团有限公司福建分公司,福建福州350013 [3]南京南自信息技术有限公司,江苏南京210000

出　　处：《现代信息科技》2025年第3期189-193,共5页Modern Information Technology

基　　金：中国华电集团有限公司新能源智慧生产管理平台项目(CHDER/ZLJL1208-2022/A/0)。

摘　　要：为弥补巡检机器人和无人机在新能源发电企业中巡检范围有限的不足,文章设计了面向新能源发电领域的音视频融合智能巡检系统。在场站感知侧,综合应用各类摄像机、音频传感器等感知设备采集风电机组、线路、光伏、升压站各区域的音视频多模态数据。经过基于深度学习的智能算法分析,数据被转化为设备状态识别结果。在区域中心应用侧,部署智能巡检应用服务,展示场站侧设备状态识别结果,并构建巡检任务管理、告警管理等智能巡检业务功能,便于设备运维人员操作。文章所提出的音视频融合的感知方案,以及YOLO、ResNet、DeepLab等算法框架的集成应用,扩大了新能源场站智能巡检点位的覆盖范围,减少了运维人员前往现场的频次,有助于推动新能源无人场站的运维管理模式落地。To compensate for the shortage of limited inspection range of inspection robots and UAVs in new energy power generation enterprises,this paper designs an audio and video fusion intelligent inspection system for the field of new energy power generation.On the station perception side,various types of cameras,audio sensors,and other perception equipment are comprehensively applied to collect audio and video multimodal data of wind turbines,power lines,photovoltaics,and booster stations in various areas.After the analysis of intelligent algorithm based on Deep Learning,the data is converted into equipment status recognition results.On the application side of the regional center,it deploys intelligent inspection application service to display the recognition results of equipment status on the station side,and builds intelligent inspection business functions such as inspection task management and alarm management,so as to facilitate the operation of equipment maintenance personnel.The perception scheme of audio and video fusion proposed in this paper,and the integrated application of algorithm frameworks such as YOLO,ResNet and DeepLab,expand the coverage of intelligent inspection points in new energy stations,reduce the frequency of personnel going to the site,and help to promote the implementation of the operation and maintenance management mode of new energy power unmanned stations.

关 键 词：智能巡检 边缘计算 感知设备 多模态数据 深度学习 

分 类 号：TP273[自动化与计算机技术—检测技术与自动化装置] TP391.4[自动化与计算机技术—控制科学与工程]