面向电力设备缺陷检测的多模态层次化分类  

Multi-modal hierarchical classification for power equipment defect detection

作　　者：白艳峰 王立彪 高卫东 马应龙[1] Bai Yanfeng;Wang Libiao;Gao Weidong;Ma Yinglong(School of Control and Computer Engineering,North China Electric Power University,Beijing 102206,China)

机构地区：[1]华北电力大学控制与计算机工程学院,北京102206

出　　处：《中国图象图形学报》2024年第7期2011-2023,共13页Journal of Image and Graphics

基　　金：国家自然科学基金项目(62072450);国家电网公司科技项目(SGGSXT00XMJS2250023)。

摘　　要：目的 电力设备的状态检测和故障维护是保障电力系统正常运行的重要基础。针对目前多数变电站存在电力设备缺陷类型复杂且现有的单分类缺陷检测方法无法满足电力设备的多标签分类缺陷检测需求的问题,提出一种面向电力设备缺陷检测的多模态层次化分类方法。方法 首先采集来自多个变电站的电力设备缺陷图像并进行人工标注、数据增强及归一化等预处理,构建了一个具有层次标签结构的电力设备缺陷图像数据集。然后提出一种基于多模态特征融合的层次化分类模型,采用ResNet50网络对图像进行特征提取,利用区域生成网络对目标进行定位以及前景、背景预测;为避免对区域生成网络生成的位置坐标进行量化时引入误差,进一步采用ROI Align(region of interest align)方法连续操作,生成位置坐标。最后采用层次化分类,将父类别标签嵌入到当前层目标特征表示进行逐层缺陷分类,最后一层得到最终的缺陷检测结果。结果 在电力设备缺陷数据集和基准数据集上,与多标签分类电力设备缺陷检测方法和流行的常用目标检测算法进行对比实验。实验结果表明,模型对绝大部分设备缺陷类别的检测准确率最高,平均检测准确率达到86.4%,相比性能第2的模型,准确率提升了5.1%,并且在基准数据集上的平均检测准确率也提高了1.1%~3%。结论 提出的电力设备缺陷检测方法充分利用设备缺陷标签的语义信息、层次结构和设备缺陷数据的图像特征,通过多模态层次化分类模型,能够提升电力设备缺陷检测的准确率。Objective Safety state detection of power equipment is a fundamental task to ensure the safe operation of power systems.The state detection and fault maintenance of power equipment are the basic prerequisites for ensuring the normal operation of the power system.With the growing diversities and complexity of defects in substations,the current defect recognition and power detection has increasingly been required to handle multi-label classification tasks based on a large number of closely related defect labels.However,due to the complex types of power equipment defects in most substations,most existing approaches for power equipment defect detection are inefficient at multi-label defect detection because the defect category labels often have different granularities in their semantic concepts and are often closely related with each other.All these problems cause existing defect detection methods to have difficulty meeting the requirements of multi-label classification-based defect detection tasks of power equipment.To address these problems,this paper proposes a multimodal hierarchical classification for power equipment defect detection,which is suitable for defect detection in complex power equipment environments.Method We propose a multi-modal hierarchical classification method,which fuses the feature information of defect images,hierarchical structure information,and the semantic information of category labels.First,defect images of power equipment from multiple substations are collected and preprocessed with manual annotation,data enhancement,and normalization to construct a power equipment defect image dataset with a hierarchical label structure.Then,a hierarchical classification model based on multi-modal feature fusion and hierarchical fine-tuning techniques is proposed,which uses the ResNet50 network to extract features from images,and a region proposal network to locate object and predict the foreground and background.The region of interest align(ROI Align) method is further used to continuously generate t

关 键 词：缺陷检测 图像识别 层次化分类 多模态特征融合 标签嵌入 区域特征聚集 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]