一种编解码结构的车牌图像超分辨率网络  

An Encoder-Decoder-Based Super Resolution Network for License Plate Images

作　　者：徐胜军[1,2] 邓博文 史亚 孟月波 刘光辉[1,2] 韩九强 XU Shengjun;DENG Bowen;SHI Ya;MENG Yuebo;LIU Guanghui;HAN Jiuqiang(School of Information and Control Engineering,Xi’an University of Architecture and Technology,Xi’an 710055,China;Xi’an Key Laboratory of Building Manufacturing Intelligent&Automation Technology,Xi’an 710055,China;Faculty of Electronic and Information Engineering,Xi’an Jiaotong University,Xi’an 710049,China)

机构地区：[1]西安建筑科技大学信息与控制工程学院,西安710055 [2]西安市建筑制造智动化技术重点实验室,西安710055 [3]西安交通大学电子与信息学部,西安710049

出　　处：《西安交通大学学报》2022年第10期101-110,共10页Journal of Xi'an Jiaotong University

基　　金：陕西省自然科学基础研究计划资助项目(2020JM472,2020JM473);陕西省重点研发计划资助项目(2021SF-429)。

摘　　要：针对复杂实际场景中模糊、污损、扭曲、倾斜等车牌图像关键信息缺失以及新能源车牌背景与字符对比度低难以识别的问题,提出了一种编解码结构的车牌图像超分辨率网络。首先,构建一种基于编解码结构的车牌重构生成器网络,利用编码器对车牌图像的纹理、字符等特征进行提取,解码器对车牌特征进行重构;然后,设计一种基于语义监督的判别器网络,在网络损失中引入了对抗损失与CTC(connectionist temporal classification)损失,增强生成器网络对车牌图像语义特征的表征能力;最后,基于VGG16网络提取车牌顶角点特征,利用坐标变换方法对车牌图像进行矫正,进一步提高重构清晰度与识别准确率。采用所提网络在自建XAUAT-Parking数据集和公开CCPD数据集上进行超分辨率重构与识别实验,结果表明:所提网络在CCPD数据集上的平均峰值信噪比可达25.5 dB,结构相似性(SSIM)可达0.989;在XAUAT-Parking数据集上峰值信噪比可达26.6 dB,结构相似性可达0.997。研究结果表明,该网络有较好的车牌图像超分辨率重建效果,而且对车牌关键信息缺失问题具有较强的鲁棒性。An encoder-decoder-based super resolution network for license plate images was proposed for the lack of key information on license plate images caused by blur,stain,damage,distortion,and tilt in complex actual scenes and for the recognition difficulty due to low contrast between the license plate background and characters of new-energy vehicles.Firstly,a license plate reconstruction generator network based on the encoder-decoder structure is constructed.The texture and characters of the license plate image are extracted by an encoder,and the license plate features are reconstructed by a decoder.Then,a discriminator network based on semantic supervision is designed,and the adversarial loss and CTC loss are introduced into the network loss to enhance the ability of the generator network to represent the semantic features of license plate images.Finally,the features of the vertex points of the license plate are extracted based on VGG16 network,and a coordinate transformation method is utilized to correct the license plate image and further improve the reconstruction quality and recognition accuracy.Super-resolution reconstruction and recognition tests were performed on the self-built XAUAT-Parking dataset and the public CCPD dataset with the proposed network.The test results showed that the proposed network has an average peak signal to noise ratio(PSNR)of 25.5 dB and a structural similarity(SSIM)of 0.989 on the CCPD dataset.The PSNR and SSIM on the XAUAT-Parking dataset can reach 26.6 dB and 0.997 respectively.According to the research results,the proposed network has a good super-resolution reconstruction effect of license plate images and a strong robustness to the missing of key license plate information.

关 键 词：车牌图像 超分辨率 图像矫正 VGG16网络 编解码结构 生成对抗网络 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]