公交场景下数据质量提升策略研究  

Research on data quality improvement strategies in the bus scenario

作　　者：佀淑燕 熊文 Si Shuyan;Xiong Wen(School of Information,Yunnan Normal University,Kunming 650000,China)

机构地区：[1]云南师范大学信息学院,云南昆明650000

出　　处：《无线互联科技》2023年第22期161-164,共4页Wireless Internet Technology

基　　金：国家自然科学基金项目,项目名称:城市交通大数据平台基准测试和性能优化关键技术研究,项目编号:61862066。

摘　　要：文章首先以文献计量分析的方式对国内外数据质量相关研究进行了全面系统地梳理,归纳出6类数据质量问题:数据冗余、数据缺失、数据异常、数据错误、数据冲突和数据稀疏;以一个公交大数据平台为例,针对GPS数据异常和进出站数据缺失,着重研究了基于启发式噪声过滤清洗GPS异常的方法(HNFC)以及基于多源数据融合的公交进出站信息补全的清洗方法(MDFC)。实验结果表明,HNFC可过滤掉2.48%的范围异常,修正2.25%的跳跃异常;而MDFC可以保证进出站时刻的平均误差稳定在1.0%以下。研究成果可为公交行业数据质量提升提供指导和借鉴。This paper first comprehensively and systematically reviews domestic and international research on data quality using bibliometric analysis,summarizing six types of data quality problems:data redundancy,data missing,data anomalies,data errors,data conflicts and data sparsity.Then,taking a bus big data platform as an example,this paper focuses on the study of a heuristic noise filtering method for cleaning GPS anomalies(HNFC)and a multi-source data fusion method for cleaning bus in-out station information completion(MDFC)for GPS data anomalies and in-out station data missing.Experimental results show that HNFC can filter out 2.48%of range anomalies and correct 2.25%of jump anomalies,while MDFC can ensure that the average error of in-out station time is stable at less than 1.0%.The research results of this paper can provide guidance and reference for data governance and quality improvement in the bus industry.

关 键 词：数据质量 脏数据 数据清洗 数据异常 数据缺失 

分 类 号：TP311.13[自动化与计算机技术—计算机软件与理论]