面向众包数据清洗的主动学习技术  

Active Learning Approach for Crowdsourcing-enhanced Data Cleaning

作　　者：叶晨[1] 王宏志[1] 高宏[1] 李建中[1] YE Chen;WANG Hong-Zhi;GAO Hong;LI Jian-Zhong(School of Computer Science and Technology,Harbin Institute of Technology,Harbin 150001,China)

机构地区：[1]哈尔滨工业大学计算机科学与技术学院,黑龙江哈尔滨150001

出　　处：《软件学报》2020年第4期1162-1172,共11页Journal of Software

基　　金：国家自然科学基金(U1509216,U1866602,61472099,61602129);国家重点研发计划(2016YFB1000703);黑龙江省留学归国人员科学基金(LC2016026)。

摘　　要：传统方法多数采用机器学习算法对数据进行清洗.这些方法虽然能够解决部分问题,但存在计算难度大、缺乏充足的知识等局限性.近年来,随着众包平台的兴起,越来越多的研究将众包引入数据清洗过程,通过众包来提供机器学习所需要的知识.由于众包的有偿性,研究如何将机器学习算法与众包有效且低成本结合在一起是必要的.提出了两种支持基于众包的数据清洗的主动学习模型,通过主动学习技术来减少众包开销,实现了对给定的数据集基于真实众包平台的数据清洗,最大程度减少成本的同时提高了数据的质量.在真实数据集上的实验结果验证了所提模型的有效性.Traditional methods usually adopt machine learning algorithms for data cleaning.Although these methods can solve some problems,there still are computational difficulties,lack of sufficient knowledge,and other limitations.In recent years,with the rise of the crowdsourcing,more and more research has introduced crowdsourcing into the process of data cleaning,providing the extra knowledge needed for machine learning.Since workers on the crowdsourcing platforms require to be paid,it is essential to study how to effectively combine machine learning algorithms with crowdsourcing on a limited budget.This study proposes two active learning models to support crowdsourcing-enhanced data cleaning.By using active learning technology to reduce crowdsourcing cost,data cleaning based on real crowdsourcing platform is realized for given data sets,which can reduce cost and improve data quality at the same time.Experimental results on the real-world datasets show the effectiveness of the proposed methods.

关 键 词：众包 数据清洗 主动学习 机器学习 领域专家 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]