非结构化表格文档数据抽取与组织模型研究  

Research on data extraction and organization model for unstructured form document

作　　者：张元鸣[1,2] 陈苗[1] 陆佳炜[1] 徐俊[1] 肖刚[1,2] 

机构地区：[1]浙江工业大学计算机科学与技术学院,浙江杭州310023 [2]浙江工业大学机械工程博士后流动站,浙江杭州310014

出　　处：《浙江工业大学学报》2016年第5期487-494,共8页Journal of Zhejiang University of Technology

基　　金：浙江省钱江人才计划D类项目(QJD1302009);浙江省重大科技专项资助项目(2014C01408)

摘　　要：针对现有文档数据抽取方法无法抽取多值属性且灵活性不高的问题,提出了一种面向非结构化表格文档的数据抽取方法与组织模型.在分析文档结构特征和数据流特征的基础上,定义了数据流生成基本规则,给出了一个基于规则的数据抽取流程,包括逻辑结构抽取、文档预处理、数据抽取和数据组织等主要步骤,设计实现了单值区域与多值区域数据抽取算法;从文档中抽取的数据被组织成适合于MapReduce分析的结构化数据模型,该模型能够为大数据分析提供模型支持.实验结果表明:该抽取方法具有较高的准确率与召回率,数据组织模型也能够有效地支持大数据分析.In order to improve the flexibility of data extraction from complex unstructured form document,this paper proposes an extraction approach and a structured data model.The document structural features and data flow characteristics of unstructured form document are studied in details.A set of data flow generation rules are concluded.A data extraction process including logical structure extraction,document preprocessing,data extraction,and data organization,is designed.Two extraction algorithms for single-value area and multi-value area of document are implemented.The extracted data are organized into a structured data model that is designed to facilitate data analysis by MapReduce.Experimental results show that the proposed approach has great accuracy rate and recall rate.The structured data model also can effectively meet the requirements of data analysis by MapReduce.

关 键 词：非结构化表格文档 数据抽取 结构化数据模型 数据分析 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]