基于Hadoop的水文时间序列相似性研究与应用  

Research and Application of Hydrological Time Series Similarity Based on Hadoop

作　　者：顾昕辰 万定生[1] 樊龙[1] 

机构地区：[1]河海大学计算机与信息学院,南京210098

出　　处：《计算机与数字工程》2014年第1期1-5,13,共6页Computer & Digital Engineering

基　　金：国家自然科学基金项目(编号:51079040);水利部948项目(编号:201016)资助

摘　　要：传统DTW算法复杂度高,特别当处理海量数据时,耗时长。为了从算法和实现手段两方面同时入手,提高DTW运算效率,提出基于Hadoop平台,以FastDTW方法实现的水文时间序列相似性查找方法。首先利用小波变换对数据去噪,接着对水文时间序列进行语义化,然后在Hadoop的MapReduce过程中调用FastDTW方法实现DTW距离的云计算,得出与查询序列最相似的匹配序列。通过实验与串行查找进行对比,验证该方法用时短,匹配效果好,能够满足实际应用需求。The traditional DTW algorithm has a high complexity.Especially when dealing with massive data,it takes too much time.In order to improve the efficiency of DTW algorithm optimizing the algorithm and by executing the program on a different platform,a new hydrological time series similarity search method is presented,which is based on Hadoop and uses FastDTW algorithm.In this method,wavelet transform is firstly used to remove noise in data,secondly it does the semantization of hydrological time series,then it calls the FastDTW method during Hadoop＇s MapReduce process to achieve the cloud computing of DTW distance.After comparing with serial computing method,it is proved that the proposed method is applicable as it takes less time and the match result is satisfactory.

关 键 词：HADOOP FastDTW方法 水文时间序列 相似度 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]