An Online Visualization System for Streaming Log Data of Computing Clusters  

作　　者：Jing Xia Feiran Wu Fangzhou Guo Cong Xie Zhen Liu Wei Chen 

机构地区：[1]State Key Laboratory of CAD&CG,Zhejiang University [2]College of Computer Science and Technology,Hangzhou Dianzi University

出　　处：《Tsinghua Science and Technology》2013年第2期196-205,共10页清华大学学报（自然科学版（英文版）

基　　金：supported by the National Natural Science Foundation of China (Nos. 61232012 and 61202279);the National High-Tech Research and Development (863) Program of China (No. 2012AA120903);the Doctoral Fund of Ministry of Education of China (No. 20120101110134)

摘　　要：Monitoring a computing cluster requires collecting and understanding log data generated at the core, computer, and cluster levels at run time. Visualizing the log data of a computing cluster is a challenging problem due to the complexity of the underlying dataset： it is streaming, hierarchical, heterogeneous, and multi-sourced. This paper presents an integrated visualization system that employs a two-stage streaming process mode. Prior to the visual display of the multi-sourced information, the data generated from the clusters is gathered, cleaned, and modeled within a data processor. The visualization supported by a visual computing processor consists of a set of multivariate and time variant visualization techniques, including time sequence chart, treemap, and parallel coordinates. Novel techniques to illustrate the time tendency and abnormal status are also introduced. We demonstrate the effectiveness and scalability of the proposed system framework on a commodity cloud-computing platform.Monitoring a computing cluster requires collecting and understanding log data generated at the core, computer, and cluster levels at run time. Visualizing the log data of a computing cluster is a challenging problem due to the complexity of the underlying dataset： it is streaming, hierarchical, heterogeneous, and multi-sourced. This paper presents an integrated visualization system that employs a two-stage streaming process mode. Prior to the visual display of the multi-sourced information, the data generated from the clusters is gathered, cleaned, and modeled within a data processor. The visualization supported by a visual computing processor consists of a set of multivariate and time variant visualization techniques, including time sequence chart, treemap, and parallel coordinates. Novel techniques to illustrate the time tendency and abnormal status are also introduced. We demonstrate the effectiveness and scalability of the proposed system framework on a commodity cloud-computing platform.

关 键 词：computing cluster performance metrics monitoring streaming data VISUALIZATION 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术] TP301.6[自动化与计算机技术—计算机科学与技术]