Hadoop平台的微博热点事件挖掘  

Mining Hot Event from Microblog with Hadoop

作　　者：谢思发 林琛[1,2] 苏旋 江弋[1] 

机构地区：[1]厦门大学信息科学与技术学院,厦门361000 [2]厦门大学深圳研究院,广东深圳518000 [3]仟首网络科技有限公司,上海200000

出　　处：《小型微型计算机系统》2014年第4期797-801,共5页Journal of Chinese Computer Systems

基　　金：国家自然科学基金项目(61102136;61001013)资助;福建省自然科学基金项目(2011J05158)资助;深圳市科技创新基础研究项目(JCYJ20120618155655087)资助

摘　　要：微博作为一种新兴的网络社交服务,其即时通讯功能强大,用户可利用各种手段在微博上实时、快捷地发布社会热点事件.但是微博平台在短时间内发布大量信息的特点在一定程度上造成了信息的碎片化,而且迅速的信息更新速度易造成重要信息的不易检索.本文采用Hadoop平台,利用其在大数据挖掘方面的优势,提出挖掘微博中热点词的分布式算法,提取热点词组织热点事件,方便用户查询.此外提出了线性时间复杂度的检测算法,检测热点事件的爆发时间段.文中采用Twitter和新浪微博上的数据集作为测试样本,进行了大量的实验,实验结果表明本文算法能有效的提取微博中的热点事件.As a newly emerging social-networking service, Microhlog has a strong immediate communication function and can release hot issues of society rapidly by various methods. However, the huge mass of data releasing in a short time leads to the fragmentation of information to some extent. Moreover, the quick updating of information results in the difficulty of retrieving essential issues. In this paper, we propose a distributed algorithm of mining hot spots from Microhlog data based on Hadoop, which is superior in big da- ta mining, and detect hot issues according to the extracted spots for users＇ searching convenience. Furthermore, we put forward the detecting algorithm with a linear time complexity, detecting the time period of the burst of the hot issues. The experiments on Twitter and Sina Weibo show that our algorithm can extract hot issues from microblog effectively.

关 键 词：微博 HADOOP 分布式 热点事件 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]