基于Hadoop的Web日志挖掘

基于单一节点的数据挖掘系统在挖掘Web海量数据源时存在计算瓶颈，针对该问题，利用云计算的分布式处理和虚拟化技术的优势，设计一种基于云计算的Hadoop集群框架的Web日志分析平台，提出一种能够在云计算环境中进行分布式处理的混合算法。为进一步验证该平台的高效性，在该平台上利用改进后的算法挖掘Web日志中用户的偏爱访问路径。实验结果表明，在集群中运用分布式算法处理大量的Web日志文件，可以明显提高Web数据挖掘的效率。

Weblog Mining Based on Hadoop

The mass data from Web are distributed,heterogeneous and dynamic,so the current data mining system based on single node has developed to a bottleneck.Using the advantage of cloud computing——distributed processing and virtualization,this paper presents a Weblog analysis platform under the Hadoop＇s cluster framework based on cloud computing,it also presents a hybrid algorithm which can distributed process in the cloud computing environment.To further verify the effectiveness and efficiency of the platform,it uses the improved algorithm to mine users＇ preferred access path in Weblog on the platform.Experimental results show that,using distributed algorithm to process large number of Weblog files in the cluster,can significantly improve the efficiency of Web data mining.