Web使用挖掘系统研制中的主要问题和应对策略

With the rapid development of WWW,Web Usage Mining,as well as Web Mining,has become a hot direction in academic and industrial circles.It is generally believed that there are three tasks,preprocessing,knowledge discovery and pattern analysis,in Web Usage Mining.Though Web Usage Mining is still ranged in the application of traditional data mining techniques,in view of changes in application environment and operated data concerned,some new difficulties have arisen accordingly.This paper takes efforts to address such challenges in the three phases and introduces some proposed solutions simultaneously.     

Mining Cross-Transaction Web Usage Patterns

Web Usage Mining is the application of data mining techniques to large web log databases in order to extract usage patterns. However, most of the previous studies on usage patterns discovery just focus on mining intra-transaction associations, i.e., the associations among items within the same user＇s transactions, m cross-transaction association rule describes the association relationships among different users＇ transactions. In this paper, the closure property of frequent itemsets, which can determine the complete set of all frequent items exactly and is usually much smaller than the latter, is used to mine cross-transaction association rules from web log databases. We give the basic notion of frequent cross-transaction closed itemsets and prove the related necessary theories. And an efficient algorithm, i.e. MFCCPS（Mining Frequent Cross-Transaction Closed Pageviews Sets）, is designed and implemented. At last, an extensive experimental result on two synthetic datasets shows that our approach outperforms previous methods.     

Efficient Incremental Maintenance of Frequent Patterns with FP-Tree

Mining frequent patterns has been studied popularly in data mining area. However, little work has been done on mining patterns when the database has an influx of fresh data constantly. In these dynamic scenarios, efficient maintenance of the discovered patterns is crucial. Most existing methods need to scan the entire database repeatedly, which is an obvious disadvantage. In this paper, an efficient incremental mining algorithm, Incremental-Mining (IM), is proposed for maintenance of the frequent patterns when new incremental data come. Based on the frequent pattern tree (FP-tree) structure, IM gives a way to make the most of the things from the previous mining process, and requires scanning the original data once at most. Furthermore, IM can identify directly the differential set of frequent patterns, which may be more informative to users. Moreover, IM can deal with changing thresholds as well as changing data, thus provide a full maintenance scheme. IM has been implemented and the performance study shows it outperforms three other incremental algorithms: FUP, DB-tree and re-running frequent pattern growth (FP-growth).     

A fast algorithm for mining sequential patterns from large databases

Mining sequential patterns from large databases has been recognized by many researchers as an attractive task of data mining and knowledge discovery.Previous algorithms scan the databases for many times,which is often unendurable due to the very large amount of databases.In this paper,the authors introduce an effective algorithm for mining sequential patterns from large databases.In the algorithm,the original database is not used at all for counting the support of sequences after the first pass.Rather,a tidlist structure generated in the previous pass is employed for the purpose based on set intersection operations,avoiding the multiple scans of the databases.     

Analyzing Sequential Patterns in Retail Databases

Finding correlated sequential patterns in large sequence databases is one of the essential tasks in data mining since a huge number of sequential patterns are usually mined, but it is hard to find sequential patterns with the correlation. According to the requirement of real applications, the needed data analysis should be different. In previous mining approaches, after mining the sequential patterns, sequential patterns with the weak affinity are found even with a high minimum support. In this paper, a new framework is suggested for mining weighted support affinity patterns in which an objective measure, sequential ws-confidence is developed to detect correlated sequential patterns with weighted support affinity patterns. To efficiently prune the weak affinity patterns, it is proved that ws-confidence measure satisfies the anti-monotone and cross weighted support properties which can be applied to eliminate sequential patterns with dissimilar weighted support levels. Based on the framework, a weighted support affinity pattern mining algorithm （WSMiner） is suggested. The performance study shows that WSMiner is efficient and scalable for mining weighted support affinity patterns.     

网站结构和内容对Web使用挖掘的影响

The Paper emphasizes relativity between Web usage mining and the application of Web site structure and content.It has shown that the amount of effort involved in processing and quantifying the structure and content of a Web site is well worth in performing Web usage mining.The necessity of combining Web site structure and content with Web usage mining process is further proved.     

Extracting frequent connected subgraphs from large graph sets

Mining frequent patterns from datasets is one of the key success of data mining research. Currently,most of the studies focus on the data sets in which the elements are independent, such as the items in the marketing basket. However, the objects in the real world often have close relationship with each other. How to extract frequent patterns from these relations is the objective of this paper. The authors use graphs to model the relations, and select a simple type for analysis. Combining the graph theory and algorithms to generate frequent patterns, a new algorithm called Topology, which can mine these graphs efficiently, has been proposed.The performance of the algorithm is evaluated by doing experiments with synthetic datasets and real data. The experimental results show that Topology can do the job well. At the end of this paper, the potential improvement is mentioned.     

一种直接在Trans-树中挖掘频繁模式的新算法

Frequent pattern mining plays an essential role in many important data mining tasks. FP-growth is a very efficient algorithm for frequent pattern mining. However, it still suffers from creating conditional FP-tree separately and recursively during the mining process. In this paper, we propose a new algorithm, called Least-Item-First Pat-tern Growth (LIFPG), for mining frequent patterns. LIFPG mines frequent patterns directly in Trans-tree withoutusing any additional data structures. The key idea is that least items are always considered first when the current pat-tern growth. By this way, conditional sub-tree can be created directly in Trans-tree by adjusting node-links and re-counting counts of some nodes. Experiments show that, in comparison with FP-Growth, our algorithm is about fourtimes faster and saves half of memory;it also has good time and space scalability with the number of transactions,and has an excellent performance in dense dataset mining as well.     

基于WWW的聚类引擎研究

Today, search engine is the most commonly used tool for Web information retrieval, data mining may discover knowledge in large data. With the era of information and digital of media, Web data mining is becoming one of the hottest topics. By combining information retrieval technology with data mining technology, a prototype system of search engine is designed and implemented in this paper. It can group Web search results in a semantic, online and tree way, in order to help users find relevant Web information easier and faster.     

Web使用挖掘技术分析

介绍了一种Web挖掘的分类，包括Web内容挖掘、Web结构挖掘和Web使用挖掘。讨论了Web使用挖掘过程的三个步骤，即数据获取与数据预处理、模式发现和模式分析，详细分析了每一个步骤中所使用的技术。指出了目前Web使用挖掘研究存在的不足，给出了Web使用挖掘未来的研究方向。     

集成Web使用挖掘和内容挖掘的用户浏览兴趣迁移挖掘算法

提出了一种集成Web使用挖掘和内容挖掘的用户浏览兴趣迁移模式的模型和算法。介绍了Web页面及其聚类。通过替代用户事务中的页面为相应聚类的方法得到用户浏览兴趣序列。从用户浏览兴趣序列中得到用户浏览兴趣迁移模式。该模型对于网络管理者理解用户的行为特征和安排Web站点结构有较大的意义。     

Web模糊聚类方法及其应用

本文提出了Web模糊聚类的概念,给出了Web模糊聚美的过程模型WFCM并进行了详细阐述,沦述了Web模糊聚类在Web访问信息挖掘中,尤其是在Web用户聚类和Web页面聚类方面的应用．最后用实例证明了在Web页面聚类中使用Web模糊聚类的可行性。     

基于web挖掘的用户服务研究

数据丰富而知识贫乏导致了知识发现和数据挖掘领域的出现。基于Web的数据挖掘，是从Web海量的数据中自动、智能地抽取隐藏于这些数据中的知识，分析了Web挖掘技术的概念、特点、技术等。根据Web数据挖掘最流行的分类，可以分为Web内容挖掘、Web结构挖掘和Web使用记录挖掘。其中Web使用挖掘就是运用数据挖掘的思想来对服务器日志进行分析处理。该文根据Web数据挖掘的最近研究状况，主要论述了一个更新的频繁路径集的挖掘浏览模式在Web用户个性化服务中的应用，同时，还对发现的知识讨论了其在在线服务中的应用并给出了相应算法。     

Web数据挖掘

Web Mining is an important branch in Data Mining.It attracts more research interest for rapidly developing Internet. Web Mining includes(1)Web Content Mining;(g)Web Usage Mining;(3) Web structure Mining.In this paper we define Web Mining and present an overview of the various research issues,techniques and development efforts.     

一种N层体系结构下的Web挖掘应用

Web应用的深入使N层体系结构的系统逐渐被广泛使用,同时网上的海量信息也为Web挖掘提供了一个广阔的应用领域。本文针对在N层体系结构中应用Web挖掘技术进行了研究;包括在N层体系结构中实现网站用户访问分析、智能搜索引擎和个性化推荐等;对数据源的处理和转换、数据仓厍的建立和使用以及业务处理过程的改进等都进行了论述。     

一种新的Web频繁访问模式挖掘算法

提出了一种基于有向图的从Web日志中挖掘用户频繁访问模式的新算法,与传统使用基于关联规则挖掘的序列模式挖掘技术相比,本算法采用有向图来记录Web访问序列和它的计数,在挖掘过程中只需要扫描数据库一次,不产生数量庞大的候选模式,即可直接挖掘出所有的Web频繁访问路径,大大提高了Web访问模式的发现效率。     

基于BIPL的Web频繁访问模式挖掘

挖掘频繁访问模式是Web日志挖掘的一个重要任务。针对类Apriori算法和GITC算法的不足,提出了基于双亲链的单次扫描求交的Web频繁访问模式挖掘算法—BIPL,该算法首先对用户的访问模式两两进行交集运算,生成候选访问模式,并在求交集过程中保存各个候选访问模式的双亲模式,然后通过简单的求和运算,计算出各个候选访问模式的支持数。最后通过理论分析和实验验证,该算法是稳定的和高效的。     

Incremental and interactive mining of web traversal patterns

Web mining involves the application of data mining techniques to large amounts of web-related data in order to improve web services. Web traversal pattern mining involves discovering users’ access patterns from web server access logs. This information can provide navigation suggestions for web users indicating appropriate actions that can be taken. However, web logs keep growing continuously, and some web logs may become out of date over time. The users’ behaviors may change as web logs are updated, or when the web site structure is changed. Additionally, it can be difficult to determine a perfect minimum support threshold during the data mining process to find interesting rules. Accordingly, we must constantly adjust the minimum support threshold until satisfactory data mining results can be found.The essence of incremental data mining and interactive data mining is the ability to use previous mining results in order to reduce unnecessary processes when web logs or web site structures are updated, or when the minimum support is changed. In this paper, we propose efficient incremental and interactive data mining algorithms to discover web traversal patterns that match users’ requirements. The experimental results show that our algorithms are more efficient than other comparable approaches.