A Survey on Preprocessing Methods for Web Usage Data
---
paper_title: Discovering More Accurate Frequent Web Usage Patterns

paper_content:

Web usage mining is a type of web mining, which exploits data mining techniques to discover valuable information from navigation behavior of World Wide Web users. As in classical data mining, data preparation and pattern discovery are the main issues in web usage mining. The first phase of web usage mining is the data processing phase, which includes the session reconstruction operation from server logs. Session reconstruction success directly affects the quality of the frequent patterns discovered in the next phase. In reactive web usage mining techniques, the source data is web server logs and the topology of the web pages served by the web server domain. Other kinds of information collected during the interactive browsing of web site by user, such as cookies or web logs containing similar information, are not used. The next phase of web usage mining is discovering frequent user navigation patterns. In this phase, pattern discovery methods are applied on the reconstructed sessions obtained in the first phase in order to discover frequent user patterns. In this paper, we propose a frequent web usage pattern discovery method that can be applied after session reconstruction phase. In order to compare accuracy performance of session reconstruction phase and pattern discovery phase, we have used an agent simulator, which models behavior of web users and generates web user navigation as well as the log data kept by the web server.
---
paper_title: Web usage mining with intentional browsing data

paper_content:

Many researches have developed Web usage mining (WUM) algorithms utilizing Web log records in order to discover useful knowledge to be used in supporting business applications and decision making. The quality of WUM in knowledge discovery, however, depends on the algorithm as well as on the data. This research explores a new data source called intentional browsing data (IBD) for potentially improving the effectiveness of WUM applications. IBD is a category of online browsing actions, such as ''copy'', ''scroll'', or ''save as,'' and is not recorded in Web log files. Consequently, the research aims to build a basic understanding of IBD which will lead to its easy adoption in WUM research and practice. Specifically, this paper formally defines IBD and clarifies its relationships with other browsing data via a proposed taxonomy. In order to make IBD available like Web log files, an online data collection mechanism for capturing IBD is also proposed and discussed. The potential benefits of IBD can be justified in terms of its enhancing and complementary effectiveness, which are illustrated by the rule implications of Web transaction mining algorithm for an EC application. Introducing IBD opens up the scope of WUM research and applications in knowledge discovery.
---
paper_title: Knowledge Discovery from Web Usage Data: Complete Preprocessing Methodology Summary

paper_content:

Summary The exponential growth of the Web in terms of Web sites and their users during the last decade has generated huge amount of data related to the user’s interactions with the Web sites. This data is recorded in the Web access log files of Web servers and usually referred as Web Usage Data (WUD). Knowledge Discovery from Web Usage Data (KDWUD) is that area of Web mining deals with the application of data mining techniques to extract interesting knowledge from the WUD. As Web sites continue to grow in size and complexity, the results of KDWUD have become very critical for efficient and effective management of the activities related to: e-business, eeducation, e-commerce, personalization, website design & management, network traffic analysis, the cache, the proxies, great diversity of Web pages in a site, search engine’s complexity, and to predict user’s actions. In this paper, we propose a complete preprocessing methodology, one of the important steps in KDWUD process. Several heuristics have been proposed for cleaning the WUD which is then aggregated and recorded in the relational data model. To validate the efficiency of the proposed preprocessing methodology, several experiments were conducted and the results shows that the proposed methodology reduces the size of Web access log files down to 73-82% of the initial size and offer richer logs that are structured for further stages of KDWUD.
---
paper_title: Exploration on Web Usage Mining and its Application

paper_content:

Along with the deepening development of e-commerce, it becomes important to know the user access mode. Through Web usage mining, the server log, registration information and other relative information left by user access can be mined with the user access mode which will provide foundation for decision-making of organizations. This paper proposes a SQL Server2000 based Web usage mining solution and illustrates with authentic examples on how to use DTS, T-SQL and other tools under SQL Server2000 to realize data transfer, data cleansing, user recognition, session recognition and other data pre-processing work, and illustrates on how to use Online Analysis and Process (OLAP) as well as Data Mining (DM) under SQL Server2000 as to realize mode discovery and mode analysis. This thesis is aimed at exploring a visual Web usage mining method.
---
paper_title: Knowledge Discovery from Users Web-Page Navigation

paper_content:

The authors propose to detect users' navigation paths to the advantage of Web site owners. First, they explain the design and implementation of a profiler which captures a client's selected links and page order, accurate page viewing time and cache references, using a Java based remote agent. The information captured by the profiler is then utilized by a knowledge discovery technique to cluster users with similar interests. They introduce a novel path clustering method based on the similarity of the history of user navigation. This approach is capable of capturing the interests of the user which could persist through several subsequent hypertext link selections. Finally, they evaluate their path clustering technique via a simulation study on a sample WWW site. They show that, depending on the level of inserted noise, they can recover the correct clusters by 10%-27% of average error margin.
---
paper_title: Web mining: information and pattern discovery on the World Wide Web

paper_content:

Application of data mining techniques to the World Wide Web, referred to as Web mining, has been the focus of several recent research projects and papers. However, there is no established vocabulary, leading to confusion when comparing research efforts. The term Web mining has been used in two distinct ways. The first, called Web content mining in this paper, is the process of information discovery from sources across the World Wide Web. The second, called Web usage mining, is the process of mining for user browsing and access patterns. We define Web mining and present an overview of the various research issues, techniques, and development efforts. We briefly describe WEBMINER, a system for Web usage mining, and conclude the paper by listing research issues.
---
paper_title: Discovering More Accurate Frequent Web Usage Patterns

paper_content:

Web usage mining is a type of web mining, which exploits data mining techniques to discover valuable information from navigation behavior of World Wide Web users. As in classical data mining, data preparation and pattern discovery are the main issues in web usage mining. The first phase of web usage mining is the data processing phase, which includes the session reconstruction operation from server logs. Session reconstruction success directly affects the quality of the frequent patterns discovered in the next phase. In reactive web usage mining techniques, the source data is web server logs and the topology of the web pages served by the web server domain. Other kinds of information collected during the interactive browsing of web site by user, such as cookies or web logs containing similar information, are not used. The next phase of web usage mining is discovering frequent user navigation patterns. In this phase, pattern discovery methods are applied on the reconstructed sessions obtained in the first phase in order to discover frequent user patterns. In this paper, we propose a frequent web usage pattern discovery method that can be applied after session reconstruction phase. In order to compare accuracy performance of session reconstruction phase and pattern discovery phase, we have used an agent simulator, which models behavior of web users and generates web user navigation as well as the log data kept by the web server.
---
paper_title: A graph-based web usage mining method considering client side data

paper_content:

To improve website, we need to evaluate current usage of it. Web usage mining and statistical analysis are two ways to evaluate usage of website. The combination of web usage mining and statistical analysis gives more accurate information about web usage. Through web usage mining methods, graph mining covers complex web browsing behaviors such as parallel browsing. Through statistical analysis methods, analyzing page browsing time gives valuable information about website and its users. This paper presents a web usage mining method which combines web usage mining and statistical analysis considering client side data. In other words, it combines graph based web usage mining and browsing time analysis with taking client side data into account. It helps us to reconstruct user session exactly as it has been and based on these data, we find web usage patterns with more accuracy.
---
paper_title: Web User Session Reconstruction Using Integer Programming

paper_content:

An important input for web usage mining is web user sessions that must be reconstructed from web logs (sessionization) when such sessions are not otherwise identified. We present a novel approach for sessionization based on an integer program. We compare results of our approach with the timeout heuristic on web logs from an academic web site. We find our integer program provides sessions that better match an expected empirical distribution with about half of the standard error of the heuristic.
---
paper_title: Smart Miner: a new framework for mining large scale web usage data

paper_content:

In this paper, we propose a novel framework called Smart-Miner for web usage mining problem which uses link information for producing accurate user sessions and frequent navigation patterns. Unlike the simple session concepts in the time and navigation based approaches, where sessions are sequences of web pages requested from the server or viewed in the browser, Smart Miner sessions are set of paths traversed in the web graph that corresponds to users' navigations among web pages. We have modeled session construction as a new graph problem and utilized a new algorithm, Smart-SRA, to solve this problem efficiently. For the pattern discovery phase, we have developed an efficient version of the Apriori-All technique which uses the structure of web graph to increase the performance. From the experiments that we have performed on both real and simulated data, we have observed that Smart-Miner produces at least 30% more accurate web usage patterns than other approaches including previous session construction methods. We have also studied the effect of having the referrer information in the web server logs to show that different versions of Smart-SRA produce similar results. Our another contribution is that we have implemented distributed version of the Smart Miner framework by employing Map/Reduce Paradigm. We conclude that we can efficiently process terabytes of web server logs belonging to multiple web sites by our scalable framework.
---
paper_title: Clickstream log acquisition with Web farming

paper_content:

Collecting customer interaction data on the e-business websites and portals will help to figure out customer behavior and build customer profile, and then perform personalized services. Traditional Web server log is hard to be associated with specific customer and impossible to log the complete actions and movements of customers across web-sites. Collecting clickstream log at the application layer with Web farming technology will help to seamlessly integrate Web usage data with other customer-related data. This model can be developed as a common plugin for most existing e-business websites and portals.
---
paper_title: Mining Weighted Frequent Patterns from Path Traversals on Weighted Graph

paper_content:

Summary A lot of real world problems can be modeled as traversals on graph, and mining from such traversals has been found useful in several applications. However, previous works considered only traversals on unweighted graph. This paper generalizes this to the case where vertices of graph are given weights to reflect their importance. Under such weight settings, traditional mining algorithms can not be adopted directly any more. To cope with the problem, this paper proposes new algorithms to discover weighted frequent patterns from the traversals. Specifically, we devise support bound paradigms for candidate generation and pruning during the mining process.
---
paper_title: Research on Path Completion Technique in Web Usage Mining

paper_content:

An implementation of data preprocessing system for Web usage mining and the details of algorithm for path completion are presented. After user session identification, the missing pages in user access paths are appended by using the referer-based method which is an effective solution to the problems introduced by using proxy servers and local caching. The reference length of pages in complete path is modified by considering the average reference length of auxiliary pages which is estimated in advance through the maximal forward references and the reference length algorithms. As verified by practical Web access log, the proposed path completion algorithm efficiently appends the lost information and improves the reliability of access data for further Web usage mining calculations.
---
paper_title: Optimal Algorithms for Generation of User Session Sequences Using Server Side Web User Logs

paper_content:

Identification of user session boundaries is one of the most important processes in the web usage mining for predictive prefetching of user next request based on their navigation behavior. This paper presents new techniques to identify user session boundaries by considering IPaddress, browsing agent, intersession and intrasession timeouts, immediate link analysis between referred pages and backward reference analysis without searching the whole tree representing the server pages. A complete set of user session sequences and the learning graph based on these user session sequences is also generated. Using this graph predictive prefetching is done. Comparison on the performance of the given approach with the existing reference length method and maximal reference method was done. Our analysis with different server's logs shows that our approach provides better results in terms of time complexity and precision to identify user session boundaries and also to generate all the relevant user session sequences.
---
paper_title: Knowledge Discovery from Users Web-Page Navigation

paper_content:

The authors propose to detect users' navigation paths to the advantage of Web site owners. First, they explain the design and implementation of a profiler which captures a client's selected links and page order, accurate page viewing time and cache references, using a Java based remote agent. The information captured by the profiler is then utilized by a knowledge discovery technique to cluster users with similar interests. They introduce a novel path clustering method based on the similarity of the history of user navigation. This approach is capable of capturing the interests of the user which could persist through several subsequent hypertext link selections. Finally, they evaluate their path clustering technique via a simulation study on a sample WWW site. They show that, depending on the level of inserted noise, they can recover the correct clusters by 10%-27% of average error margin.
---

Title: A Survey on Preprocessing Methods for Web Usage Data

Section 1: INTRODUCTION
Description 1: Provide an overview of web usage mining, its importance, and the process involved in it, including its various stages and types.

Section 2: DATA COLLECTION
Description 2: Describe the initial step of gathering relevant web data from various sources like server-side, client-side, and proxy servers.

Section 3: DATA PREPROCESSING
Description 3: Explain the process and methods involved in cleaning and transforming raw web log data into a set of user profiles, focusing on sub-tasks like data cleaning, user identification, session identification, and path completion.

Section 4: Data Cleaning
Description 4: Discuss techniques to remove irrelevant items from web logs to improve data quality, including handling of images, scripts, and status codes.

Section 5: User Identification
Description 5: Detail methods to identify individual users accessing a web site, addressing challenges with IP addresses, proxy servers, and sessions.

Section 6: Session Identification
Description 6: Describe the process of grouping user activities into sessions based on time-oriented and navigation-oriented heuristics, and various algorithms used for accurate session reconstruction.

Section 7: Path Completion
Description 7: Explain approaches to address missing pages in constructed transactions due to caching and proxy servers, ensuring complete user activity paths.

Section 8: PATTERN DISCOVERY AND ANALYSIS
Description 8: Review data mining techniques used to discover patterns in web usage data, including classification, association, clustering, and pattern analysis.

Section 9: WEB USAGE MINING APPLICATIONS
Description 9: Discuss various applications of web usage mining, such as personalization, e-commerce, system improvement, and site modification.

Section 10: FUTURE
Description 10: Identify challenges and potential future research directions in preprocessing of log data for web usage mining.

Section 11: CONCLUSION
Description 11: Summarize the significance of preprocessing in web usage mining and highlight the need for efficient session reconstruction for accurate user pattern analysis.