Big Data Mining using Map Reduce: A Survey Paper
---
paper_title: Data mining with big data

paper_content:

Big Data concern large-volume, complex, growing data sets with multiple, autonomous sources. With the fast development of networking, data storage, and the data collection capacity, Big Data are now rapidly expanding in all science and engineering domains, including physical, biological and biomedical sciences. This paper presents a HACE theorem that characterizes the features of the Big Data revolution, and proposes a Big Data processing model, from the data mining perspective. This data-driven model involves demand-driven aggregation of information sources, mining and analysis, user interest modeling, and security and privacy considerations. We analyze the challenging issues in the data-driven model and also in the Big Data revolution.
---
paper_title: Data mining with big data

paper_content:

Big Data concern large-volume, complex, growing data sets with multiple, autonomous sources. With the fast development of networking, data storage, and the data collection capacity, Big Data are now rapidly expanding in all science and engineering domains, including physical, biological and biomedical sciences. This paper presents a HACE theorem that characterizes the features of the Big Data revolution, and proposes a Big Data processing model, from the data mining perspective. This data-driven model involves demand-driven aggregation of information sources, mining and analysis, user interest modeling, and security and privacy considerations. We analyze the challenging issues in the data-driven model and also in the Big Data revolution.
---
paper_title: Scalable Data Cube Analysis over Big Data

paper_content:

Data cubes are widely used as a powerful tool to provide multidimensional views in data warehousing and On-Line Analytical Processing (OLAP). However, with increasing data sizes, it is becoming computationally expensive to perform data cube analysis. The problem is exacerbated by the demand of supporting more complicated aggregate functions (e.g. CORRELATION, Statistical Analysis) as well as supporting frequent view updates in data cubes. This calls for new scalable and efficient data cube analysis systems. In this paper, we introduce HaCube, an extension of MapReduce, designed for efficient parallel data cube analysis on large-scale data by taking advantages from both MapReduce (in terms of scalability) and parallel DBMS (in terms of efficiency). We also provide a general data cube materialization algorithm which is able to facilitate the features in MapReduce-like systems towards an efficient data cube computation. Furthermore, we demonstrate how HaCube supports view maintenance through either incremental computation (e.g. used for SUM or COUNT) or recomputation (e.g. used for MEDIAN or CORRELATION). We implement HaCube by extending Hadoop and evaluate it based on the TPC-D benchmark over billions of tuples on a cluster with over 320 cores. The experimental results demonstrate the efficiency, scalability and practicality of HaCube for cube analysis over a large amount of data in a distributed environment.
---
paper_title: Distributed cube materialization on holistic measures

paper_content:

Cube computation over massive datasets is critical for many important analyses done in the real world. Unlike commonly studied algebraic measures such as SUM that are amenable to parallel computation, efficient cube computation of holistic measures such as TOP-K is non-trivial and often impossible with current methods. In this paper we detail real-world challenges in cube materialization tasks on Web-scale datasets. Specifically, we identify an important subset of holistic measures and introduce MR-Cube, a MapReduce based framework for efficient cube computation on these measures. We provide extensive experimental analyses over both real and synthetic data. We demonstrate that, unlike existing techniques which cannot scale to the 100 million tuple mark for our datasets, MR-Cube successfully and efficiently computes cubes with holistic measures over billion-tuple datasets.
---
paper_title: Data Cube Materialization and Mining over MapReduce

paper_content:

Computing interesting measures for data cubes and subsequent mining of interesting cube groups over massive data sets are critical for many important analyses done in the real world. Previous studies have focused on algebraic measures such as SUM that are amenable to parallel computation and can easily benefit from the recent advancement of parallel computing infrastructure such as MapReduce. Dealing with holistic measures such as TOP-K, however, is nontrivial. In this paper, we detail real-world challenges in cube materialization and mining tasks on web-scale data sets. Specifically, we identify an important subset of holistic measures and introduce MR-Cube, a MapReduce-based framework for efficient cube computation and identification of interesting cube groups on holistic measures. We provide extensive experimental analyses over both real and synthetic data. We demonstrate that, unlike existing techniques which cannot scale to the 100 million tuple mark for our data sets, MR-Cube successfully and efficiently computes cubes with holistic measures over billion-tuple data sets.
---

Title: Big Data Mining using MapReduce: A Survey Paper

Section 1: Introduction
Description 1: Describe the rapid growth and complexity of Big Data, the challenges in managing, processing, and extracting useful information from it, and the role of MapReduce in addressing these challenges.

Section 2: Challenges In Big Data Mining
Description 2: Discuss various characteristics of Big Data that make it challenging to mine useful information, categorize the challenges into a three-tier framework, and describe each tier with its specific issues.

Section 3: Method Overview
Description 3: Provide an overview of the primary methodologies used in Big Data mining with MapReduce, including cube materialization, MapReduce programming model, and the MR-Cube approach.

Section 4: Data Cube
Description 4: Explain the concept of Data Cube, its significance in multi-dimensional data analysis, different techniques for cube computation, and the limitations of existing methods.

Section 5: MapReduce
Description 5: Describe the MapReduce programming model, its components (Map and Reduce functions), the role of YARN, and its applications in parallel and distributed data processing.

Section 6: MR-Cube Approach
Description 6: Detail the MR-Cube algorithm, its importance in efficient cube computation, the concept of partially algebraic measures, and the value partitioning technique used to optimize data distribution.

Section 7: Cube Materialization
Description 7: Discuss the process of cube materialization within the MR-Cube approach, the tasks involved, and how it helps in identifying interesting cube groups for mining algorithms.

Section 8: Conclusion
Description 8: Summarize the challenges of managing and mining Big Data, the effectiveness of the MapReduce framework (especially MR-Cube), and the significance of these technologies in providing accurate and real-time feedback for societal understanding.