Big Data : A Survey The New Paradigms , Methodologies and Tools
---
paper_title: Big Data: the current wave front of the tsunami

paper_content:

In recent years, a real tsunami has flooded many human activities. Genomics, Astronomy, Particle Physics and Social Sciences are just a few examples of fields which have been intensively invaded by a massive amount of data coming from simulation, experiments or exploration. This huge pile of data requires a new way to deal with, a real paradigmatic shift respect to the past as for theories, technologies or approaches in data management. This work outlines the current wave front of Big Data, starting from a possible characterization of this new paradigm to its most compelling applications and tools, with an exploratory research of Big Data challenges in manufacturing engineering.
---
paper_title: The Unreasonable Effectiveness of Data

paper_content:

At Brown University, there is excitement of having access to the Brown Corpus, containing one million English words. Since then, we have seen several notable corpora that are about 100 times larger, and in 2006, Google released a trillion-word corpus with frequency counts for all sequences up to five words long. In some ways this corpus is a step backwards from the Brown Corpus: it's taken from unfiltered Web pages and thus contains incomplete sentences, spelling errors, grammatical errors, and all sorts of other errors. It's not annotated with carefully hand-corrected part-of-speech tags. But the fact that it's a million times larger than the Brown Corpus outweighs these drawbacks. A trillion-word corpus - along with other Web-derived corpora of millions, billions, or trillions of links, videos, images, tables, and user interactions - captures even very rare aspects of human behavior. So, this corpus could serve as the basis of a complete model for certain tasks - if only we knew how to extract the model from the data.
---
paper_title: Perspectives on Big Data

paper_content:

There is much hype associated with the term ‘Big Data’ (BD), and much opportunity in the data that are associated with that term, along with the tools and techniques in existence and being developed to leverage it for decision making and improving the condition of living beings, firms and society. However, many are not clear on what are, or what is meant by the term, ‘Big Data’. The focus of this research was to explore the meaning of BD and to identify important paths of research on BD. As part of this process, we called upon a diverse set of marketing scholars who possess expertise and special insights. We discovered that different communities have different perspectives. It could be argued that they are all correct as they reflect the preferred perspectives of different communities. We find it helpful to think of BD as a term that represents a period of time or era, a process, and data that are from a variety of sources, of various structures or forms, and in a variety of locations. Important research questions and issues related to BD are discussed.
---
paper_title: Hadoop: The Definitive Guide

paper_content:

Hadoop: The Definitive Guide helps you harness the power of your data. Ideal for processing large datasets, the Apache Hadoop framework is an open source implementation of the MapReduce algorithm on which Google built its empire. This comprehensive resource demonstrates how to use Hadoop to build reliable, scalable, distributed systems: programmers will find details for analyzing large datasets, and administrators will learn how to set up and run Hadoop clusters. Complete with case studies that illustrate how Hadoop solves specific problems, this book helps you: Use the Hadoop Distributed File System (HDFS) for storing large datasets, and run distributed computations over those datasets using MapReduce Become familiar with Hadoop's data and I/O building blocks for compression, data integrity, serialization, and persistence Discover common pitfalls and advanced features for writing real-world MapReduce programs Design, build, and administer a dedicated Hadoop cluster, or run Hadoop in the cloud Use Pig, a high-level query language for large-scale data processing Take advantage of HBase, Hadoop's database for structured and semi-structured data Learn ZooKeeper, a toolkit of coordination primitives for building distributed systems If you have lots of data -- whether it's gigabytes or petabytes -- Hadoop is the perfect solution. Hadoop: The Definitive Guide is the most thorough book available on the subject. "Now you have the opportunity to learn about Hadoop from a master-not only of the technology, but also of common sense and plain talk." -- Doug Cutting, Hadoop Founder, Yahoo!
---
paper_title: Big Data: A Survey

paper_content:

In this paper, we review the background and state-of-the-art of big data. We first introduce the general background of big data and review related technologies, such as could computing, Internet of Things, data centers, and Hadoop. We then focus on the four phases of the value chain of big data, i.e., data generation, data acquisition, data storage, and data analysis. For each phase, we introduce the general background, discuss the technical challenges, and review the latest advances. We finally examine the several representative applications of big data, including enterprise management, Internet of Things, online social networks, medial applications, collective intelligence, and smart grid. These discussions aim to provide a comprehensive overview and big-picture to readers of this exciting area. This survey is concluded with a discussion of open problems and future directions.
---
paper_title: Taming the Big Data Tidal Wave: Finding Opportunities in Huge Data Streams with Advanced Analytics

paper_content:

You receive an e-mail. It contains an offer for a complete personal computer system. It seems like the retailer read your mind since you were exploring computers on their web site just a few hours prior. As you drive to the store to buy the computer bundle, you get an offer for a discounted coffee from the coffee shop you are getting ready to drive past. It says that since youre in the area, you can get 10% off if you stop by in the next 20 minutes. As you drink your coffee, you receive an apology from the manufacturer of a product that you complained about yesterday on your Facebook page, as well as on the companys web site. Finally, once you get back home, you receive notice of a special armor upgrade available for purchase in your favorite online video game. It is just what is needed to get past some spots youve been struggling with. Sound crazy? Are these things that can only happen in the distant future? No. All of these scenarios are possible today! Big data. Advanced analytics. Big data analytics. It seems you cant escape such terms today. Everywhere you turn people are discussing, writing about, and promoting big data and advanced analytics. Well, you can now add this book to the discussion. What is real and what is hype? Such attention can lead one to the suspicion that perhaps the analysis of big data is something that is more hype than substance. While there has been a lot of hype over the past few years, the reality is that we are in a transformative era in terms of analytic capabilities and the leveraging of massive amounts of data. If you take the time to cut through the sometimes-over-zealous hype present in the media, youll find something very real and very powerful underneath it. With big data, the hype is driven by genuine excitement and anticipation of the business and consumer benefits that analyzing it will yield over time. Big data is the next wave of new data sources that will drive the next wave of analytic innovation in business, government, and academia. These innovations have the potential to radically change how organizations view their business. The analysis that big data enables will lead to decisions that are more informed and, in some cases, different from what they are today. It will yield insights that many can only dream about today. As youll see, there are many consistencies with the requirements to tame big data and what has always been needed to tame new data sources. However, the additional scale of big data necessitates utilizing the newest tools, technologies, methods, and processes. The old way of approaching analysis just wont work. It is time to evolve the world of advanced analytics to the next level. Thats what this book is about. Taming the Big Data Tidal Wave isnt just the title of this book, but rather an activity that will determine which businesses win and which lose in the next decade. By preparing and taking the initiative, organizations can ride the big data tidal wave to success rather than being pummeled underneath the crushing surf. What do you need to know and how do you prepare in order to start taming big data and generating exciting new analytics from it? Sit back, get comfortable, and prepare to find out!
---
paper_title: An integrated framework for evaluating big-data storage solutions - IDA case study

paper_content:

The amount of data stored is rapidly increasing due to consumer, business, scientific, and government generated content. In addition to keeping pace with storing generated data, there is a need to comply with laws and Service Level Agreements (SLA) to protect and preserve stored data. The issues of capacity and scale in data storage are of constant concern to ensure the ability to absorb data growth, to manage existing data and to analyze the data. Evaluating data storage for IT infrastructure is a complex task that has multiple variables. These variables include capacity, scalability, financial, workload requirements, security, privacy, availability, reliability, analytics and operational. Current frameworks that attempt to address data storage evaluations focus either on a single aspect of these variables or use generic IT frameworks to evaluate data storage as a sub-component. The complexity of data storage requirements merits a holistic framework in the data storage domain. The contribution of this paper is an integrated framework to evaluate and assist in selecting optimum storage solution for multi-variable requirements. This paper examines Information Dispersal Algorithm (IDA) storage technology using this framework and is the first in a series to examine four different big-data storage technologies using this framework.
---
paper_title: Benchmarking cloud serving systems with YCSB

paper_content:

While the use of MapReduce systems (such as Hadoop) for large scale data analysis has been widely recognized and studied, we have recently seen an explosion in the number of systems developed for cloud data serving. These newer systems address "cloud OLTP" applications, though they typically do not support ACID transactions. Examples of systems proposed for cloud serving use include BigTable, PNUTS, Cassandra, HBase, Azure, CouchDB, SimpleDB, Voldemort, and many others. Further, they are being applied to a diverse range of applications that differ considerably from traditional (e.g., TPC-C like) serving workloads. The number of emerging cloud serving systems and the wide range of proposed applications, coupled with a lack of apples-to-apples performance comparisons, makes it difficult to understand the tradeoffs between systems and the workloads for which they are suited. We present the "Yahoo! Cloud Serving Benchmark" (YCSB) framework, with the goal of facilitating performance comparisons of the new generation of cloud data serving systems. We define a core set of benchmarks and report results for four widely used systems: Cassandra, HBase, Yahoo!'s PNUTS, and a simple sharded MySQL implementation. We also hope to foster the development of additional cloud benchmark suites that represent other classes of applications by making our benchmark tool available via open source. In this regard, a key feature of the YCSB framework/tool is that it is extensible--it supports easy definition of new workloads, in addition to making it easy to benchmark new systems.
---
paper_title: The Hadoop Distributed File System

paper_content:

The Hadoop Distributed File System (HDFS) is designed to store very large data sets reliably, and to stream those data sets at high bandwidth to user applications. In a large cluster, thousands of servers both host directly attached storage and execute user application tasks. By distributing storage and computation across many servers, the resource can grow with demand while remaining economical at every size. We describe the architecture of HDFS and report on experience using HDFS to manage 25 petabytes of enterprise data at Yahoo!.
---
paper_title: Scalable SQL and NoSQL data stores

paper_content:

In this paper, we examine a number of SQL and socalled "NoSQL" data stores designed to scale simple OLTP-style application loads over many servers. Originally motivated by Web 2.0 applications, these systems are designed to scale to thousands or millions of users doing updates as well as reads, in contrast to traditional DBMSs and data warehouses. We contrast the new systems on their data model, consistency mechanisms, storage mechanisms, durability guarantees, availability, query support, and other dimensions. These systems typically sacrifice some of these dimensions, e.g. database-wide transaction consistency, in order to achieve others, e.g. higher availability and scalability.
---
paper_title: Mining data streams: a review

paper_content:

The recent advances in hardware and software have enabled the capture of different measurements of data in a wide range of fields. These measurements are generated continuously and in a very high fluctuating data rates. Examples include sensor networks, web logs, and computer network traffic. The storage, querying and mining of such data sets are highly computationally challenging tasks. Mining data streams is concerned with extracting knowledge structures represented in models and patterns in non stopping streams of information. The research in data stream mining has gained a high attraction due to the importance of its applications and the increasing generation of streaming information. Applications of data stream analysis can vary from critical scientific and astronomical applications to important business and financial ones. Algorithms, systems and frameworks that address streaming challenges have been developed over the past three years. In this review paper, we present the state-of-the-art in this growing vital field.
---
paper_title: The Hadoop Distributed File System

paper_content:

The Hadoop Distributed File System (HDFS) is designed to store very large data sets reliably, and to stream those data sets at high bandwidth to user applications. In a large cluster, thousands of servers both host directly attached storage and execute user application tasks. By distributing storage and computation across many servers, the resource can grow with demand while remaining economical at every size. We describe the architecture of HDFS and report on experience using HDFS to manage 25 petabytes of enterprise data at Yahoo!.
---
paper_title: A multimedia ontology model based on linguistic properties and audio-visual features

paper_content:

The exponential growth of informative contents needs intelligent information systems able to use data to create information. To aim this goal, these systems should have formal models to represent knowledge. In this way complex data can be managed and used to perform new tasks and implement innovative functionalities. This article describes a general and formal ontology model to represent knowledge using multimedia data and linguistic properties to bridge the gap between the target semantic classes and the available low-level multimedia descriptors. This model has been implemented in a system to edit, manage and share ontology in the WEB. The system provides a graphical interface to add multimedia objects by means of user interaction. The multimedia features are automatically extracted using algorithms based on MPEG-7 descriptors.
---
paper_title: A Classification of Ontology Change

paper_content:

The problem of modifying an ontology in response to a certain need for change is a complex and multifaceted one, being addressed by several different, but closely related and often overlapping research disciplines. Unfortunately, the boundaries of each such discipline are not clear, as certain terms are often used with different meanings in the relevant literature. The purpose of this paper is to identify the exact relationships, connections and overlaps between these research areas and determine the boundaries of each field, by performing a broad review of the relevant literature.
---
paper_title: Information Retrieval from the Web: an Interactive Paradigm

paper_content:

Information retrieval is moving beyond the stage where users simply type one or more keywords and retrieve a ranked list of documents. In such a scenario users have to go through the returned documents in order to find what they are actually looking for. More often they would like to get targeted answers to their queries without extraneous information, even if their requirements are not well specified. In this paper we propose an approach for designing a web retrieval system able to find the desired information through several interactions with the users. The proposed approach allows to overcome the problems deriving from ambiguous or too vague queries, using semantic search and topic detection techniques. The results of the very first experiments on a prototype system are also reported.
---
paper_title: A content-based approach for document representation and retrieval

paper_content:

In the last few years, the problem of defining efficient techniques for knowledge representation is becoming a challenging topic in both academic and industrial community. The large amount of available data creates several problems in terms of information overload. In this framework, we assume that new approaches for knowledge definition and representation may be useful, in particular the ones based on the concept of ontology. In this paper we propose a suitable model for knowledge representation purposes using linguistic concepts and properties. We implement our model in a system which, using novel techniques and metrics, analyzes documents from a semantic point of view using as context of interest the Web. Experiments are performed on a test set built using a directory service to have information about analyzed documents. The obtained results compared with other similar systems show an effective improvement.
---
paper_title: Partitioning of ontologies driven by a structure-based approach

paper_content:

In this paper, we propose a novel structure-based partitioning algorithm able to break a large ontology into different modules related to specific topics for the domain of interest. In particular, we leverage the topological properties of the ontology graph and exploit several techniques derived from Network Analysis to produce an effective partitioning without considering any information about semantics of ontology relationships. An automated partitioning tool has been developed and several preliminary experiments have been conducted to validate the effectiveness of our approach with respect to other techniques.
---
paper_title: Bigtable: A Distributed Storage System for Structured Data

paper_content:

Bigtable is a distributed storage system for managing structured data that is designed to scale to a very large size: petabytes of data across thousands of commodity servers. Many projects at Google store data in Bigtable, including web indexing, Google Earth, and Google Finance. These applications place very different demands on Bigtable, both in terms of data size (from URLs to web pages to satellite imagery) and latency requirements (from backend bulk processing to real-time data serving). Despite these varied demands, Bigtable has successfully provided a flexible, high-performance solution for all of these Google products. In this paper we describe the simple data model provided by Bigtable, which gives clients dynamic control over data layout and format, and we describe the design and implementation of Bigtable.
---

Title: Big Data: A Survey of New Paradigms, Methodologies, and Tools

Section 1: INTRODUCTION
Description 1: This section provides an overview of the increasing availability of data from various sources and introduces the concept of Big Data. It also highlights the challenges and opportunities associated with managing large data sets and the importance of selecting the right tools and methodologies.

Section 2: BIG DATA DIMENSIONS AND TECHNOLOGICAL SOLUTIONS
Description 2: This section discusses the various dimensions of Big Data, such as Volume, Velocity, and Variety, and presents technological solutions to address the challenges associated with these dimensions. It also explores additional dimensions introduced in the literature, such as viscosity, variability, veracity, and volatility.

Section 3: A FRAMEWORK FOR A QUALITATIVE EVALUATION OF BIG DATA SOLUTIONS
Description 3: This section introduces a framework for the qualitative evaluation of Big Data tools and solutions. It identifies various technical challenges in the Big Data life cycle, such as data heterogeneity, scalability, timeliness, privacy, and visualization.

Section 4: Relational DBMS and SQL Language
Description 4: This section describes the characteristics of traditional Relational Database Management Systems (RDBMS) and the SQL language. It highlights the benefits and limitations of using RDBMS for managing large data sets.

Section 5: New SQL
Description 5: This section introduces NewSQL, a class of modern relational database management systems that combine the scalability of NoSQL systems with the ACID properties of traditional RDBMS. It discusses the main features and advantages of NewSQL systems.

Section 6: The Survey of the Analyzed Solutions
Description 6: This section provides a comprehensive survey of the most widely used NoSQL and NewSQL solutions. It evaluates each solution based on criteria such as scalability, consistency, failover support, and available APIs. The section includes descriptions of various Big Data tools like Google BigTable, MongoDB, Neo4j, Apache CouchDB, and others.

Section 7: CONCLUSIONS
Description 7: This section summarizes the major findings of the survey, highlighting the flexibility and multi-platform support of many Big Data tools. It also underscores the significant interest from the developer community in open-source projects and concludes with implications for future research and development in Big Data technologies.