A Survey of Data Management System for Cloud Computing: Models and Searching Methods
---
paper_title: Semantics Centric Solutions for Application and Data Portability in Cloud Computing

paper_content:

Cloud computing has become one of the key considerations both in academia and industry. Cheap, seemingly unlimited computing resources that can be allocated almost instantaneously and pay-as-you-go pricing schemes are some of the reasons for the success of Cloud computing. The Cloud computing landscape, however, is plagued by many issues hindering adoption. One such issue is vendor lock-in, forcing the Cloud users to adhere to one service provider in terms of data and application logic. Semantic Web has been an important research area that has seen significant attention from both academic and industrial researchers. One key property of Semantic Web is the notion of interoperability and portability through high level models. Significant work has been done in the areas of data modeling, matching, and transformations. The issues the Cloud computing community is facing now with respect to portability of data and application logic are exactly the same issue the Semantic Web community has been trying to address for some time. In this paper we present an outline of the use of well established semantic technologies to overcome the vendor lock-in issues in Cloud computing. We present a semantics-centric programming paradigm to create portable Cloud applications and discuss MobiCloud, our early attempt to implement the proposed approach.
---
paper_title: The RDF-3X engine for scalable management of RDF data

paper_content:

RDF is a data model for schema-free structured information that is gaining momentum in the context of Semantic-Web data, life sciences, and also Web 2.0 platforms. The "pay-as-you-go" nature of RDF and the flexible pattern-matching capabilities of its query language SPARQL entail efficiency and scalability challenges for complex queries including long join paths. This paper presents the RDF-3X engine, an implementation of SPARQL that achieves excellent performance by pursuing a RISC-style architecture with streamlined indexing and query processing. The physical design is identical for all RDF-3X databases regardless of their workloads, and completely eliminates the need for index tuning by exhaustive indexes for all permutations of subject-property-object triples and their binary and unary projections. These indexes are highly compressed, and the query processor can aggressively leverage fast merge joins with excellent performance of processor caches. The query optimizer is able to choose optimal join orders even for complex queries, with a cost model that includes statistical synopses for entire join paths. Although RDF-3X is optimized for queries, it also provides good support for efficient online updates by means of a staging architecture: direct updates to the main database indexes are deferred, and instead applied to compact differential indexes which are later merged into the main indexes in a batched manner. Experimental studies with several large-scale datasets with more than 50 million RDF triples and benchmark queries that include pattern matching, manyway star-joins, and long path-joins demonstrate that RDF-3X can outperform the previously best alternatives by one or two orders of magnitude.
---
paper_title: Enabling search services on outsourced private spatial data

paper_content:

Cloud computing services enable organizations and individuals to outsource the management of their data to a service provider in order to save on hardware investments and reduce maintenance costs. Only authorized users are allowed to access the data. Nobody else, including the service provider, should be able to view the data. For instance, a real-estate company that owns a large database of properties wants to allow its paying customers to query for houses according to location. On the other hand, the untrusted service provider should not be able to learn the property locations and, e.g., selling the information to a competitor. To tackle the problem, we propose to transform the location datasets before uploading them to the service provider. The paper develops a spatial transformation that re-distributes the locations in space, and it also proposes a cryptographic-based transformation. The data owner selects the transformation key and shares it with authorized users. Without the key, it is infeasible to reconstruct the original data points from the transformed points. The proposed transformations present distinct trade-offs between query efficiency and data confidentiality. In addition, we describe attack models for studying the security properties of the transformations. Empirical studies demonstrate that the proposed methods are efficient and applicable in practice.
---
paper_title: REST-based management of loosely coupled services

paper_content:

Applications increasingly make use of the distributed platform that the World Wide Web provides - be it as a Software-as-a-Service such as salesforce.com, an application infrastructure such as facebook.com, or a computing infrastructure such as a "cloud". A common characteristic of applications of this kind is that they are deployed on infrastructure or make use of components that reside in different management domains. Current service management approaches and systems, however, often rely on a centrally managed configuration management database (CMDB), which is the basis for centrally orchestrated service management processes, in particular change management and incident management. The distribution of management responsibility of WWW based applications requires a decentralized approach to service management. This paper proposes an approach of decentralized service management based on distributed configuration management and service process co-ordination, making use RESTful access to configuration information and ATOM-based distribution of updates as a novel foundation for service management processes.
---
paper_title: IRain: A Personal Storage Cloud for Integrating Web Data Services

paper_content:

In this paper, we design and implement IRain, a prototype of storage cloud for computer scientists and graduate students to manage personal data that spreads over the web. IRain (1) integrates personal data with various metainfo structures that comes from different web sites and personal computers; (2) provides a global, unified environment to users and supports user-defined file views via flexible combination of tags; (3) offers an easy way to integrate new web services via a VFS-liked interface.
---
paper_title: How replicated data management in the cloud can benefit from a data grid protocol: the Re:GRIDiT Approach

paper_content:

Cloud computing has recently received considerable attention both in industry and academia. Due to the great success of the first generation of Cloud-based services, providers have to deal with larger and larger volumes of data. Quality of service agreements with customers require data to be replicated across data centers in order to guarantee a high degree of availability. In this context, Cloud Data Management has to address several challenges, especially when replicated data are concurrently updated at different sites or when the system workload and the resources requested by clients change dynamically. Mostly independent from recent developments in Cloud Data Management, Data Grids have undergone a transition from pure file management with read-only access to more powerful systems. In our recent work, we have developed the Re:GRIDiT protocol for managing data in the Grid which provides concurrent access to replicated data at different sites without any global component and supports the dynamic deployment of replicas. Since it is independent from the underlying Grid middleware, it can be seamlessly transferred to other environments like the Cloud. In this paper, we compare Data Management in the Grid and the Cloud, briefly introduce the Re:GRIDiT protocol and show its applicability for Cloud Data Management.
---
paper_title: Cloud service and service selection algorithm research

paper_content:

This paper describes the cloud service architecture and key technologies for service selection algorithm. Cloud computing is a hot topic on software and distributed computing based on Internet, which means users can access storages and applications from remote servers by web browsers or other fixed or mobile terminals. Because the constrained resources of fixed or mobile terminals, cloud computing will provide terminals with powerful complementation resources to acquire complicated services. The paper discusses the cloud service architecture and key algorithms about service selection with adaptive performances and minimum cost. The cloud service architecture is reasonable and the proposed service selection algorithms are available, scalable, and adaptive to different types of environments of services and clients.
---
paper_title: Five Dimension Environmental Data Resource Brokering on Computational Grids and Scientific Clouds

paper_content:

In this paper we describe how the grid computing resource brokering approach, classically divided into matchmaking discovery and optimize selection, is applied to five dimensional environmental data distribution. This result is achieved thanks to the integration of a resource broker service we developed from scratch. This service implements the key feature of autonomic mapping of Globus toolkit index service resources on the Condor ClassAd resource description. Our five dimensional distribution data service relays on this software infrastructure to advertise metadata and it provides the requested data leveraging on the Web service resource framework. We provide an example based on a grid aware application component, showing the application of resource brokering to environmental problems. We focused our interests mainly about high resolution weather forecast data management and results quality assessment. Our approach would be both modular and distributed computing technology aware in order to provide the e-science community of grid based cloud computing enabled tools fully benefiting of this technology.
---
paper_title: Creating Next Generation Cloud Computing Based Network Services and the Contributions of Social Cloud Operation Support System (OSS) to Society

paper_content:

Emerging virtualization technologies are making ubiquitous access to on-demand computing, network and storage resources to deliver various applications over public Internet. In this paper we present how the telecom operation support systems (OSS) that provide Enterprise to Enterprise (E2E) transactions, switching management, on-demand service management and scalability have evolved to provide next generation cloud management. Fujitsu’s Social Cloud OSS provides multi-vendor, multi-network management, multi-layer Service Level Agreement (SLA) assurance, on-demand service management and impact analysis to businesses. The Social Cloud OSS service management solution for cloud computing will be the next killer application that will facilitate easy access to cloud services with appropriate SLAs and enable the society to use social networking applications that are currently being delivered using clouds.
---
paper_title: Heuristics-Based Query Processing for Large RDF Graphs Using Cloud Computing

paper_content:

Semantic web is an emerging area to augment human reasoning. Various technologies are being developed in this arena which have been standardized by the World Wide Web Consortium (W3C). One such standard is the Resource Description Framework (RDF). Semantic web technologies can be utilized to build efficient and scalable systems for Cloud Computing. With the explosion of semantic web technologies, large RDF graphs are common place. This poses significant challenges for the storage and retrieval of RDF graphs. Current frameworks do not scale for large RDF graphs and as a result do not address these challenges. In this paper, we describe a framework that we built using Hadoop to store and retrieve large numbers of RDF triples by exploiting the cloud computing paradigm. We describe a scheme to store RDF data in Hadoop Distributed File System. More than one Hadoop job (the smallest unit of execution in Hadoop) may be needed to answer a query because a single triple pattern in a query cannot simultaneously take part in more than one join in a single Hadoop job. To determine the jobs, we present an algorithm to generate query plan, whose worst case cost is bounded, based on a greedy approach to answer a SPARQL Protocol and RDF Query Language (SPARQL) query. We use Hadoop's MapReduce framework to answer the queries. Our results show that we can store large RDF graphs in Hadoop clusters built with cheap commodity class hardware. Furthermore, we show that our framework is scalable and efficient and can handle large amounts of RDF data, unlike traditional approaches.
---
paper_title: A comparison of approaches to large-scale data analysis

paper_content:

There is currently considerable enthusiasm around the MapReduce (MR) paradigm for large-scale data analysis [17]. Although the basic control flow of this framework has existed in parallel SQL database management systems (DBMS) for over 20 years, some have called MR a dramatically new computing model [8, 17]. In this paper, we describe and compare both paradigms. Furthermore, we evaluate both kinds of systems in terms of performance and development complexity. To this end, we define a benchmark consisting of a collection of tasks that we have run on an open source version of MR as well as on two parallel DBMSs. For each task, we measure each system's performance for various degrees of parallelism on a cluster of 100 nodes. Our results reveal some interesting trade-offs. Although the process to load data into and tune the execution of parallel DBMSs took much longer than the MR system, the observed performance of these DBMSs was strikingly better. We speculate about the causes of the dramatic performance difference and consider implementation concepts that future systems should take from both kinds of architectures.
---
paper_title: Pig latin: a not-so-foreign language for data processing

paper_content:

There is a growing need for ad-hoc analysis of extremely large data sets, especially at internet companies where innovation critically depends on being able to analyze terabytes of data collected every day. Parallel database products, e.g., Teradata, offer a solution, but are usually prohibitively expensive at this scale. Besides, many of the people who analyze this data are entrenched procedural programmers, who find the declarative, SQL style to be unnatural. The success of the more procedural map-reduce programming model, and its associated scalable implementations on commodity hardware, is evidence of the above. However, the map-reduce paradigm is too low-level and rigid, and leads to a great deal of custom user code that is hard to maintain, and reuse. We describe a new language called Pig Latin that we have designed to fit in a sweet spot between the declarative style of SQL, and the low-level, procedural style of map-reduce. The accompanying system, Pig, is fully implemented, and compiles Pig Latin into physical plans that are executed over Hadoop, an open-source, map-reduce implementation. We give a few examples of how engineers at Yahoo! are using Pig to dramatically reduce the time required for the development and execution of their data analysis tasks, compared to using Hadoop directly. We also report on a novel debugging environment that comes integrated with Pig, that can lead to even higher productivity gains. Pig is an open-source, Apache-incubator project, and available for general use.
---
paper_title: Data Intensive Query Processing for Large RDF Graphs Using Cloud Computing Tools

paper_content:

Cloud computing is the newest paradigm in the IT world and hence the focus of new research. Companies hosting cloud computing services face the challenge of handling data intensive applications. Semantic web technologies can be an ideal candidate to be used together with cloud computing tools to provide a solution. These technologies have been standardized by the World Wide Web Consortium (W3C). One such standard is the Resource Description Framework (RDF). With the explosion of semantic web technologies, large RDF graphs are common place. Current frameworks do not scale for large RDF graphs. In this paper, we describe a framework that we built using Hadoop, a popular open source framework for Cloud Computing, to store and retrieve large numbers of RDF triples. We describe a scheme to store RDF data in Hadoop Distributed File System. We present an algorithm to generate the best possible query plan to answer a SPARQL Protocol and RDF Query Language (SPARQL) query based on a cost model. We use Hadoop's MapReduce framework to answer the queries. Our results show that we can store large RDF graphs in Hadoop clusters built with cheap commodity class hardware. Furthermore, we show that our framework is scalable and efficient and can easily handle billions of RDF triples, unlike traditional approaches.
---
paper_title: Query processing of massive trajectory data based on mapreduce

paper_content:

With the development of positioning technologies and the boosting deployment of inexpensive location-aware sensors, large volumes of trajectory data have emerged. However, efficient and scalable query processing over trajectory data remains a big challenge. We explore a new approach to this target in this paper, presenting a new framework for query processing over trajectory data based on MapReduce. Traditional trajectory data partitioning, indexing, and query processing technologies are extended so that they may fully utilize the highly parallel processing power of large-scale clusters. We also show that the append-only scheme of MapReduce storage model can be a nice base for handling updates of moving objects. Preliminary experiments show that this framework scales well in terms of the size of trajectory data set. It is also discussed the limitation of traditional trajectory data processing techniques and our future research directions.
---
paper_title: A Vision for Next Generation Query Processors and an Associated Research Agenda

paper_content:

Query processing is one of the most important mechanisms for data management, and there exist mature techniques for effective query optimization and efficient query execution. The vast majority of these techniques assume workloads of rather small transactional tasks with strong requirements for ACID properties. However, the emergence of new computing paradigms, such as grid and cloud computing, the increasingly large volumes of data commonly processed, the need to support data driven research, intensive data analysis and new scenarios, such as processing data streams on the fly or querying web services, the fact that the metadata fed to optimizers are often missing at compile time, and the growing interest in novel optimization criteria, such as monetary cost or energy consumption, create a unique set of new requirements for query processing systems. These requirements cannot be met by modern techniques in their entirety, although interesting solutions and efficient tools have already been developed for some of them in isolation. Next generation query processors are expected to combine features addressing all of these issues, and, consequently, lie at the confluence of several research initiatives. This paper aims to present a vision for such processors, to explain their functionality requirements, and to discuss the open issues, along with their challenges.
---
paper_title: MapReduce: a flexible data processing tool

paper_content:

MapReduce advantages over parallel databases include storage-system independence and fine-grain fault tolerance for large jobs.
---
paper_title: Consistency Rationing in the Cloud: Pay only when it matters

paper_content:

Cloud storage solutions promise high scalability and low cost. Existing solutions, however, differ in the degree of consistency they provide. Our experience using such systems indicates that there is a non-trivial trade-off between cost, consistency and availability. High consistency implies high cost per transaction and, in some situations, reduced availability. Low consistency is cheaper but it might result in higher operational cost because of, e.g., overselling of products in a Web shop. ::: ::: In this paper, we present a new transaction paradigm, that not only allows designers to define the consistency guarantees on the data instead at the transaction level, but also allows to automatically switch consistency guarantees at runtime. We present a number of techniques that let the system dynamically adapt the consistency level by monitoring the data and/or gathering temporal statistics of the data. We demonstrate the feasibility and potential of the ideas through extensive experiments on a first prototype implemented on Amazon's S3 and running the TPC-W benchmark. Our experiments indicate that the adaptive strategies presented in the paper result in a significant reduction in response time and costs including the cost penalties of inconsistencies.
---
paper_title: Cloud Computing: A Digital Libraries Perspective

paper_content:

Provisioning and maintenance of infrastructure for Web based digital library search engines such as CiteSeer$^x$ present several challenges. CiteSeer$^x$ provides autonomous citation indexing, full text indexing, and extensive document metadata from document scrawled from the web across computer and information sciences and related fields. Infrastructure virtualization and cloud computing are particularly attractive choices for CiteSeer$^x$, which is challenged by both growth in the size of the indexed document collection, new features and most prominently usage. In this paper, we discuss constraints and choices faced by information retrieval systems like CiteSeer$^x$ by exploring in detail aspects of placing CiteSeer$^x$ into current cloud infrastructure offerings. We also implement an ad-hoc virtualized storage system for experimenting with adoption of cloud infrastructure services. Our results show that a cloud implementation of CiteSeer$^x$ may be a feasible alternative for its continued operation and growth
---
paper_title: SCOPE: easy and efficient parallel processing of massive data sets

paper_content:

Companies providing cloud-scale services have an increasing need to store and analyze massive data sets such as search logs and click streams. For cost and performance reasons, processing is typically done on large clusters of shared-nothing commodity machines. It is imperative to develop a programming model that hides the complexity of the underlying system but provides flexibility by allowing users to extend functionality to meet a variety of requirements. ::: ::: In this paper, we present a new declarative and extensible scripting language, SCOPE (Structured Computations Optimized for Parallel Execution), targeted for this type of massive data analysis. The language is designed for ease of use with no explicit parallelism, while being amenable to efficient parallel execution on large clusters. SCOPE borrows several features from SQL. Data is modeled as sets of rows composed of typed columns. The select statement is retained with inner joins, outer joins, and aggregation allowed. Users can easily define their own functions and implement their own versions of operators: extractors (parsing and constructing rows from a file), processors (row-wise processing), reducers (group-wise processing), and combiners (combining rows from two inputs). SCOPE supports nesting of expressions but also allows a computation to be specified as a series of steps, in a manner often preferred by programmers. We also describe how scripts are compiled into efficient, parallel execution plans and executed on large clusters.
---

Title: A Survey of Data Management System for Cloud Computing: Models and Searching Methods

Section 1: INTRODUCTION
Description 1: Provide an overview of cloud computing development, current challenges in data management, and the motivation behind researching data modeling and searching methods in cloud computing.

Section 2: DATA MODEL AND SYSTEM ARCHITECTURE
Description 2: Discuss various data models and system architectures proposed by both academic and industrial communities, highlighting key technologies and their applications in cloud computing environments.

Section 3: QUERYING AND SEARCHING METHODS
Description 3: Explore different querying and searching methods specific to cloud computing, focusing on MapReduce-oriented methods and SQL-like querying methods, including their advantages and use cases.

Section 4: CONCLUSION AND OPEN ISSUES
Description 4: Summarize the key points discussed in the survey, and outline potential open research issues and areas for future study in cloud data management and searching methods.