An overview of the commercial cloud monitoring tools: research dimensions, design issues, and state-of-the-art
---
paper_title: Cloud Computing: a Perspective Study

paper_content:

The Cloud computing emerges as a new computing paradigm which aims to provide reliable, customized and QoS guaranteed dynamic computing environments for end-users. In this paper, we study the Cloud computing paradigm from various aspects, such as definitions, distinct features, and enabling technologies. This paper brings an introductional review on the Cloud computing and provides the state-of-the-art of Cloud computing technologies.
---
paper_title: Towards building a cloud for scientific applications

paper_content:

The Cloud computing becomes an innovative computing paradigm, which aims to provide reliable, customized and QoS guaranteed computing infrastructures for users. This paper presents our early experience of Cloud computing based on the Cumulus project for compute centers. In this paper, we give the Cloud computing definition and Cloud computing functionalities. This paper also introduces the Cumulus project with its various aspects, such as design pattern, infrastructure, and middleware. This paper delivers the state-of-the-art for Cloud computing with theoretical definition and practical experience.
---
paper_title: Towards building a multi‐datacenter infrastructure for massive remote sensing image processing

paper_content:

SUMMARY ::: Earth observation applications are now facing the challenges of managing and processing massive data sets from multiple sources from large-scale distributed data centers (DCs). To solve this research problem, this paper presents an infrastructure of multiple data centers (MDC) for managing and processing massive remote sensing images. The proposed system is built on both groups of distributed DCs/clusters, which are equipped with DC or cluster resource manager. Access security and information service are introduced to support this architecture of MDC. We collaboratively organized the algorithm, and data belonged to the MDC in the manner of workflow. In practice, we succeeded in working out the concrete problems regarding procedures in processing applications collaboratively and transfer the massive remote sensing dataset fast and with stable cross-MDC. On the basis of the previously mentioned research work, we will investigate the platform integration of MDC. Copyright © 2012 John Wiley & Sons, Ltd.
---
paper_title: Resource management of distributed virtual machines

paper_content:

Using Virtual Machines (VMs) as a computing resource within a Service Oriented Architecture (SOA) creates a variety of new issues and challenges. Traditionally parallel task scheduling algorithms only focus on handling CPU resources but with use of a VM there are many more resources properties to monitor and manage. The objective of this paper is to address these challenges of a multi-dimensional scheduling algorithm for VMs within a SOA. To do this we deploy a testbed of SOA environment with VMs that are capable of being registered, indexed, allocated, accessed and controlled by our new parallel task scheduling algorithms.
---
paper_title: Challenges and Opportunities with Big Data

paper_content:

The promise of data-driven decision-making is now being recognized broadly, and there is growing enthusiasm for the notion of "Big Data," including the recent announcement from the White House about new funding initiatives across different agencies, that target research for Big Data. While the promise of Big Data is real -- for example, it is estimated that Google alone contributed 54 billion dollars to the US economy in 2009 -- there is no clear consensus on what is Big Data. In fact, there have been many controversial statements about Big Data, such as "Size is the only thing that matters." In this panel we will try to explore the controversies and debunk the myths surrounding Big Data.
---
paper_title: Distributed data structure templates for data-intensive remote sensing applications

paper_content:

SUMMARY ::: The remotely sensed images continuously acquired by satellite and airborne sensors are increasing dramatically. Remote sensing applications are overwhelmed with tons of remote sensing data with complex data structures. Efficient programming in parallel systems for data-intensive applications like massive remote sensing data processing will be a challenge. We propose a generic data-structure oriented programming template to support massive remote sensing data processing in high-performance clusters. These templates provide distributed abstractions for large remote sensing image data with complex data structure and allow these distributed data to be accessed as a global one. Through data serialization and one-sided message passing primitives provided by message passing interface, the distributed remote sensing data template whose sliced data blocks are scattered among nodes could offer a simple and effective way to distribute and communicate massive remote sensing data. Efficient parallel input/output directly to and from the distributed data structure will also be offered to address the input/output bottleneck caused by massive image data. Developers can take the advantage of our templates to program efficient parallel remote sensing algorithms without dealing with data slicing and communication through low-level message passing interface APIs. Through experiments on remote sensing applications, we confirmed that our templates were productive and efficient. Copyright © 2012 John Wiley & Sons, Ltd.
---
paper_title: Shared Resource Monitoring and Throughput Optimization in Cloud-Computing Datacenters

paper_content:

Many data centers employ server consolidation to maximize the efficiency of platform resource usage. As a result, multiple virtual machines (VMs) simultaneously run on each data center platform. Contention for shared resources between these virtual machines has an undesirable and non-deterministic impact on their performance behavior in such platforms. This paper proposes the use of shared resource monitoring to (a) understand the resource usage of each virtual machine on each platform, (b) collect resource usage and performance across different platforms to correlate implications of usage to performance, and (c) migrate VMs that are resource-constrained to improve overall data center throughput and improve Quality of Service (QoS). We focus our efforts on monitoring and addressing shared cache contention and propose a new optimization metric that captures the priority of the VM and the overall weighted throughput of the data center. We conduct detailed experiments emulating data center scenarios including on-line transaction processing workloads (based on TPC-C) middle-tier workloads (based on SPECjbb and SPECjAppServer) and financial workloads (based on PARSEC). We show that monitoring shared resource contention (such as shared cache) is highly beneficial to better manage throughput and QoS in a cloud-computing data center environment.
---
paper_title: Cloud Computing Research and Development Trend

paper_content:

With the development of parallel computing, distributed computing, grid computing, a new computing model appeared. The concept of computing comes from grid, public computing and SaaS. It is a new method that shares basic framework. The basic principles of cloud computing is to make the computing be assigned in a great number of distributed computers, rather then local computer or remoter server. The running of the enterprise’s data center is just like Internet. This makes the enterprise use the resource in the application that is needed, and access computer and storage system according to the requirement. This article introduces the background and principle of cloud computing, the character, style and actuality. This article also introduces the application field the merit of cloud computing, such as, it do not need user’s high level equipment, so it reduces the user’s cost. It provides secure and dependable data storage center, so user needn’t do the awful things such storing data and killing virus, this kind of task can be done by professionals. It can realize data share through different equipments. It analyses some questions and hidden troubles, and puts forward some solutions, and discusses the future of cloud computing. Cloud computing is a computing style that provide power referenced with IT as a service. Users can enjoy the service even he knows nothing about the technology of cloud computing and the professional knowledge in this field and the power to control it.
---
paper_title: Towards enabling Cyberinfrastructure as a Service in Clouds

paper_content:

A modern cyberinfrastructure is normally viewed as a highly complicated system due to complex user interfaces, heterogeneous and distributed software implementation flavors, and different hardware platforms. To mitigate the difficulties of developing a cyberinfrastructure, this study proposes a new computing paradigm: Cyberinfrastructure as a Service (CaaS), i.e., providing a service to enable building a cyberinfrastructure on demand. The paradigm of CaaS has been developed a lightweight as a distributed middleware, namely Cyberaide Creative. The Cyberaide Creative service can on-demand create distributed virtual machines, install operating systems, deploy user-required software, and finally deliver a cyberinfrastructure to users. We demonstrate the its usage via a real application, the Emergency Services Directory (ESD) system, and test it with a High Performance Computing (HPC) benchmark. Experimental results justify the design of CaaS and the implementation of the Cyberaide Creative service.
---
paper_title: An Advanced Survey on Cloud Computing and State-of-the-art Research Issues

paper_content:

Cloud Computing is considered as one of the emerging arenas of computer science in recent times. It is providing excellent facilities to business entrepreneurs by flexible infrastructure. Although, cloud computing is facilitating the Information Technology industry, the research and development in this arena is yet to be satisfactory. Our contribution in this paper is an advanced survey focusing on cloud computing concept and most advanced research issues. This paper provides a better understanding of the cloud computing and identifies important research issues in this burgeoning area of computer science.
---
paper_title: Design and implementation of task scheduling strategies for massive remote sensing data processing across multiple data centers

paper_content:

Data intensive applications of remote sensing data processing are more and more widespread resulting from the evolutions in computer and network technologies. Especially, bags-of-tasks BoTs applications with a mass of sharing input files and directed acyclic graph DAG applications with data dependencies in a widely distributed computing environment bring new challenges. In this article, a strategy of partitioning group based on hypergraph PGH is introduced to formulate the model of sharing files. Within the PGH algorithm, BoTs applications would be partitioned into several groups to minimize the time of data transferring. We also adopted another scheduling policy, which is called optimized task tree OTT strategy to handle the DAG workflow of massive remote sensing data processing with data dependencies. A scheduling queue of DAG tasks would be updated according to the priorities changing. With the help of GridSim simulation environment, we designed the Gridlets within scheduler to test the performance of PGH and OTT. Copyright © 2013 John Wiley & Sons, Ltd.
---
paper_title: Experimental study of virtual machine migration in support of reservation of cluster resources

paper_content:

Virtual Machines are becoming increasingly valuable to resource consolidation and management, providing efficient and secure resource containers, along with desired application execution environments. This paper focuses on the VM-based resource reservation problem, that is, the reservations of CPU, memory and network resources for individual VM instances, as well as for VM clusters. In particular, it considers the scenario where one or several physical servers need to be vacated to start a cluster of VMs for dedicated execution of parallel jobs. VMs provide a primitive for transparently vacating workloads through migration; however, the process of migrating several VMs can be time-consuming and needs to be estimated. To achieve this goal, this paper seeks to provide a model that can characterize the VM migration process and predict its performance, based on a comprehensive experimental analysis. The results show that, given a certain VM's migration time, it is feasible to predict the time for a VM with other configurations, as well as the time for migrating a number of VMs. The paper also shows that migration of VMs in parallel results in shorter aggregate migration times, but with higher per-VM migration latencies. Experimental results also quantify the benefits of buffering the state of migrated VMs in main memory without committing to hard disks.
---
paper_title: CloudWatcher: Network security monitoring using OpenFlow in dynamic cloud networks (or: How to provide security monitoring as a service in clouds?)

paper_content:

Cloud computing is becoming a popular paradigm. Many recent new services are based on cloud environments, and a lot of people are using cloud networks. Since many diverse hosts and network configurations coexist in a cloud network, it is essential to protect each of them in the cloud network from threats. To do this, basically, we can employ existing network security devices, but applying them to a cloud network requires more considerations for its complexity, dynamism, and diversity. In this paper, we propose a new framework, CloudWatcher, which provides monitoring services for large and dynamic cloud networks. This framework automatically detours network packets to be inspected by pre-installed network security devices. In addition, all these operations can be implemented by writing a simple policy script, thus, a cloud network administrator is able to protect his cloud network easily. We have implemented the proposed framework, and evaluated it on different test network environments.
---
paper_title: Design Principles for Effective Knowledge Discovery from Big Data

paper_content:

Big data phenomenon refers to the practice of collection and processing of very large data sets and associated systems and algorithms used to analyze these massive datasets. Architectures for big data usually range across multiple machines and clusters, and they commonly consist of multiple special purpose sub-systems. Coupled with the knowledge discovery process, big data movement offers many unique opportunities for organizations to benefit (with respect to new insights, business optimizations, etc.). However, due to the difficulty of analyzing such large datasets, big data presents unique systems engineering and architectural challenges. In this paper, we present three system design principles that can inform organizations on effective analytic and data collection processes, system organization, and data dissemination practices. The principles presented derive from our own research and development experiences with big data problems from various federal agencies, and we illustrate each principle with our own experiences and recommendations.
---
paper_title: A SLA-based method for big-data transfers with multi-criteria optimization constraints for IaaS

paper_content:

When one pays for a Cloud Service he wants the Service to be compliant with standards and to respect his needs (respect the Service Level Agreement). When manipulating bigdata (like picture collections, satellite images or digital libraries), transfers must be optimized. The Cloud Infrastructure itself utilizes big-data transfers while migrating Virtual Machines inside an Infrastructure as a Service system. In this paper we discuss data transfers in the cloud, which affects performance in the case of Virtual Machine migration and of user submitted bigdata transfers, by moving resources where they are needed. We suggest a scheduling policy and offer two greedy scheduling algorithms that minimize individual transfer times. This is mostly important for being able to have a High Performance Computing environment inside the cloud, for scenarios in which individual data transfer times need to be but when the data is transferred is not an issue. We present an empirical validation based on simulation experiments.
---
paper_title: Virtual Machine Provisioning Based on Analytical Performance and QoS in Cloud Computing Environments

paper_content:

Cloud computing is the latest computing paradigm that delivers IT resources as services in which users are free from the burden of worrying about the low-level implementation or system administration details. However, there are significant problems that exist with regard to efficient provisioning and delivery of applications using Cloud-based IT resources. These barriers concern various levels such as workload modeling, virtualization, performance modeling, deployment, and monitoring of applications on virtualized IT resources. If these problems can be solved, then applications can operate more efficiently, with reduced financial and environmental costs, reduced under-utilization of resources, and better performance at times of peak load. In this paper, we present a provisioning technique that automatically adapts to workload changes related to applications for facilitating the adaptive management of system and offering end-users guaranteed Quality of Services (QoS) in large, autonomous, and highly dynamic environments. We model the behavior and performance of applications and Cloud-based IT resources to adaptively serve end-user requests. To improve the efficiency of the system, we use analytical performance (queueing network system model) and workload information to supply intelligent input about system requirements to an application provisioner with limited information about the physical infrastructure. Our simulation-based experimental results using production workload models indicate that the proposed provisioning technique detects changes in workload intensity (arrival pattern, resource demands) that occur over time and allocates multiple virtualized IT resources accordingly to achieve application QoS targets.
---
paper_title: Peer-to-Peer Cloud Provisioning: Service Discovery and Load-Balancing

paper_content:

Clouds have evolved as the next-generation platform that facilitates creation of wide-area on-demand renting of computing or storage services for hosting application services that experience highly variable workloads and requires high availability and performance. Interconnecting Cloud computing system components (servers, virtual machines (VMs), application services) through peer-to-peer routing and information dissemination structure are essential to avoid the problems of provisioning efficiency bottleneck and single point of failure that are predominantly associated with traditional centralized or hierarchical approaches. These limitations can be overcome by connecting Cloud system components using a structured peer-to-peer network model (such as distributed hash tables (DHTs)). DHTs offer deterministic information/query routing and discovery with close to logarithmic bounds as regards network message complexity. By maintaining a small routing state of O (log n) per VM, a DHT structure can guarantee deterministic look-ups in a completely decentralized and distributed manner. This chapter presents: (i) a layered peer-to-peer Cloud provisioning architecture; (ii) a summary of the current state-of-the-art in Cloud provisioning with particular emphasis on service discovery and load-balancing; (iii) a classification of the existing peer-to-peer network management model with focus on extending the DHTs for indexing and managing complex provisioning information; and (iv) the design and implementation of novel, extensible software fabric (Cloud peer) that combines public/private clouds, overlay networking, and structured peer-to-peer indexing techniques for supporting scalable and self-managing service discovery and load-balancing in Cloud computing environments. Finally, an experimental evaluation is presented that demonstrates the feasibility of building next-generation Cloud provisioning systems based on peer-to-peer network management and information dissemination models. The experimental test-bed has been deployed on a public cloud computing platform, Amazon EC2, which demonstrates the effectiveness of the proposed peer-to-peer Cloud provisioning software fabric.
---
paper_title: A Generic QoS Framework for Cloud Workflow Systems

paper_content:

Due to the dynamic nature of cloud computing, how to achieve satisfactory QoS (Quality of Service) in cloud workflow systems becomes a challenge. Meanwhile, since QoS requirements have many dimensions, a unified system design for different QoS management components is required to reduce the system complexity and software development cost. Therefore, this paper proposes a generic QoS framework for cloud workflow systems. Covering the major stages of a workflow lifecycle, the framework consists of four components, viz. QoS requirement specification, QoS-aware service selection, QoS consistency monitoring and QoS violation handling. While there are many QoS dimensions, this paper illustrates a concrete performance framework as a case study and briefly touches others. We also demonstrate the system implementation and evaluate the effectiveness of the performance framework in our cloud workflow system.
---
paper_title: Toward an architecture for the automated provisioning of cloud services

paper_content:

The automated provisioning of services in cloud computing presents many challenges. Users can request virtual machines from cloud infrastructure providers, but these machines have to be configured and managed properly. This article describes an architecture that enables the automated deployment and management of the virtual infrastructure and software of services deployed in the cloud. The architecture takes a template description of a service, which encapsulates requirements, options, as well as behavior for a collection of resources and orchestrates the provisioning of this service into a newly created set of virtual resources. The template is used for integrating the deployment and reconfiguration behavior of a service in which logical components are described along with options to scale them and appropriately change their configuration. Services are described through a set of components, which can easily be mapped and remapped to dynamically created resources, letting services take full advantage of flexible cloud resources.
---
paper_title: Shared Resource Monitoring and Throughput Optimization in Cloud-Computing Datacenters

paper_content:

Many data centers employ server consolidation to maximize the efficiency of platform resource usage. As a result, multiple virtual machines (VMs) simultaneously run on each data center platform. Contention for shared resources between these virtual machines has an undesirable and non-deterministic impact on their performance behavior in such platforms. This paper proposes the use of shared resource monitoring to (a) understand the resource usage of each virtual machine on each platform, (b) collect resource usage and performance across different platforms to correlate implications of usage to performance, and (c) migrate VMs that are resource-constrained to improve overall data center throughput and improve Quality of Service (QoS). We focus our efforts on monitoring and addressing shared cache contention and propose a new optimization metric that captures the priority of the VM and the overall weighted throughput of the data center. We conduct detailed experiments emulating data center scenarios including on-line transaction processing workloads (based on TPC-C) middle-tier workloads (based on SPECjbb and SPECjAppServer) and financial workloads (based on PARSEC). We show that monitoring shared resource contention (such as shared cache) is highly beneficial to better manage throughput and QoS in a cloud-computing data center environment.
---
paper_title: Monitoring Cloud Computing by Layer, Part 1

paper_content:

The general characteristics of cloud computing's three service models-software as a service (SaaS), platform as a service (PaaS), and infra structure as a service (IaaS)-include on-demand self service, broad network access, pooling of resources, rapid elasticity of provisioning resources, and service or resource monitoring. On the basis of the Cloud Security Alliance's work, a cloud is modeled in seven layers: facility, network, hardware, OS, middle ware, application, and the user. These layers can be controlled by either the cloud provider or the cloud customer. The paper present a set of recommended restrictions and audits to facilitate cloud security. Although the recommendations might be overkill for deployments involving no sensitive data, they might be insufficient to allow certain information to be hosted in any public or community cloud. Owing to space constraints, the paper only cover the lower four layers. Part 2 will complete the discussion by covering the middleware, application, and user layers.
---
paper_title: Cloud Monitor: Monitoring Applications in Cloud

paper_content:

This paper presents Cloud Monitor - a scalable framework for monitoring cloud applications. Cloud Monitor monitors cluster nodes for errors. It supports dependent monitors, redundancy, multiple notification levels and auto-healing. Cloud Manager supports a flexible architecture where users can add custom monitors and associated self-heal actions. This paper presents Cloud Monitor - a scalable framework for monitoring cloud applications. Cloud Monitor monitors cluster nodes for errors. It supports dependent monitors, redundancy, multiple notification levels and auto-healing. Cloud Manager supports a flexible architecture where users can add custom monitors and associated self-heal actions.
---
paper_title: Auto-Scaling , Load Balancing and Monitoring in Commercial and Open-Source Clouds

paper_content:

Over the past years, the Cloud phenomenon had an impressive increase in popularity in both the software industry and research worlds. The most interesting feature that Cloud Computing brings, from a Cloud client's point of view, is the on-demand resource provisioning model. This allows Cloud client platforms to be scaled up in order to accommodate more incoming clients and to scale down when the platform has unused resources, and this can all be done while the platform is running. As a result, the physical resources are used more efficiently and the Cloud client saves expenses. Achieving the above mentioned is not trivial and is done by leveraging more direct functionalities that Clouds provide. Three of these key functionalities are automatic scaling, load balancing and monitoring. They represent the focus of the current work. This report is divided into three main parts, one for each of the three main topics of interest: auto-scaling, load balancing and monitoring. We detail each of the three topics and present details on their presence and implementation in the commercial, open-source and research worlds. Among the commercial Cloud platform providers we have focused on Amazon EC2, Microsoft Azure, GoGrid and RackSpace. From the open-source initiatives we have focused our attention on Nimbus, Eucalyptus and OpenNebula. We have also detailed endeavours in the research world that are relevant to the topic of interest and are not connected to any of the Cloud providers. By studying this report, the reader will be able to understand the role that each of the three Cloud features plays for a Cloud client, will understand their inner workings and will be aware of the state-of-the-art available in current commercial and open-source Cloud platforms.
---
paper_title: A Runtime Model Based Monitoring Approach for Cloud

paper_content:

Monitoring plays a significant role in improving the quality of service in cloud computing. It helps clouds to scale resource utilization adaptively, to identify defects in services for service developers, and to discover usage patterns of numerous end users. However, due to the heterogeneity of components in clouds and the complexity arising from the wealth of runtime information, monitoring in clouds faces many new challenges. In this paper, we propose a runtime model for cloud monitoring (RMCM), which denotes an intuitive representation of a running cloud by focusing on common monitoring concerns. Raw monitoring data gathered by multiple monitoring techniques are organized by RMCM to present a more intuitive profile of a running cloud. We applied RMCM in the implementation of a flexible monitoring framework, which can achieve a balance between runtime overhead and monitoring capability via adaptive management of monitoring facilities. Our experience of utilizing the monitoring framework on a real cloud demonstrates the feasibility and effectiveness of our approach.
---
paper_title: Programming Cloud Resource Orchestration Framework: Operations and Research Challenges

paper_content:

The emergence of cloud computing over the past five years is potentially one of the breakthrough advances in the history of computing. It delivers hardware and software resources as virtualization-enabled services and in which administrators are free from the burden of worrying about the low level implementation or system administration details. Although cloud computing offers considerable opportunities for the users (e.g. application developers, governments, new startups, administrators, consultants, scientists, business analyst, etc.) such as no up-front investment, lowering operating cost, and infinite scalability, it has many unique research challenges that need to be carefully addressed in the future. In this paper, we present a survey on key cloud computing concepts, resource abstractions, and programming operations for orchestrating resources and associated research challenges, wherever applicable.
---
paper_title: Monalytics: online monitoring and analytics for managing large scale data centers

paper_content:

To effectively manage large-scale data centers and utility clouds, operators must understand current system and application behaviors. This requires continuous monitoring along with online analysis of the data captured by the monitoring system. As a result, there is a need to move to systems in which both tasks can be performed in an integrated fashion, thereby better able to drive online system management. Coining the term 'monalytics' to refer to the combined monitoring and analysis systems used for managing large-scale data center systems, this paper articulates principles for monalytics systems, describes software approaches for implementing them, and provides experimental evaluations justifying principles and implementation approach. Specific technical contributions include consideration of scalability across both 'space' and 'time', the ability to dynamically deploy and adjust monalytics functionality at multiple levels of abstraction in target systems, and the capability to operate across the range of application to hypervisor layers present in large-scale data center or cloud computing systems. Our monalytics implementation targets virtualized systems and cloud infrastructures, via the integration of its functionality into the Xen hypervisor.
---
paper_title: Broadband internet performance: a view from the gateway

paper_content:

We present the first study of network access link performance measured directly from home gateway devices. Policymakers, ISPs, and users are increasingly interested in studying the performance of Internet access links. Because of many confounding factors in a home network or on end hosts, however, thoroughly understanding access network performance requires deploying measurement infrastructure in users' homes as gateway devices. In conjunction with the Federal Communication Commission's study of broadband Internet access in the United States, we study the throughput and latency of network access links using longitudinal measurements from nearly 4,000 gateway devices across 8 ISPs from a deployment of over 4,200 devices. We study the performance users achieve and how various factors ranging from the user's choice of modem to the ISP's traffic shaping policies can affect performance. Our study yields many important findings about the characteristics of existing access networks. Our findings also provide insights into the ways that access network performance should be measured and presented to users, which can help inform ongoing broader efforts to benchmark the performance of access networks.
---
paper_title: Monitoring Cloud Computing by Layer, Part 1

paper_content:

The general characteristics of cloud computing's three service models-software as a service (SaaS), platform as a service (PaaS), and infra structure as a service (IaaS)-include on-demand self service, broad network access, pooling of resources, rapid elasticity of provisioning resources, and service or resource monitoring. On the basis of the Cloud Security Alliance's work, a cloud is modeled in seven layers: facility, network, hardware, OS, middle ware, application, and the user. These layers can be controlled by either the cloud provider or the cloud customer. The paper present a set of recommended restrictions and audits to facilitate cloud security. Although the recommendations might be overkill for deployments involving no sensitive data, they might be insufficient to allow certain information to be hosted in any public or community cloud. Owing to space constraints, the paper only cover the lower four layers. Part 2 will complete the discussion by covering the middleware, application, and user layers.
---
paper_title: Auto-Scaling , Load Balancing and Monitoring in Commercial and Open-Source Clouds

paper_content:

Over the past years, the Cloud phenomenon had an impressive increase in popularity in both the software industry and research worlds. The most interesting feature that Cloud Computing brings, from a Cloud client's point of view, is the on-demand resource provisioning model. This allows Cloud client platforms to be scaled up in order to accommodate more incoming clients and to scale down when the platform has unused resources, and this can all be done while the platform is running. As a result, the physical resources are used more efficiently and the Cloud client saves expenses. Achieving the above mentioned is not trivial and is done by leveraging more direct functionalities that Clouds provide. Three of these key functionalities are automatic scaling, load balancing and monitoring. They represent the focus of the current work. This report is divided into three main parts, one for each of the three main topics of interest: auto-scaling, load balancing and monitoring. We detail each of the three topics and present details on their presence and implementation in the commercial, open-source and research worlds. Among the commercial Cloud platform providers we have focused on Amazon EC2, Microsoft Azure, GoGrid and RackSpace. From the open-source initiatives we have focused our attention on Nimbus, Eucalyptus and OpenNebula. We have also detailed endeavours in the research world that are relevant to the topic of interest and are not connected to any of the Cloud providers. By studying this report, the reader will be able to understand the role that each of the three Cloud features plays for a Cloud client, will understand their inner workings and will be aware of the state-of-the-art available in current commercial and open-source Cloud platforms.
---
paper_title: A Monitoring and Audit Logging Architecture for Data Location Compliance in Federated Cloud Infrastructures

paper_content:

Current cloud infrastructures have opaque service offerings where customers cannot monitor the underlying physical infrastructure. This situation raises concerns for meeting compliance obligations by critical business applications with data location constraints that are deployed in a Cloud. When federated cloud infrastructures span across different countries where data can migrate from one country to another, it should be possible for data owners to monitor the location of their data. This paper shows how an existing federated Cloud monitoring infrastructure can be used for data location monitoring without compromising Cloud isolation. In the proposed approach collaboration is required between the cloud infrastructure provider (IP) and the user of the cloud, the service provider (SP): the IP monitors the virtual machines (VM) on the SP's behalf and makes the infrastructure level monitoring information available to him. With the monitoring information the SP can create the audit logs required for compliance auditing. The proposed logging architecture is validated by an e-Government case study with legal data location constraints.
---
paper_title: Cloud Monitor: Monitoring Applications in Cloud

paper_content:

This paper presents Cloud Monitor - a scalable framework for monitoring cloud applications. Cloud Monitor monitors cluster nodes for errors. It supports dependent monitors, redundancy, multiple notification levels and auto-healing. Cloud Manager supports a flexible architecture where users can add custom monitors and associated self-heal actions. This paper presents Cloud Monitor - a scalable framework for monitoring cloud applications. Cloud Monitor monitors cluster nodes for errors. It supports dependent monitors, redundancy, multiple notification levels and auto-healing. Cloud Manager supports a flexible architecture where users can add custom monitors and associated self-heal actions.
---
paper_title: A Generic QoS Framework for Cloud Workflow Systems

paper_content:

Due to the dynamic nature of cloud computing, how to achieve satisfactory QoS (Quality of Service) in cloud workflow systems becomes a challenge. Meanwhile, since QoS requirements have many dimensions, a unified system design for different QoS management components is required to reduce the system complexity and software development cost. Therefore, this paper proposes a generic QoS framework for cloud workflow systems. Covering the major stages of a workflow lifecycle, the framework consists of four components, viz. QoS requirement specification, QoS-aware service selection, QoS consistency monitoring and QoS violation handling. While there are many QoS dimensions, this paper illustrates a concrete performance framework as a case study and briefly touches others. We also demonstrate the system implementation and evaluate the effectiveness of the performance framework in our cloud workflow system.
---
paper_title: Q-clouds: managing performance interference effects for QoS-aware clouds

paper_content:

Cloud computing offers users the ability to access large pools of computational and storage resources on demand. Multiple commercial clouds already allow businesses to replace, or supplement, privately owned IT assets, alleviating them from the burden of managing and maintaining these facilities. However, there are issues that must be addressed before this vision of utility computing can be fully realized. In existing systems, customers are charged based upon the amount of resources used or reserved, but no guarantees are made regarding the application level performance or quality-of-service (QoS) that the given resources will provide. As cloud providers continue to utilize virtualization technologies in their systems, this can become problematic. In particular, the consolidation of multiple customer applications onto multicore servers introduces performance interference between collocated workloads, significantly impacting application QoS. To address this challenge, we advocate that the cloud should transparently provision additional resources as necessary to achieve the performance that customers would have realized if they were running in isolation. Accordingly, we have developed Q-Clouds, a QoS-aware control framework that tunes resource allocations to mitigate performance interference effects. Q-Clouds uses online feedback to build a multi-input multi-output (MIMO) model that captures performance interference interactions, and uses it to perform closed loop resource management. In addition, we utilize this functionality to allow applications to specify multiple levels of QoS as application Q-states. For such applications, Q-Clouds dynamically provisions underutilized resources to enable elevated QoS levels, thereby improving system efficiency. Experimental evaluations of our solution using benchmark applications illustrate the benefits: performance interference is mitigated completely when feasible, and system utilization is improved by up to 35% using Q-states.
---
paper_title: Auto-Scaling , Load Balancing and Monitoring in Commercial and Open-Source Clouds

paper_content:

Over the past years, the Cloud phenomenon had an impressive increase in popularity in both the software industry and research worlds. The most interesting feature that Cloud Computing brings, from a Cloud client's point of view, is the on-demand resource provisioning model. This allows Cloud client platforms to be scaled up in order to accommodate more incoming clients and to scale down when the platform has unused resources, and this can all be done while the platform is running. As a result, the physical resources are used more efficiently and the Cloud client saves expenses. Achieving the above mentioned is not trivial and is done by leveraging more direct functionalities that Clouds provide. Three of these key functionalities are automatic scaling, load balancing and monitoring. They represent the focus of the current work. This report is divided into three main parts, one for each of the three main topics of interest: auto-scaling, load balancing and monitoring. We detail each of the three topics and present details on their presence and implementation in the commercial, open-source and research worlds. Among the commercial Cloud platform providers we have focused on Amazon EC2, Microsoft Azure, GoGrid and RackSpace. From the open-source initiatives we have focused our attention on Nimbus, Eucalyptus and OpenNebula. We have also detailed endeavours in the research world that are relevant to the topic of interest and are not connected to any of the Cloud providers. By studying this report, the reader will be able to understand the role that each of the three Cloud features plays for a Cloud client, will understand their inner workings and will be aware of the state-of-the-art available in current commercial and open-source Cloud platforms.
---

Title: An Overview of the Commercial Cloud Monitoring Tools: Research Dimensions, Design Issues, and State-of-the-Art
Section 1: Introduction
Description 1: Describe the motivation and necessity for studying cloud monitoring tools, including the concepts of cloud computing, the challenges faced in monitoring, and the scope of the paper.

Section 2: Our contributions
Description 2: Summarize the main contributions of the paper and provide an overview of the structure of subsequent sections.

Section 3: Cloud resource provisioning
Description 3: Discuss the process of cloud resource provisioning, including virtual machine provisioning, resource provisioning, and application provisioning.

Section 4: Cloud monitoring
Description 4: Present the basic components, phases, and layers of application architecture on clouds, and discuss the state of the art in cloud monitoring, including its correlation with QoS and SLA.

Section 5: Application life cycle
Description 5: Outline the various phases of the application life cycle in a cloud environment, focusing on resource selection, deployment, monitoring, and control.

Section 6: Monitoring, QoS, and SLA
Description 6: Explain the relationship between cloud monitoring, Quality of Service (QoS), and Service Level Agreements (SLA), including different monitoring methodologies and data metrics.

Section 7: Monitoring across different applications and layers
Description 7: Examine the need for monitoring QoS parameters across various cloud layers such as PaaS and IaaS, and present examples of layer-specific and layer-agnostic monitoring tools.

Section 8: Evaluation dimensions
Description 8: Describe the evaluation dimensions used to assess cloud monitoring tools, including architectural considerations, interoperability, QoS matrix, and cross-layer monitoring.

Section 9: Monitoring architectures
Description 9: Discuss the different architectural approaches to cloud monitoring, comparing centralized versus decentralized architectures.

Section 10: Programming interfaces
Description 10: Detail the programming interfaces, including APIs, command-lines, and widgets used in cloud monitoring tools.

Section 11: Communication protocols
Description 11: Outline the communication protocols adopted by commercial cloud monitoring tools for data transfer.

Section 12: Classification and analysis of cloud monitoring tools based on taxonomy
Description 12: Provide a classification and analysis of existing cloud monitoring tools based on the outlined evaluation dimensions and summarize their capabilities.

Section 13: Conclusion and future research directions
Description 13: Summarize the key findings of the paper and propose future research directions in the development of cloud monitoring tools.