Observing the clouds: a survey and taxonomy of cloud monitoring
---
paper_title: DARGOS: A highly adaptable and scalable monitoring architecture for multi-tenant Clouds

paper_content:

One of the most important features in Cloud environments is to know the status and the availability of the physical resources and services present in the current infrastructure. A full knowledge and control of the current status of those resources enables Cloud administrators to design better Cloud provisioning strategies and to avoid SLA violations. However, it is not easy to manage such information in a reliable and scalable way, especially when we consider Cloud environments used and shared by several tenants and when we need to harmonize their different monitoring needs at different Cloud software stack layers. To cope with these issues, we propose Distributed Architecture for Resource manaGement and mOnitoring in cloudS (DARGOS), a completely distributed and highly efficient Cloud monitoring architecture to disseminate resource monitoring information. DARGOS ensures an accurate measurement of physical and virtual resources in the Cloud keeping at the same time a low overhead. In addition, DARGOS is flexible and adaptable and allows defining and monitoring new metrics easily. The proposed monitoring architecture and related tools have been integrated into a real Cloud deployment based on the OpenStack platform: they are openly available for the research community and include a Web-based customizable Cloud monitoring console. We report experimental results to assess our architecture and quantitatively compare it with a selection of other Cloud monitoring systems similar to ours showing that DARGOS introduces a very limited and scalable monitoring overhead.
---
paper_title: Elastic monitoring framework for cloud infrastructures

paper_content:

This study presents a scalable and elastic distributed system for monitoring cloud infrastructure based on a pure peer-to-peer architecture. Its distributed nature enables deploying long-living queries across the network to monitor a diverse set of entities and metrics, spanning across all layers of a cloud stack that can change rapidly. This allows for aggregating low-level metrics from operating systems, to higher-level application-specific metrics derived from services, databases or application log files. The observed metrics and information can be evaluated and used to reliably trigger policies to automate complex management tasks within a cloud environment. The architecture incorporates a query framework for obtaining high-level information and a policy framework to provide self-management capabilities to monitored cloud infrastructure. The system has been implemented as a proof of concept. Details and statistical results are provided to validate the scalability of the underlying architecture.
---
paper_title: GEMS: Gossip-Enabled Monitoring Service for Scalable Heterogeneous Distributed Systems

paper_content:

Gossip protocols have proven to be effective means by which failures can be detected in large, distributed systems in an asynchronous manner without the limitations associated with reliable multicasting for group communications. In this paper, we discuss the development and features of a Gossip-Enabled Monitoring Service (GEMS), a highly responsive and scalable resource monitoring service, to monitor health and performance information in heterogeneous distributed systems. GEMS has many novel and essential features such as detection of network partitions and dynamic insertion of new nodes into the service. Easily extensible, GEMS also incorporates facilities for distributing arbitrary system and application-specific data. We present experiments and analytical projections demonstrating scalability, fast response times and low resource utilization requirements, making GEMS a potent solution for resource monitoring in distributed computing.
---
paper_title: Exploring the Performance Fluctuations of HPC Workloads on Clouds

paper_content:

Clouds enable novel execution modes often supported by advanced capabilities such as autonomic schedulers. These capabilities are predicated upon an accurate estimation and calculation of runtimes on a given infrastructure. Using a well understood high-performance computing workload, we find strong fluctuations from the mean performance on EC2 and Eucalyptus-based cloud systems. Our analysis eliminates variations in IO and computational times as possible causes, we find that variations in communication times account for the bulk of the experiment-to-experiment fluctuations of the performance.
---
paper_title: A Performance Analysis of EC2 Cloud Computing Services for Scientific Computing

paper_content:

Cloud Computing is emerging today as a commercial infrastructure that eliminates the need for maintaining expensive computing hardware. Through the use of virtualization, clouds promise to address with the same shared set of physical resources a large user base with different needs. Thus, clouds promise to be for scientists an alternative to clusters, grids, and supercomputers. However, virtualization may induce significant performance penalties for the demanding scientific computing workloads. In this work we present an evaluation of the usefulness of the current cloud computing services for scientific computing. We analyze the performance of the Amazon EC2 platform using micro-benchmarks and kernels. While clouds are still changing, our results indicate that the current cloud services need an order of magnitude in performance improvement to be useful to the scientific community.
---
paper_title: Semantic based data collection for large scale cloud systems

paper_content:

Current tools for monitoring cloud systems are designed for physical servers and are not intended to handle rapid elasticity or dynamic behaviour while operating at scale. Though current monitoring tools can be applied to small cloud systems, the volume of data and computational overhead associated with their operation render them unsuitable for large scale cloud deployments. The metrics obtained by current solutions also lack a machine readable structure, limiting the ability of both software and humans to interpret the data. As cloud adoption continues, the scale and complexity of cloud systems will present significant challenges to current tools. This paper proposes a scalable distributed data collection system which forms the basis of a cloud monitoring system. Utilising technologies from the semantic web, our architecture generates a machine readable overview of a cloud system without the need for an additional dedicated monitoring system. We present an exemplar implementation of our architecture written using the Python programming language and perform an evaluation demonstrating its ability to provide scalable data collection services fit for cloud computing
---
paper_title: Cloud Computing and Grid Computing 360-Degree Compared

paper_content:

Cloud Computing has become another buzzword after Web 2.0. However, there are dozens of different definitions for Cloud Computing and there seems to be no consensus on what a Cloud is. On the other hand, Cloud Computing is not a completely new concept; it has intricate connection to the relatively new but thirteen-year established Grid Computing paradigm, and other relevant technologies such as utility computing, cluster computing, and distributed systems in general. This paper strives to compare and contrast Cloud Computing with Grid Computing from various angles and give insights into the essential characteristics of both.
---
paper_title: The cost of doing science on the cloud: The Montage example

paper_content:

Utility grids such as the Amazon EC2 cloud and Amazon S3 offer computational and storage resources that can be used on-demand for a fee by compute and data-intensive applications. The cost of running an application on such a cloud depends on the compute, storage and communication resources it will provision and consume. Different execution plans of the same application may result in significantly different costs. Using the Amazon cloud fee structure and a real-life astronomy application, we study via simulation the cost performance tradeoffs of different execution and resource provisioning plans. We also study these trade-offs in the context of the storage and communication fees of Amazon S3 when used for long-term application data archival. Our results show that by provisioning the right amount of storage and compute resources, cost can be significantly reduced with no significant impact on application performance.
---
paper_title: Next Generation Cloud Computing Architecture: Enabling Real-Time Dynamism for Shared Distributed Physical Infrastructure

paper_content:

Cloud computing is fundamentally altering the expectations for how and when computing, storage and networking resources should be allocated, managed and consumed. End-users are increasingly sensitive to the latency of services they consume. Service Developers want the Service Providers to ensure or provide the capability to dynamically allocate and manage resources in response to changing demand patterns in real-time. Ultimately, Service Providers are under pressure to architect their infrastructure to enable real-time end-to-end visibility and dynamic resource management with fine grained control to reduce total cost of ownership while also improving agility. The current approaches to enabling real-time, dynamic infrastructure are inadequate, expensive and not scalable to support consumer mass-market requirements. Over time, the server-centric infrastructure management systems have evolved to become a complex tangle of layered systems designed to automate systems administration functions that are knowledge and labor intensive. This expensive and non-real time paradigm is ill suited for a world where customers are demanding communication, collaboration and commerce at the speed of light. Thanks to hardware assisted virtualization, and the resulting decoupling of infrastructure and application management, it is now possible to provide dynamic visibility and control of services management to meet the rapidly growing demand for cloud-based services. What is needed is a rethinking of the underlying operating system and management infrastructure to accommodate the ongoing transformation of the data center from the traditional server-centric architecture model to a cloud or network-centric model. This paper proposes and describes a reference model for a network-centric datacenter infrastructure management stack that borrows and applies key concepts that have enabled dynamism, scalability, reliability and security in the telecom industry, to the computing industry. Finally, the paper will describe a proof-of-concept system that was implemented to demonstrate how dynamic resource management can be implemented to enable real-time service assurance for network centric datacenter architecture.
---
paper_title: Improving Utilization of Infrastructure Clouds

paper_content:

A key advantage of infrastructure-as-a-service (IaaS) clouds is providing users on-demand access to resources. To provide on-demand access, however, cloud providers must either significantly overprovision their infrastructure (and pay a high price for operating resources with low utilization) or reject a large proportion of user requests (in which case the access is no longer on-demand). At the same time, not all users require truly on-demand access to resources. Many applications and workflows are designed for recoverable systems where interruptions in service are expected. For instance, many scientists utilize high-throughput computing (HTC)-enabled resources, such as Condor, where jobs are dispatched to available resources and terminated when the resource is no longer available. We propose a cloud infrastructure that combines on-demand allocation of resources with opportunistic provisioning of cycles from idle cloud nodes to other processes by deploying backfill virtual machines (VMs). For demonstration and experimental evaluation, we extend the Nimbus cloud computing toolkit to deploy backfill VMs on idle cloud nodes for processing an HTC workload. Initial tests show an increase in IaaS cloud utilization from 37.5% to 100% during a portion of the evaluation trace but only 6.39% overhead cost for processing the HTC workload. We demonstrate that a shared infrastructure between IaaS cloud providers and an HTC job management system can be highly beneficial to both the IaaS cloud provider and HTC users by increasing the utilization of the cloud infrastructure (thereby decreasing the overall cost) and contributing cycles that would otherwise be idle to processing HTC jobs.
---
paper_title: The ganglia distributed monitoring system: design, implementation, and experience

paper_content:

Abstract Ganglia is a scalable distributed monitoring system for high performance computing systems such as clusters and Grids. It is based on a hierarchical design targeted at federations of clusters. It relies on a multicast-based listen/announce protocol to monitor state within clusters and uses a tree of point-to-point connections amongst representative cluster nodes to federate clusters and aggregate their state. It leverages widely used technologies such as XML for data representation, XDR for compact, portable data transport, and RRDtool for data storage and visualization. It uses carefully engineered data structures and algorithms to achieve very low per-node overheads and high concurrency. The implementation is robust, has been ported to an extensive set of operating systems and processor architectures, and is currently in use on over 500 clusters around the world. This paper presents the design, implementation, and evaluation of Ganglia along with experience gained through real world deployments on systems of widely varying scale, configurations, and target application domains over the last two and a half years.
---
paper_title: visPerf: Monitoring Tool for Grid Computing

paper_content:

It is difficult to see the status of a working production grid system without a customized monitoring system. Most grid middleware provide simple system monitoring tools, or simple tools for checking system status. visPerf is a general purpose grid monitoring tool for visualizing, investigating, and controlling the system in a distributed manner. visPerf is a system based on a distributed monitoring sensor, visSensor, in which the sensor uses methods to monitor the status of grid middleware with little or no modifications to the underlying system.
---
paper_title: Probabilistic Reliable Dissemination in Large-Scale Systems

paper_content:

The growth of the Internet raises new challenges for the design of distributed systems and applications. In the context of group communication protocols, gossip-based schemes have attracted interest as they are scalable, easy to deploy, and resilient to network and process failures. However, traditional gossip-based protocols have two major drawbacks: 1) they rely on each peer having knowledge of the global membership; and 2) being oblivious to the network topology, they can impose a high load on network links when applied to wide-area settings. In this paper, we provide a theoretical analysis of gossip-based protocols which relates their reliability to key system parameters (the system size, failure rates, and number of gossip targets). The results provide guidelines for the design of practical protocols. In particular, they show how reliability can be maintained while alleviating drawback by: 1) providing each peer with only a small subset of the total membership information and drawback; and 2) organizing members into a hierarchical structure that reflects their proximity according to some network-related metric. We validate the analytical results by simulations and verify that the hierarchical gossip protocol considerably reduces the load on the network compared to the original, non-hierarchical protocol.
---
paper_title: GEMS: Gossip-Enabled Monitoring Service for Scalable Heterogeneous Distributed Systems

paper_content:

Gossip protocols have proven to be effective means by which failures can be detected in large, distributed systems in an asynchronous manner without the limitations associated with reliable multicasting for group communications. In this paper, we discuss the development and features of a Gossip-Enabled Monitoring Service (GEMS), a highly responsive and scalable resource monitoring service, to monitor health and performance information in heterogeneous distributed systems. GEMS has many novel and essential features such as detection of network partitions and dynamic insertion of new nodes into the service. Easily extensible, GEMS also incorporates facilities for distributing arbitrary system and application-specific data. We present experiments and analytical projections demonstrating scalability, fast response times and low resource utilization requirements, making GEMS a potent solution for resource monitoring in distributed computing.
---
paper_title: A publish/subscribe middleware for dependable and real-time resource monitoring in the cloud

paper_content:

Providing scalable and QoS-enabled (i.e., real-time and reliable) monitoring of resources (both virtual and physical) in the cloud is essential to supporting application QoS properties in the cloud as well as identifying security threats. Existing approaches to resource monitoring in the cloud are based on web interfaces, such as RESTful APIs and SOAP, which cannot provide real-time information efficiently and scalably because of a lack of support for fine-grained and differentiated monitoring capabilities. Moreover, their implementation overhead results in a distinct loss in performance, incurs latency jitter, and degrades reliable delivery of time-sensitive information. To address these challenges this paper presents a novel lighter weight and scalable resource monitoring and dissemination solution based on the publish/subscribe (pub/sub) paradigm. Our solution called SQRT-C leverages the OMG Data Distribution Service (DDS) real-time pub/sub middleware, and uses effective software engineering principles to make it usable with multiple cloud platforms. Preliminary empirical results comparing SQRT-C with contemporary web-based resource usage monitoring services reveals that SQRT-C is significantly better than the conventional approaches in terms of latency, jitter and scalability.
---
paper_title: Elastic monitoring framework for cloud infrastructures

paper_content:

This study presents a scalable and elastic distributed system for monitoring cloud infrastructure based on a pure peer-to-peer architecture. Its distributed nature enables deploying long-living queries across the network to monitor a diverse set of entities and metrics, spanning across all layers of a cloud stack that can change rapidly. This allows for aggregating low-level metrics from operating systems, to higher-level application-specific metrics derived from services, databases or application log files. The observed metrics and information can be evaluated and used to reliably trigger policies to automate complex management tasks within a cloud environment. The architecture incorporates a query framework for obtaining high-level information and a policy framework to provide self-management capabilities to monitored cloud infrastructure. The system has been implemented as a proof of concept. Details and statistical results are provided to validate the scalability of the underlying architecture.
---
paper_title: The SmartFrog configuration management framework

paper_content:

SmartFrog is a framework for creating configuration-driven systems. It has been designed with the express purpose of making the design, deployment and management of distributed component-based systems simpler and more robust. Over the last decade it has been the focus for ongoing research into aspects of configuration management and large-scale distributed systems, providing a platform for experimentation. The paper covers the rationale for the design of the framework, details of its design, plus a description of the further research that is in progress.
---
paper_title: Resource Usage Monitoring in Clouds

paper_content:

Monitoring of infrastructural resources in clouds plays a crucial role in providing application guarantees like performance, availability, and security. Monitoring is crucial from two perspectives - the cloud-user and the service provider. The cloud user's interest is in doing an analysis to arrive at appropriate Service-level agreement (SLA) demands and thecloud provider's interest is to assess if the demand can be met. To support this, a monitoring framework is necessary particularly since cloud hosts are subject to varying load conditions. To illustrate the importance of such a framework, we choose the example of performance being the Quality of Service (QoS) requirement and show how inappropriate provisioning of resources may lead to unexpected performance bottlenecks. We evaluate existing monitoring frameworks to bring out the motivation for buildingmuch more powerful monitoring frameworks. We then propose a distributed monitoring framework, which enables i¬ne grained monitoring for applications and demonstrate with a prototype system implementation for typical use cases.
---
paper_title: DARGOS: A highly adaptable and scalable monitoring architecture for multi-tenant Clouds

paper_content:

One of the most important features in Cloud environments is to know the status and the availability of the physical resources and services present in the current infrastructure. A full knowledge and control of the current status of those resources enables Cloud administrators to design better Cloud provisioning strategies and to avoid SLA violations. However, it is not easy to manage such information in a reliable and scalable way, especially when we consider Cloud environments used and shared by several tenants and when we need to harmonize their different monitoring needs at different Cloud software stack layers. To cope with these issues, we propose Distributed Architecture for Resource manaGement and mOnitoring in cloudS (DARGOS), a completely distributed and highly efficient Cloud monitoring architecture to disseminate resource monitoring information. DARGOS ensures an accurate measurement of physical and virtual resources in the Cloud keeping at the same time a low overhead. In addition, DARGOS is flexible and adaptable and allows defining and monitoring new metrics easily. The proposed monitoring architecture and related tools have been integrated into a real Cloud deployment based on the OpenStack platform: they are openly available for the research community and include a Web-based customizable Cloud monitoring console. We report experimental results to assess our architecture and quantitatively compare it with a selection of other Cloud monitoring systems similar to ours showing that DARGOS introduces a very limited and scalable monitoring overhead.
---
paper_title: OMG data distribution service: architectural overview

paper_content:

The OMG data-distribution service (DDS) is an emerging specification for publish-subscribe data-distribution systems. The purpose of the specification is to provide a common application-level interface that clearly defines the data-distribution service. The specification describes the service using UML, thus providing a platform-independent model that can then be mapped into a variety of concrete platforms and programming languages. The OMG DDS attempts to unify the common practice of several existing implementations [(M. Boasson et al., July 1993), (Pardo-Castellote et al., 1999] enumerating and providing formal definitions for the QoS (quality of service) settings used to configure the service. This paper introduces the OMG DDS specification, and describes the main aspects of the model.
---
paper_title: An Introduction To Compressive Sampling

paper_content:

Conventional approaches to sampling signals or images follow Shannon's theorem: the sampling rate must be at least twice the maximum frequency present in the signal (Nyquist rate). In the field of data conversion, standard analog-to-digital converter (ADC) technology implements the usual quantized Shannon representation - the signal is uniformly sampled at or above the Nyquist rate. This article surveys the theory of compressive sampling, also known as compressed sensing or CS, a novel sensing/sampling paradigm that goes against the common wisdom in data acquisition. CS theory asserts that one can recover certain signals and images from far fewer samples or measurements than traditional methods use.
---
paper_title: Cloudsense: Continuous fine-grain cloud monitoring with compressive sensing

paper_content:

Continuous fine-grain status monitoring of a cloud data center enables rapid response to anomalies, but handling the resulting torrent of data poses a significant challenge. As a solution, we propose CloudSense, a new switch design that performs in-network compression of status streams via compressive sensing. Using MapReduce straggler detection as an example of cloud monitoring, we give evidence that CloudSense allows earlier detection of stragglers, since finer-grain status can be reported for a given bandwidth budget. Furthermore, CloudSense showcases the advantage of an intrinsic property of compressive sensing decoding that enables detection of the slowest stragglers first. Finally, CloudSense achieves in-network compression via a low-complexity encoding scheme, which is easy and convenient to implement in a switch. We envision that CloudSense switches could form the foundation of a "compressed status information plane" that is useful for monitoring not only the cloud data center itself, but also the user applications that it hosts.
---
paper_title: GMonE: A complete approach to cloud monitoring

paper_content:

The inherent complexity of modern cloud infrastructures has created the need for innovative monitoring approaches, as state-of-the-art solutions used for other large-scale environments do not address specific cloud features. Although cloud monitoring is nowadays an active research field, a comprehensive study covering all its aspects has not been presented yet. This paper provides a deep insight into cloud monitoring. It proposes a unified cloud monitoring taxonomy, based on which it defines a layered cloud monitoring architecture. To illustrate it, we have implemented GMonE, a general-purpose cloud monitoring tool which covers all aspects of cloud monitoring by specifically addressing the needs of modern cloud infrastructures. Furthermore, we have evaluated the performance, scalability and overhead of GMonE with Yahoo Cloud Serving Benchmark (YCSB), by using the OpenNebula cloud middleware on the Grid'5000 experimental testbed. The results of this evaluation demonstrate the benefits of our approach, surpassing the monitoring performance and capabilities of cloud monitoring alternatives such as those present in state-of-the-art systems such as Amazon EC2 and OpenNebula.
---
paper_title: Monitoring virtual networks with Lattice

paper_content:

The use of the Lattice monitoring framework as a fundamental part of a overlay management system for virtual networks is presented. Lattice has been specially designed for monitoring resources and services in virtualized environments, including virtual networks. Monitoring of virtualized resources and services has many criteria which are not relevant for monitoring systems that are used for traditional fixed resources. We present the main aspects of the framework together with details of measurement transmission and meta-data encoding. Finally, the use of the Lattice framework for monitoring virtual machines executing under hypervisor control is presented.
---
paper_title: Varanus: In Situ Monitoring for Large Scale Cloud Systems

paper_content:

Monitoring is an essential aspect of maintaining and developing computer systems which increases in difficulty proportional to the size of the system. The need for robust monitoring tools has become more evident with the advent of cloud computing. Infrastructure as a Service (IaaS) clouds allow end users to deploy vast numbers of virtual machines as part of dynamic and transient architectures. Current monitoring solutions, including many of those in the open-source domain, rely on outdated concepts including manual configuration and centralised data collection and adapt poorly to membership churn. In this paper we propose the development of a cloud monitoring system to provide scalable and robust lookup, data collection and analysis services for large-scale cloud systems. In lieu of centrally managed monitoring we propose a multi-tier architecture using a layered gossip protocol to aggregate monitoring information and facilitate lookup, information collection and the identification of redundant capacity. This allows for a resource aware data collection and storage architecture that operates over the system being monitored. This in turn enables monitoring to be done in situ without the need for significant additional infrastructure to facilitate monitoring services. We evaluate this approach against alternative monitoring paradigms and demonstrate how our solution is well adapted to usage in a cloud-computing context.
---

Title: Observing the Clouds: A Survey and Taxonomy of Cloud Monitoring

Section 1: Introduction
Description 1: Write an introduction explaining the significance of cloud monitoring, outlining the challenges, and describing the paper's structure and goals.

Section 2: Monitoring
Description 2: Describe the traditional monitoring process and its application to cloud monitoring, including the stages of collection, analysis, and decision-making.

Section 3: Motivation for Cloud Monitoring
Description 3: Explain the importance of cloud monitoring and detail the unique behaviors and properties of cloud environments that necessitate specific monitoring approaches.

Section 4: Cloud Monitoring Requirements
Description 4: Present a set of requirements for cloud monitoring tools derived from literature and standards, highlighting aspects like scalability, cloud awareness, fault tolerance, autonomic behavior, granularity, comprehensiveness, and time sensitivity.

Section 5: Survey of General Monitoring Systems
Description 5: Survey monitoring systems that predate cloud computing but are used in cloud monitoring, detailing their architecture, functionality, and relevance.

Section 6: Cloud Monitoring Systems
Description 6: Survey monitoring tools designed specifically for cloud computing, describing their architecture, features, and how they address cloud-specific challenges.

Section 7: Monitoring as a Service Tools
Description 7: Survey recent developments in monitoring as a service tools, which abstract the complexity of monitoring away from the user, describing their features and usage.

Section 8: Taxonomy
Description 8: Extrapolate a taxonomy from the surveyed tools, categorizing them based on architecture, communication mechanisms, and origin, and analyze how well current tools address cloud monitoring issues.

Section 9: Monitoring as an Engineering Practice
Description 9: Discuss monitoring from a practical standpoint, considering socio-technical and organizational concerns, and the implementation of effective monitoring strategies.

Section 10: Conclusion
Description 10: Conclude the paper by summarizing the findings, stressing the importance of developing suitable monitoring strategies and tools for cloud environments, and pointing out future research directions.