Auto-Scaling Web Applications in Clouds: A Taxonomy and Survey
---
paper_title: PEAS: A Performance Evaluation Framework for Auto-Scaling Strategies in Cloud Applications

paper_content:

Numerous auto-scaling strategies have been proposed in the past few years for improving various Quality of Service (QoS) indicators of cloud applications, for example, response time and throughput, by adapting the amount of resources assigned to the application to meet the workload demand. However, the evaluation of a proposed auto-scaler is usually achieved through experiments under specific conditions and seldom includes extensive testing to account for uncertainties in the workloads and unexpected behaviors of the system. These tests by no means can provide guarantees about the behavior of the system in general conditions. In this article, we present a Performance Evaluation framework for Auto-Scaling (PEAS) strategies in the presence of uncertainties. The evaluation is formulated as a chance constrained optimization problem, which is solved using scenario theory. The adoption of such a technique allows one to give probabilistic guarantees of the obtainable performance. Six different auto-scaling strategies have been selected from the literature for extensive test evaluation and compared using the proposed framework. We build a discrete event simulator and parameterize it based on real experiments. Using the simulator, each auto-scaler’s performance is evaluated using 796 distinct real workload traces from projects hosted on the Wikimedia foundations’ servers, and their performance is compared using PEAS. The evaluation is carried out using different performance metrics, highlighting the flexibility of the framework, while providing probabilistic bounds on the evaluation and the performance of the algorithms. Our results highlight the problem of generalizing the conclusions of the original published studies and show that based on the evaluation criteria, a controller can be shown to be better than other controllers.
---
paper_title: Autonomic Provisioning with Self-Adaptive Neural Fuzzy Control for End-to-end Delay Guarantee

paper_content:

Autonomic server provisioning for performance assurance is a critical issue in data centers. It is important but challenging to guarantee an important performance metric, percentile-based end-to-end delay of requests flowing through a virtualized multi-tier server cluster. It is mainly due to dynamically varying workload and the lack of an accurate system performance model. In this paper, we propose a novel autonomic server allocation approach based on a model-independent and self-adaptive neural fuzzy control. There are model-independent fuzzy controllers that utilize heuristic knowledge in the form of rule base for performance assurance. Those controllers are designed manually on trial and error basis, often not effective in the face of highly dynamic workloads. We design the neural fuzzy controller as a hybrid of control theoretical and machine learning techniques. It is capable of self-constructing its structure and adapting its parameters through fast online learning. Unlike other supervised machine learning techniques, it does not require off-line training. We further enhance the neural fuzzy controller to compensate for the effect of server switching delays. Extensive simulations demonstrate the effectiveness of our new approach in achieving the percentile-based end-to-end delay guarantees. Compared to a rule-based fuzzy controller enabled server allocation approach, the new approach delivers superior performance in the face of highly dynamic workloads. It is robust to workload variation, change in delay target and server switching delays.
---
paper_title: A Cost-Aware Elasticity Provisioning System for the Cloud

paper_content:

In this paper we present Kingfisher, a {\em cost-aware} system that provides efficient support for elasticity in the cloud by (i) leveraging multiple mechanisms to reduce the time to transition to new configurations, and (ii) optimizing the selection of a virtual server configuration that minimizes the cost. We have implemented a prototype of Kingfisher and have evaluated its efficacy on a laboratory cloud platform. Our experiments with varying application workloads demonstrate that Kingfisher is able to (i) decrease the cost of virtual server resources by as much as $24\%$ compared to the current cost-unaware approach, (ii) reduce by an order of magnitude the time to transition to a new configuration through multiple elasticity mechanisms in the cloud, and (iii), illustrate the opportunity for design alternatives which trade-off the cost of server resources with the time required to scale the application.
---
paper_title: Adaptive resource provisioning for read intensive multi-tier applications in the cloud

paper_content:

A Service-Level Agreement (SLA) provides surety for specific quality attributes to the consumers of services. However, current SLAs offered by cloud infrastructure providers do not address response time, which, from the user's point of view, is the most important quality attribute for Web applications. Satisfying a maximum average response time guarantee for Web applications is difficult for two main reasons: first, traffic patterns are highly dynamic and difficult to predict accurately; second, the complex nature of multi-tier Web applications increases the difficulty of identifying bottlenecks and resolving them automatically. This paper proposes a methodology and presents a working prototype system for automatic detection and resolution of bottlenecks in a multi-tier Web application hosted on a cloud in order to satisfy specific maximum response time requirements. It also proposes a method for identifying and retracting over-provisioned resources in multi-tier cloud-hosted Web applications. We demonstrate the feasibility of the approach in an experimental evaluation with a testbed EUCALYPTUS-based cloud and a synthetic workload. Automatic bottleneck detection and resolution under dynamic resource management has the potential to enable cloud infrastructure providers to provide SLAs for Web applications that guarantee specific response time requirements while minimizing resource utilization.
---
paper_title: Autonomic mix-aware provisioning for non-stationary data center workloads

paper_content:

Online Internet applications see dynamic workloads that fluctuate over multiple time scales. This paper argues that the non-stationarity in Internet application workloads, which causes the request mix to change over time, can have a significant impact on the overall processing demands imposed on data center servers. We propose a novel mix-aware dynamic provisioning technique that handles both the non-stationarity in the workload as well as changes in request volumes when allocating server capacity in Internet data centers. Our technique employs the k-means clustering algorithm to automatically determine the workload mix and a queuing model to predict the server capacity for a given workload mix. We implement a prototype provisioning system that incorporates our technique and experimentally evaluate its efficacy on a laboratory Linux data center running the TPC-W web benchmark. Our results show that our k-means clustering technique accurately captures workload mix changes in Internet applications. We also demonstrate that mix-aware dynamic provisioning eliminates SLA violations due to under-provisioning with non-stationary web workloads, and that it offers a better resource usage by reducing over-provisioning when compared to a baseline provisioning approach that only reacts to workload volume changes. We also present a case study of our provisioning approach on Amazon's EC2 cloud platform.
---
paper_title: A Decentralized Autonomic Architecture for Performance Control in the Cloud

paper_content:

In this paper, we introduce a decentralized autonomic architecture for multi-tier applications deployed in cloud environments. The architecture maintains the application's service level objective at a predefined level and, implicitly, reduces the cost. The architecture uses a series of autonomic controllers, in which each controller independently regulates a tier of the application. The architecture utilizes feedback loops and implements Proportional, Integrative and Derivative control laws at each autonomic controller. A prototype is described and an initial set of experiments is conducted on a public commercial cloud. The experiments demonstrate the effectiveness of this approach at maintaining a service level objective through the decomposition of an application's aggregate performance into its set of discretely managed component tiers.
---
paper_title: Agile dynamic provisioning of multi-tier Internet applications

paper_content:

Dynamic capacity provisioning is a useful technique for handling the multi-time-scale variations seen in Internet workloads. In this article, we propose a novel dynamic provisioning technique for multi-tier Internet applications that employs (1) a flexible queuing model to determine how much of the resources to allocate to each tier of the application, and (2) a combination of predictive and reactive methods that determine when to provision these resources, both at large and small time scales. We propose a novel data center architecture based on virtual machine monitors to reduce provisioning overheads. Our experiments on a forty-machine Xen/Linux-based hosting platform demonstrate the responsiveness of our technique in handling dynamic workloads. In one scenario where a flash crowd caused the workload of a three-tier application to double, our technique was able to double the application capacity within five minutes, thus maintaining response-time targets. Our technique also reduced the overhead of switching servers across applications from several minutes to less than a second, while meeting the performance targets of residual sessions.
---
paper_title: A Regression-Based Analytic Model for Dynamic Resource Provisioning of Multi-Tier Applications

paper_content:

The multi-tier implementation has become the industry standard for developing scalable client-server enterprise applications. Since these applications are performance sensitive, effective models for dynamic resource provisioning and for delivering quality of service to these applications become critical. Workloads in such environments are characterized by client sessions of interdependent requests with changing transaction mix and load over time, making model adaptivity to the observed workload changes a critical requirement for model effectiveness. In this work, we apply a regression-based approximation of the CPU demand of client transactions on a given hardware. Then we use this approximation in an analytic model of a simple network of queues, each queue representing a tier, and show the approximation's effectiveness for modeling diverse workloads with a changing transaction mix over time. Using the TPC- W benchmark and its three different transaction mixes we investigate factors that impact the efficiency and accuracy of the proposed performance prediction models. Experimental results show that this regression-based approach provides a simple and powerful solution for efficient capacity planning and resource provisioning of multi-tier applications under changing workload conditions.
---
paper_title: Generating Adaptation Policies for Multi-tier Applications in Consolidated Server Environments

paper_content:

Creating good adaptation policies is critical to building complex autonomic systems since it is such policies that define the system configuration used in any given situation. While online approaches based on control theory and rule- based expert systems are possible solutions, each has its disadvantages. Here, a hybrid approach is described that uses modeling and optimization offline to generate suitable configurations, which are then encoded as policies that are used at runtime. The approach is demonstrated on the problem of providing dynamic management in virtualized consolidated server environments that host multiple multi-tier applications. Contributions include layered queuing models for Xen-based virtual machine environments, a novel optimization technique that uses a combination of bin packing and gradient search, and experimental results that show that automatic offline policy generation is viable and can be accurate even with modest computational effort.
---
paper_title: Enabling cost-aware and adaptive elasticity of multi-tier cloud applications

paper_content:

Elasticity (on-demand scaling) of applications is one of the most important features of cloud computing. This elasticity is the ability to adaptively scale resources up and down in order to meet varying application demands. To date, most existing scaling techniques can maintain applications' Quality of Service (QoS) but do not adequately address issues relating to minimizing the costs of using the service. In this paper, we propose an elastic scaling approach that makes use of cost-aware criteria to detect and analyse the bottlenecks within multi-tier cloud-based applications. We present an adaptive scaling algorithm that reduces the costs incurred by users of cloud infrastructure services, allowing them to scale their applications only at bottleneck tiers, and present the design of an intelligent platform that automates the scaling process. Our approach is generic for a wide class of multi-tier applications, and we demonstrate its effectiveness against other approaches by studying the behaviour of an example e-commerce application using a standard workload benchmark.
---
paper_title: Provisioning multi-tier cloud applications using statistical bounds on sojourn time

paper_content:

In this paper we present a simple and effective approach for resource provisioning to achieve a percentile bound on the end to end response time of a multi-tier application. We, at first, model the multi-tier application as an open tandem network of M/G/1-PS queues and develop a method that produces a near optimal application configuration, i.e, number of servers at each tier, to meet the percentile bound in a homogeneous server environment -- using a single type of server. We then extend our solution to a K-server case and our technique demonstrates a good accuracy, independent of the variability of service-times. Our approach demonstrates a provisioning error of no more than 3% compared to a 140% worst case provisioning error obtained by techniques based on an M/M/1-FCFS queue model. In addition, we extend our approach to handle a heterogenous server environment, i.e., with multiple types of servers. We find that fewer high-capacity servers are preferable for high percentile provisioning. Finally, we extend our approach to account for the rental cost of each server-type and compute a cost efficient application configuration with savings of over 80%. We demonstrate the applicability of our approach in a real world system by employing it to provision the two tiers of the java implementation of TPC-W -- a multi-tier transactional web benchmark that represents an e-commerce web application, i.e. an online bookstore.
---
paper_title: Efficient server provisioning with end-to-end delay guarantee on multi-tier clusters

paper_content:

Dynamic server provisioning is critical to quality-of-service assurance for multi-tier Internet applications. In this paper, we address three important and challenging problems. First, we propose an efficient server provisioning approach on multi-tier clusters based on an end-to-end resource allocation optimization model. It is to minimize the number of servers allocated to the system while the average end-to-end delay guarantee is satisfied. Second, we design a model-independent fuzzy controller for bounding an important performance metric, the 90 th -percentile delay of requests flowing through the multi-tier architecture. Third, to compensate for the latency due to the dynamic addition of servers, we design a self-tuning component that adaptively adjusts the output scaling factor of the fuzzy controller according to the transient behavior of the end-to-end delay. Extensive simulation results, using one representative customer behavior model in a typical three-tier web cluster, demonstrate that the provisioning approach is able to significantly reduce the server utilization compared to an existing representative approach. The approach integrated with the model-independent self-tuning fuzzy controller can efficiently assure the average and the 90 th -percentile end-to-end delay guarantees on multi-tier server clusters.
---
paper_title: Automated control for elastic n-tier workloads based on empirical modeling

paper_content:

Elastic n-tier applications have non-stationary workloads that require adaptive control of resources allocated to them. This presents not only an opportunity in pay-as-you-use clouds, but also a challenge to dynamically allocate virtual machines appropriately. Previous approaches based on control theory, queuing networks, and machine learning work well for some situations, but each model has its own limitations due to inaccuracies in performance prediction. In this paper we propose a multi-model controller, which integrates adaptation decisions from several models, choosing the best. The focus of our work is an empirical model, based on detailed measurement data from previous application runs. The main advantage of the empirical model is that it returns high quality performance predictions based on measured data. For new application scenarios, we use other models or heuristics as a starting point, and all performance data are continuously incorporated into the empirical model's knowledge base. Using a prototype implementation of the multi-model controller, a cloud testbed, and an n-tier benchmark (RUBBoS), we evaluated and validated the advantages of the empirical model. For example, measured data show that it is more effective to add two nodes as a group, one for each tier, when two tiers approach saturation simultaneously.
---
paper_title: A resource elasticity framework for QoS-aware execution of cloud applications

paper_content:

Abstract Cloud infrastructures consisting of heterogeneous resources are increasingly being utilized for hosting large-scale distributed applications from diverse users with discrete needs. The multifarious cloud applications impose varied demands for computational resources along with multitude of performance implications. Successful hosting of cloud applications necessitates service providers to take into account the heterogeneity existing in the behavior of users, applications and system resources while respecting the user’s agreed Quality of Service (QoS) criteria. In this work, we propose a QoS-Aware Resource Elasticity (QRE) framework that allows service providers to make an assessment of the application behavior and develop mechanisms that enable dynamic scalability of cloud resources hosting the application components. Experimental results conducted on the Amazon EC2 cloud clearly demonstrate the effectiveness of our approach while complying with the agreed QoS attributes of users.
---
paper_title: Automated control of multiple virtualized resources

paper_content:

Virtualized data centers enable sharing of resources among hosted applications. However, it is difficult to satisfy service-level objectives(SLOs) of applications on shared infrastructure, as application workloads and resource consumption patterns change over time. In this paper, we present AutoControl, a resource control system that automatically adapts to dynamic workload changes to achieve application SLOs. AutoControl is a combination of an online model estimator and a novel multi-input, multi-output (MIMO) resource controller. The model estimator captures the complex relationship between application performance and resource allocations, while the MIMO controller allocates the right amount of multiple virtualized resources to achieve application SLOs. Our experimental evaluation with RUBiS and TPC-W benchmarks along with production-trace-driven workloads indicates that AutoControl can detect and mitigate CPU and disk I/O bottlenecks that occur over time and across multiple nodes by allocating each resource accordingly. We also show that AutoControl can be used to provide service differentiation according to the application priorities during resource contention.
---
paper_title: Autonomous resource provisioning for multi-service web applications

paper_content:

Dynamic resource provisioning aims at maintaining the end-to-end response time of a web application within a pre-defined SLA. Although the topic has been well studied for monolithic applications, provisioning resources for applications composed of multiple services remains a challenge. When the SLA is violated, one must decide which service(s) should be reprovisioned for optimal effect. We propose to assign an SLA only to the front-end service. Other services are not given any particular response time objectives. Services are autonomously responsible for their own provisioning operations and collaboratively negotiate performance objectives with each other to decide the provisioning service(s). We demonstrate through extensive experiments that our system can add/remove/shift both servers and caches within an entire multi-service application under varying workloads to meet the SLA target and improve resource utilization.
---
paper_title: Multi-Cloud Provisioning and Load Distribution for Three-Tier Applications

paper_content:

Cloud data centers are becoming the preferred deployment environment for a wide range of business applications because they provide many benefits compared to private in-house infrastructure. However, the traditional approach of using a single cloud has several limitations in terms of availability, avoiding vendor lock-in, and providing legislation-compliant services with suitable Quality of Experience (QoE) to users worldwide. One way for cloud clients to mitigate these issues is to use multiple clouds (i.e., a Multi-Cloud). In this article, we introduce an approach for deploying three-tier applications across multiple clouds in order to satisfy their key nonfunctional requirements. We propose adaptive, dynamic, and reactive resource provisioning and load distribution algorithms that heuristically optimize overall cost and response delays without violating essential legislative and regulatory requirements. Our simulation with realistic workload, network, and cloud characteristics shows that our method improves the state of the art in terms of availability, regulatory compliance, and QoE with acceptable sacrifice in cost and latency.
---
paper_title: Scalability and Performance of Web Applications in a Compute Cloud

paper_content:

Scalability and performance are key factors to the success of many enterprises involved in doing business on the web. Maintaining sufficient web resources just to meet performance during peak demands can be costly. Compute Cloud provides a powerful environment to allow dynamic scaling of web applications without the needs for user intervention. In this paper, we present a case study on the scalability and performance of web applications in a Cloud. We describe a novel dynamic scaling architecture with a front-end load-balancer for routing user requests to web applications deployed on virtual machine instances with the goal of maximizing resource utilization in instances while minimizing total number of instances. A scaling algorithm for automated provisioning of virtual resources based on threshold number of active user sessions will be introduced. The on-demand capability of the Cloud to rapidly provision and dynamically allocate resources to users will be discussed. Our work has demonstrated the compelling benefits of a Cloud which is capable of sustaining performance upon sudden load surges, delivering satisfactory IT resources on-demands to users, and maintaining high resource utilization, thus reducing infrastructure and management costs.
---
paper_title: A Feedback-Control Approach for Resource Management in Public Clouds

paper_content:

Nowadays, more and more the industry and market depend on cloud-based infrastructures for delivering IT services. To this aim cloud-based infrastructures are changing continuously, increasing their complexity especially for the management of cloud resources. Control and management of resources (e.g., virtual machines, VMs) are of paramount importance to adjust resources automatically allocated to an application and for delivering quality-assured services to final users. In this paper, we propose a feedback-based control approach for the management of VMs in the AWS EC2 public cloud. First, we evaluate the proposed Gain Scheduling policy against different workloads. Second, we provide results on the robustness of the proposed Gain Scheduling policy in presence of failures. Finally, we compare our approach to state-of-the-art control approaches for cloud resources. Our results indicate that the proposed control strategy guarantees, without the need of a priori information on system dynamics or complex estimations of the operating conditions, high performance with respect to both constant and time- varying workloads as well as in spite of sudden VM failures.
---
paper_title: Self-adaptive and self-configured CPU resource provisioning for virtualized servers using Kalman filters

paper_content:

Data center virtualization allows cost-effective server consolidation which can increase system throughput and reduce power consumption. Resource management of virtualized servers is an important and challenging task, especially when dealing with fluctuating workloads and complex multi-tier server applications. Recent results in control theory-based resource management have shown the potential benefits of adjusting allocations to match changing workloads. This paper presents a new resource management scheme that integrates the Kalman filter into feedback controllers to dynamically allocate CPU resources to virtual machines hosting server applications. The novelty of our approach is the use of the Kalman filter-the optimal filtering technique for state estimation in the sum of squares sense-to track the CPU utilizations and update the allocations accordingly. Our basic controllers continuously detect and self-adapt to unforeseen workload intensity changes. Our more advanced controller self-configures itself to any workload condition without any a priori information. Indicatively, it results in within 4.8% of the performance of workload-aware controllers under high intensity workload changes, and performs equally well under medium intensity traffic. In addition, our controllers are enhanced to deal with multi-tier server applications: by using the pair-wise resource coupling between application components, they provide a 3% on average server performance improvement when facing large unexpected workload increases when compared to controllers with no such resource-coupling mechanism. We evaluate our techniques by controlling a 3-tier Rubis benchmark web site deployed on a prototype Xen-virtualized cluster.
---
paper_title: Yaksha: a self-tuning controller for managing the performance of 3-tiered Web sites

paper_content:

Managing the performance of multiple-tiered Web sites under high client loads is a critical problem with the advent of dynamic content and database-driven servers on the Internet. This paper presents a control-theoretic approach for admission control in multitiered Web sites that both prevents overload and enforces absolute client response times, while still maintaining high throughput under load. We use classical control theoretic techniques to design a proportional integral (PI) controller for admission control of client HTTP requests. In addition, we present a processor-sharing model that is used to make the controller self-tuning, so that no parameter setting is required beyond a target response time. Our controller is implemented as a proxy, called Yaksha, which operates by taking simple external measurements of the client response times. Our design is noninvasive and requires minimal operator intervention. We evaluate our techniques experimentally using a 3-tiered dynamic content Web site as a testbed. Using the industry standard TPC-W client workload generator, we study the performance of the PI admission controller with extensive experiments. We show that the controller effectively bounds the response times of requests for dynamic content while still maintaining high throughput levels, even when the client request rate is many times that of the server's maximum processing rate. We demonstrate the effectiveness of our self-tuning mechanism, showing that it responds and adapts smoothly to changes in the workload.
---
paper_title: A multi-model framework to implement self-managing control systems for QoS management

paper_content:

Many control theory based approaches have been proposed to provide QoS assurance in increasingly complex software systems. These approaches generally use single model based, fixed or adaptive control techniques for QoS management of such systems. With varying system dynamics and unpredictable environmental changes, however, it is difficult to design a single model or controller to achieve the desired QoS performance across all the operating regions of these systems. In this paper, we propose a multi-model framework to capture the multi-model nature of software systems and implement self-managing control systems for them. A reference-model and extendable class library are introduced to implement such self-managing control systems. The proposed approach is also validated and compared to fixed and adaptive control schemes through a range of experiments.
---
paper_title: Workload Classification for Efficient Auto-Scaling of Cloud Resources

paper_content:

Elasticity algorithms for cloud infrastructures dynamically change the amount of resources allocated to a running service according to the current and predicted future load. Since there is no perfe ...
---
paper_title: Self-Adaptive and Online QoS Modeling for Cloud-Based Software Services

paper_content:

In the presence of scale, dynamism, uncertainty and elasticity, cloud software engineers faces several challenges when modeling Quality of Service (QoS) for cloud-based software services. These challenges can be best managed through self-adaptivity because engineers’ intervention is difficult, if not impossible, given the dynamic and uncertain QoS sensitivity to the environment and control knobs in the cloud. This is especially true for the shared infrastructure of cloud, where unexpected interference can be caused by co-located software services running on the same virtual machine; and co-hosted virtual machines within the same physical machine. In this paper, we describe the related challenges and present a fully dynamic, self-adaptive and online QoS modeling approach, which grounds on sound information theory and machine learning algorithms, to create QoS model that is capable to predict the QoS value as output over time by using the information on environmental conditions, control knobs and interference as inputs. In particular, we report on in-depth analysis on the correlations of selected inputs to the accuracy of QoS model in cloud. To dynamically selects inputs to the models at runtime and tune accuracy, we design self-adaptive hybrid dual-learners that partition the possible inputs space into two sub-spaces, each of which applies different symmetric uncertainty based selection techniques; the results of sub-spaces are then combined. Subsequently, we propose the use of adaptive multi-learners for building the model. These learners simultaneously allow several learning algorithms to model the QoS function, permitting the capability for dynamically selecting the best model for prediction on the fly. We experimentally evaluate our models in the cloud environment using RUBiS benchmark and realistic FIFA 98 workload. The results show that our approach is more accurate and effective than state-of-the-art modelings.
---
paper_title: Autonomic mix-aware provisioning for non-stationary data center workloads

paper_content:

Online Internet applications see dynamic workloads that fluctuate over multiple time scales. This paper argues that the non-stationarity in Internet application workloads, which causes the request mix to change over time, can have a significant impact on the overall processing demands imposed on data center servers. We propose a novel mix-aware dynamic provisioning technique that handles both the non-stationarity in the workload as well as changes in request volumes when allocating server capacity in Internet data centers. Our technique employs the k-means clustering algorithm to automatically determine the workload mix and a queuing model to predict the server capacity for a given workload mix. We implement a prototype provisioning system that incorporates our technique and experimentally evaluate its efficacy on a laboratory Linux data center running the TPC-W web benchmark. Our results show that our k-means clustering technique accurately captures workload mix changes in Internet applications. We also demonstrate that mix-aware dynamic provisioning eliminates SLA violations due to under-provisioning with non-stationary web workloads, and that it offers a better resource usage by reducing over-provisioning when compared to a baseline provisioning approach that only reacts to workload volume changes. We also present a case study of our provisioning approach on Amazon's EC2 cloud platform.
---
paper_title: A Decentralized Autonomic Architecture for Performance Control in the Cloud

paper_content:

In this paper, we introduce a decentralized autonomic architecture for multi-tier applications deployed in cloud environments. The architecture maintains the application's service level objective at a predefined level and, implicitly, reduces the cost. The architecture uses a series of autonomic controllers, in which each controller independently regulates a tier of the application. The architecture utilizes feedback loops and implements Proportional, Integrative and Derivative control laws at each autonomic controller. A prototype is described and an initial set of experiments is conducted on a public commercial cloud. The experiments demonstrate the effectiveness of this approach at maintaining a service level objective through the decomposition of an application's aggregate performance into its set of discretely managed component tiers.
---
paper_title: A Regression-Based Analytic Model for Dynamic Resource Provisioning of Multi-Tier Applications

paper_content:

The multi-tier implementation has become the industry standard for developing scalable client-server enterprise applications. Since these applications are performance sensitive, effective models for dynamic resource provisioning and for delivering quality of service to these applications become critical. Workloads in such environments are characterized by client sessions of interdependent requests with changing transaction mix and load over time, making model adaptivity to the observed workload changes a critical requirement for model effectiveness. In this work, we apply a regression-based approximation of the CPU demand of client transactions on a given hardware. Then we use this approximation in an analytic model of a simple network of queues, each queue representing a tier, and show the approximation's effectiveness for modeling diverse workloads with a changing transaction mix over time. Using the TPC- W benchmark and its three different transaction mixes we investigate factors that impact the efficiency and accuracy of the proposed performance prediction models. Experimental results show that this regression-based approach provides a simple and powerful solution for efficient capacity planning and resource provisioning of multi-tier applications under changing workload conditions.
---
paper_title: Adaptive, Model-driven Autoscaling for Cloud Applications

paper_content:

Applications with a dynamic workload demand need access to a flexible infrastructure to meet performance guarantees and minimize resource costs. While cloud computing provides the elasticity to scale the infrastructure on demand, cloud service providers lack control and visibility of user space applications, making it difficult to accurately scale the underlying infrastructure. Thus, the burden of scaling falls on the user. In this paper, we propose a new cloud service, Dependable Compute Cloud (DC2), that automatically scales the infrastructure to meet the user-specified performance requirements. DC2 employs Kalman filtering to automatically learn the (possibly changing) system parameters for each application, allowing it to proactively scale the infrastructure to meet performance guarantees. DC2 is designed for the cloud it is application-agnostic and does not require any offline application profiling or benchmarking. Our implementation results on OpenStack using a multi-tier application under a range of workload traces demonstrate the robustness and superiority of DC2 over existing rule-based approaches.
---
paper_title: Enabling cost-aware and adaptive elasticity of multi-tier cloud applications

paper_content:

Elasticity (on-demand scaling) of applications is one of the most important features of cloud computing. This elasticity is the ability to adaptively scale resources up and down in order to meet varying application demands. To date, most existing scaling techniques can maintain applications' Quality of Service (QoS) but do not adequately address issues relating to minimizing the costs of using the service. In this paper, we propose an elastic scaling approach that makes use of cost-aware criteria to detect and analyse the bottlenecks within multi-tier cloud-based applications. We present an adaptive scaling algorithm that reduces the costs incurred by users of cloud infrastructure services, allowing them to scale their applications only at bottleneck tiers, and present the design of an intelligent platform that automates the scaling process. Our approach is generic for a wide class of multi-tier applications, and we demonstrate its effectiveness against other approaches by studying the behaviour of an example e-commerce application using a standard workload benchmark.
---
paper_title: Prediction-based real-time resource provisioning for massively multiplayer online games

paper_content:

Massively Multiplayer Online Games (MMOGs) are a class of computationally intensive client-server applications with severe real-time Quality of Service (QoS) requirements, such as the number of updates per second each client needs to receive from the servers for a fluent and realistic experience. To guarantee the QoS requirements, game providers currently over-provision a large amount of their resources, which makes the overall efficiency of provisioning and utilization of resources rather low and prohibits any but the largest providers from joining the market. To address this deficiency, we propose a new prediction-based method for dynamic resource provisioning and scaling of MMOGs in distributed Grid environments. Firstly, a load prediction service anticipates the future game world entity distribution from historical trace data using a fast and flexible neural network-based method. On top of it, we developed generic analytical game load models used to foresee future hot-spots that congest the game servers and make the overall environment fragmented and unplayable. Finally, a resource allocation service performs dynamic load distribution, balancing, and migration of entities that keep the game servers reasonably loaded such that the real-time QoS requirements are maintained. Experimental results based on a realistic simulation environment demonstrate the advantages of our prediction service compared to other conventional methods, especially due to its ability to adapt to different user load patterns, and a reduction of the average over-allocation from 250% (in the case of static over-provisioning) to around 25% using our dynamic provisioning method.
---
paper_title: A resource elasticity framework for QoS-aware execution of cloud applications

paper_content:

Abstract Cloud infrastructures consisting of heterogeneous resources are increasingly being utilized for hosting large-scale distributed applications from diverse users with discrete needs. The multifarious cloud applications impose varied demands for computational resources along with multitude of performance implications. Successful hosting of cloud applications necessitates service providers to take into account the heterogeneity existing in the behavior of users, applications and system resources while respecting the user’s agreed Quality of Service (QoS) criteria. In this work, we propose a QoS-Aware Resource Elasticity (QRE) framework that allows service providers to make an assessment of the application behavior and develop mechanisms that enable dynamic scalability of cloud resources hosting the application components. Experimental results conducted on the Amazon EC2 cloud clearly demonstrate the effectiveness of our approach while complying with the agreed QoS attributes of users.
---
paper_title: Autonomous resource provisioning for multi-service web applications

paper_content:

Dynamic resource provisioning aims at maintaining the end-to-end response time of a web application within a pre-defined SLA. Although the topic has been well studied for monolithic applications, provisioning resources for applications composed of multiple services remains a challenge. When the SLA is violated, one must decide which service(s) should be reprovisioned for optimal effect. We propose to assign an SLA only to the front-end service. Other services are not given any particular response time objectives. Services are autonomously responsible for their own provisioning operations and collaboratively negotiate performance objectives with each other to decide the provisioning service(s). We demonstrate through extensive experiments that our system can add/remove/shift both servers and caches within an entire multi-service application under varying workloads to meet the SLA target and improve resource utilization.
---
paper_title: An Architecture for Automatic Scaling of Replicated Services

paper_content:

Replicated services that allow to scale dynamically can adapt to requests load. Choosing the right number of replicas is fundamental to avoid performance worsening when input spikes occur and to save resources when the load is low. Current mechanisms for automatic scaling are mostly based on fixed thresholds on CPU and memory usage, which are not sufficiently accurate and often entail late countermeasures. We propose Make Your Service Elastic (MYSE), an architecture for automatic scaling of generic replicated services based on queuing models for accurate response time estimation. Requests and service times patterns are analyzed to learn and predict over time their distribution so as to allow for early scaling. A novel heuristic is proposed to avoid the flipping phenomenon. We carried out simulations that show promising results for what concerns the effectiveness of our approach.
---
paper_title: On the Use of Fuzzy Modeling in Virtualized Data Center Management

paper_content:

One of the most important goals of data-center management is to reduce cost through efficient use of resources. Virtualization techniques provide the opportunity of carving individual physical servers into multiple virtual containers that can be run and managed separately. A key challenge that comes with virtualization is the simultaneous on-demand provisioning of shared resources to virtual containers and the management of their capacities to meet service quality targets at the least cost. This paper proposes a two-level resource management system with local controllers at the virtual-container level and a global controller at the resource-pool level. Autonomic resource allocation is realized through the interaction of the local and global controllers. A novelty of the controller designs is their use of fuzzy logic to efficiently and robustly deal with the complexity of the virtualized data center and the uncertainties of the dynamically changing workloads. Experimental results obtained through a prototype implementation demonstrate that, for the scenarios under consideration, the proposed resource management system can significantly reduce resource consumption while still achieving application performance targets.
---
paper_title: SmartScale: Automatic Application Scaling in Enterprise Clouds

paper_content:

Enterprise clouds today support an on demand resource allocation model and can provide resources requested by applications in a near online manner using virtual machine resizing or cloning. However, in order to take advantage of an on demand resource model, enterprise applications need to be automatically scaled in a way that makes the most efficient use of resources. In this work, we present the SmartScale automated scaling framework. SmartScale uses a combination of vertical (adding more resources to existing VM instances) and horizontal (adding more VM instances) scaling to ensure that the application is scaled in a manner that optimizes both resource usage and the reconfiguration cost incurred due to scaling. The SmartScale methodology is proactive and ensures that the application converges quickly to the desired scaling level even when the workload intensity changes significantly. We evaluate SmartScale using real production traces on Olio, an emerging cloud benchmark, running on a ???-based cloud testbed. We present both theoretical and experimental evidence that comprehensively establish the effectiveness of SmartScale.
---
paper_title: Efficient Autoscaling in the Cloud Using Predictive Models for Workload Forecasting

paper_content:

Large-scale component-based enterprise applications that leverage Cloud resources expect Quality of Service(QoS) guarantees in accordance with service level agreements between the customer and service providers. In the context of Cloud computing, auto scaling mechanisms hold the promise of assuring QoS properties to the applications while simultaneously making efficient use of resources and keeping operational costs low for the service providers. Despite the perceived advantages of auto scaling, realizing the full potential of auto scaling is hard due to multiple challenges stemming from the need to precisely estimate resource usage in the face of significant variability in client workload patterns. This paper makes three contributions to overcome the general lack of effective techniques for workload forecasting and optimal resource allocation. First, it discusses the challenges involved in auto scaling in the cloud. Second, it develops a model-predictive algorithm for workload forecasting that is used for resource auto scaling. Finally, empirical results are provided that demonstrate that resources can be allocated and deal located by our algorithm in a way that satisfies both the application QoS while keeping operational costs low.
---
paper_title: Adaptive, Model-driven Autoscaling for Cloud Applications

paper_content:

Applications with a dynamic workload demand need access to a flexible infrastructure to meet performance guarantees and minimize resource costs. While cloud computing provides the elasticity to scale the infrastructure on demand, cloud service providers lack control and visibility of user space applications, making it difficult to accurately scale the underlying infrastructure. Thus, the burden of scaling falls on the user. In this paper, we propose a new cloud service, Dependable Compute Cloud (DC2), that automatically scales the infrastructure to meet the user-specified performance requirements. DC2 employs Kalman filtering to automatically learn the (possibly changing) system parameters for each application, allowing it to proactively scale the infrastructure to meet performance guarantees. DC2 is designed for the cloud it is application-agnostic and does not require any offline application profiling or benchmarking. Our implementation results on OpenStack using a multi-tier application under a range of workload traces demonstrate the robustness and superiority of DC2 over existing rule-based approaches.
---
paper_title: VScaler: Autonomic Virtual Machine Scaling

paper_content:

Recent research results in cloud community found that cloud users increasingly force providers to shift from fixed bundle instance types(e.g. Amazon instances) to flexible bundles and shrinked billing cycles. This means that cloud applications can dynamically provision the used amount of resources in a more fine-grained fashion. This observation calls for approaches which are able to automatically implement fine granular VM resource allocation with respect to user-provided SLAs. In this work we propose VScaler, a framework which implements autonomic resource allocation using a novel approach to reinforcement learning.
---
paper_title: Prediction-based real-time resource provisioning for massively multiplayer online games

paper_content:

Massively Multiplayer Online Games (MMOGs) are a class of computationally intensive client-server applications with severe real-time Quality of Service (QoS) requirements, such as the number of updates per second each client needs to receive from the servers for a fluent and realistic experience. To guarantee the QoS requirements, game providers currently over-provision a large amount of their resources, which makes the overall efficiency of provisioning and utilization of resources rather low and prohibits any but the largest providers from joining the market. To address this deficiency, we propose a new prediction-based method for dynamic resource provisioning and scaling of MMOGs in distributed Grid environments. Firstly, a load prediction service anticipates the future game world entity distribution from historical trace data using a fast and flexible neural network-based method. On top of it, we developed generic analytical game load models used to foresee future hot-spots that congest the game servers and make the overall environment fragmented and unplayable. Finally, a resource allocation service performs dynamic load distribution, balancing, and migration of entities that keep the game servers reasonably loaded such that the real-time QoS requirements are maintained. Experimental results based on a realistic simulation environment demonstrate the advantages of our prediction service compared to other conventional methods, especially due to its ability to adapt to different user load patterns, and a reduction of the average over-allocation from 250% (in the case of static over-provisioning) to around 25% using our dynamic provisioning method.
---
paper_title: Cloud QoS Scaling by Fuzzy Logic

paper_content:

One of the biggest advantages of cloud infrastructures is the elasticity. Cloud services are monitored and based on the resource utilization and performance load, they get scaled up or down, by provision or de-provision of cloud resources. The goal is to guarantee the customers an acceptable performance with a minimum of resources. Such Quality of Service (QoS) characteristics are stated in a contract, called Service Level Agreement (SLA) negotiated between customer and provider. The approach of this paper shows that with additional imprecise information (e.g. expected daytime/week- time performance) modeled with fuzzy logic and used in a behavior, load and performance prediction model, the up and down scaling mechanism of a cloud service can be optimized. Evaluation results confirm, that using this approach, SLA violation can be minimized.
---
paper_title: Automated control of multiple virtualized resources

paper_content:

Virtualized data centers enable sharing of resources among hosted applications. However, it is difficult to satisfy service-level objectives(SLOs) of applications on shared infrastructure, as application workloads and resource consumption patterns change over time. In this paper, we present AutoControl, a resource control system that automatically adapts to dynamic workload changes to achieve application SLOs. AutoControl is a combination of an online model estimator and a novel multi-input, multi-output (MIMO) resource controller. The model estimator captures the complex relationship between application performance and resource allocations, while the MIMO controller allocates the right amount of multiple virtualized resources to achieve application SLOs. Our experimental evaluation with RUBiS and TPC-W benchmarks along with production-trace-driven workloads indicates that AutoControl can detect and mitigate CPU and disk I/O bottlenecks that occur over time and across multiple nodes by allocating each resource accordingly. We also show that AutoControl can be used to provide service differentiation according to the application priorities during resource contention.
---
paper_title: Autoscaling Web Applications in Heterogeneous Cloud Infrastructures

paper_content:

Improving resource provisioning of heterogeneous cloud infrastructures is an important research challenge. The wide diversity of cloud-based applications and customers with different QoS requirements have recently exhibited the weaknesses of current provisioning systems. Today's cloud infrastructures provide provisioning systems that dynamically adapt the computational power of applications by adding or releasing resources. Unfortunately, these scaling systems are fairly limited:(i) They restrict themselves to a single type of resource, (ii)they are unable to fulfill QoS requirements in face of spiky workload, and (iii) they offer the same QoS level to all their customers, independent of customer preferences such as different levels of service availability and performance. In this paper, we present an autoscaling system that overcomes these limitations by exploiting heterogeneous types of resources, and by defining multiple levels of QoS requirements. The proposed system selects a resource scaling plan according to both workload and customer requirements. Our experiments conducted on both public and private infrastructures show significant reductions in QoS-level violations when faced with highly variable workloads.
---
paper_title: DEPAS: a decentralized probabilistic algorithm for auto-scaling

paper_content:

The dynamic provisioning of virtualized resources offered by cloud computing infrastructures allows applications deployed in a cloud environment to automatically increase and decrease the amount of used resources. This capability is called auto-scaling and its main purpose is to automatically adjust the scale of the system that is running the application to satisfy the varying workload with minimum resource utilization. The need for auto-scaling is particularly important during workload peaks, in which applications may need to scale up to extremely large-scale systems. Both the research community and the main cloud providers have already developed auto-scaling solutions. However, most research solutions are centralized and not suitable for managing large-scale systems, moreover cloud providers' solutions are bound to the limitations of a specific provider in terms of resource prices, availability, reliability, and connectivity. In this paper we propose DEPAS, a decentralized probabilistic auto-scaling algorithm integrated into a P2P architecture that is cloud provider independent, thus allowing the auto-scaling of services over multiple cloud infrastructures at the same time. Our simulations, which are based on real service traces, show that our approach is capable of: (i) keeping the overall utilization of all the instantiated cloud resources in a target range, (ii) maintaining service response times close to the ones obtained using optimal centralized auto-scaling approaches.
---
paper_title: Impact of CPU Utilization Thresholds and Scaling Size on Autoscaling Cloud Resources

paper_content:

Cloud computing is currently one of the most hyped information technology fields and it has become one of the fastest growing segments of IT. A cloud introduces a resource-rich computing model with features such as flexibility, pay per use, elasticity, scalability, and others. In the context of cloud computing, auto scaling and elasticity are methods used to assure SLO (Service Level Objectives) for cloud services as well as the efficient usage of resources. There are many factors related to the auto scaling mechanism that might affect the performance of the cloud services. One of such important factors is the setting of CPU thresholds that control the triggering of the auto scaling policies, for the purpose of adding or terminating resources from the auto-scaling group. Another important factor is the scaling size, which is the number of instances that will be added every time such provisioning process takes place to add more resources to cope with workload spikes. In this paper, we simulate and study the impact of setting the upper CPU utilization threshold and the scaling size factors on the performance of the cloud services. Another contribution of this paper is on formulating and solving optimization problems for tuning these parameters based on input loads, considering both the cost and SLO response time. The study helps in deciding about the optimal setting that enables the use of the least number of cloud resources to satisfy QoS or SLO requirements.
---
paper_title: SLA-Driven Adaptive Resource Management for Web Applications on a Heterogeneous Compute Cloud

paper_content:

Current service-level agreements (SLAs) offered by cloud providers make guarantees about quality attributes such as availability. However, although one of the most important quality attributes from the perspective of the users of a cloud-based Web application is its response time, current SLAs do not guarantee response time. Satisfying a maximum average response time guarantee for Web applications is difficult due to unpredictable traffic patterns, but in this paper we show how it can be accomplished through dynamic resource allocation in a virtual Web farm. We present the design and implementation of a working prototype built on a EUCALYPTUS-based heterogeneous compute cloud that actively monitors the response time of each virtual machine assigned to the farm and adaptively scales up the application to satisfy a SLA promising a specific average response time. We demonstrate the feasibility of the approach in an experimental evaluation with a testbed cloud and a synthetic workload. Adaptive resource management has the potential to increase the usability of Web applications while maximizing resource utilization.
---
paper_title: Automated control in cloud computing: challenges and opportunities

paper_content:

With advances in virtualization technology, virtual machine services offered by cloud utility providers are becoming increasingly powerful, anchoring the ecosystem of cloud services. Virtual computing services are attractive in part because they enable customers to acquire and release computing resources for guest applications adaptively in response to load surges and other dynamic behaviors. ``Elastic'' cloud computing APIs present a natural opportunity for feedback controllers to automate this adaptive resource provisioning, and many recent works have explored feedback control policies for a variety of network services under various assumptions. This paper addresses the challenge of building an effective controller as a customer add-on outside of the cloud utility service itself. Such external controllers must function within the constraints of the utility service APIs. It is important to consider techniques for effective feedback control using cloud APIs, as well as how to design those APIs to enable more effective control. As one example, we explore proportional thresholding, a policy enhancement for feedback controllers that enables stable control across a wide range of guest cluster sizes using the coarse-grained control offered by popular virtual compute cloud services.
---
paper_title: Automated control for elastic storage

paper_content:

Elasticity - where systems acquire and release resources in response to dynamic workloads, while paying only for what they need - is a driving property of cloud computing. At the core of any elastic system is an automated controller. This paper addresses elastic control for multi-tier application services that allocate and release resources in discrete units, such as virtual server instances of predetermined sizes. It focuses on elastic control of the storage tier, in which adding or removing a storage node or "brick" requires rebalancing stored data across the nodes. The storage tier presents new challenges for elastic control: actuator delays (lag) due to rebalancing, interference with applications and sensor measurements, and the need to synchronize the multiple control elements, including rebalancing. We have designed and implemented a new controller for elastic storage systems to address these challenges. Using a popular distributed storage system - the Hadoop Distributed File System (HDFS) - under dynamic Web 2.0 workloads, we show how the controller adapts to workload changes to maintain performance objectives efficiently in a pay-as-you-go cloud computing environment.
---
paper_title: Elastic Virtual Machine for Fine-Grained Cloud Resource Provisioning

paper_content:

Elasticity is one of the distinguishing characteristics associated with Cloud computing emergence. It enables cloud resources to auto-scale to cope with workload demand. Multi-instances horizontal scaling is the common scalability architecture in Cloud; however, its current implementation is coarse-grained, while it considers Virtual Machine (VM) as a scaling unit, this implies additional scaling-out overhead and limits it to specific applications. To overcome these limitations, we propose Elastic VM as a fine-grained vertical scaling architecture. Our results proved that Elastic VM architecture implies less consumption of resources, mitigates Service Level Objectives (SLOs) violation, and avoids scaling-up overhead. Furthermore, it scales broader range of applications including databases.
---
paper_title: Exploiting User Patience for Scaling Resource Capacity in Cloud Services

paper_content:

An important feature of cloud computing is its elasticity, that is, the ability to have resource capacity dynamically modified according to the current system load. Auto-scaling is challenging because it must account for two conflicting objectives: minimising system capacity available to users and maximising QoS, which typically translates to short response times. Current auto-scaling techniques are based solely on load forecasts and ignore the perception that users have from cloud services. As a consequence, providers tend to provision a volume of resources that is significantly larger than necessary to keep users satisfied. In this article, we propose a scheduling algorithm and an auto-scaling triggering technique that explore user patience in order to identify critical times when auto-scaling is needed and the appropriate volume of capacity by which the cloud platform should either extend or shrink. The proposed technique assists service providers in reducing costs related to resource allocation while keeping the same QoS to users. Our experiments show that it is possible to reduce resource-hour by up to approximately 8% compared to auto-scaling based on system utilisation.
---
paper_title: Lightweight Resource Scaling for Cloud Applications

paper_content:

Elastic resource provisioning is a key feature of cloud computing, allowing users to scale up or down resource allocation for their applications at run-time. To date, most practical approaches to managing elasticity are based on allocation/de-allocation of the virtual machine (VM) instances to the application. This VM-level elasticity typically incurs both considerable overhead and extra costs, especially for applications with rapidly fluctuating demands. In this paper, we propose a lightweight approach to enable cost-effective elasticity for cloud applications. Our approach operates fine-grained scaling at the resource level itself (CPUs, memory, I/O, etc) in addition to VM-level scaling. We also present the design and implementation of an intelligent platform for light-weight resource management of cloud applications. We describe our algorithms for light-weight scaling and VM-level scaling and show their interaction. We then use an industry standard benchmark to evaluate the effectiveness of our approach and compare its performance against traditional approaches.
---
paper_title: Evaluating Auto-scaling Strategies for Cloud Computing Environments

paper_content:

Auto-scaling is a key feature in clouds responsible for adjusting the number of available resources to meet service demand. Resource pool modifications are necessary to keep performance indicators, such as utilisation level, between user-defined lower and upper bounds. Auto-scaling strategies that are not properly configured according to user workload characteristics may lead to unacceptable QoS and large resource waste. As a consequence, there is a need for a deeper understanding of auto-scaling strategies and how they should be configured to minimise these problems. In this work, we evaluate various auto-scaling strategies using log traces from a production Google data centre cluster comprising millions of jobs. Using utilisation level as performance indicator, our results show that proper management of auto-scaling parameters reduces the difference between the target utilisation interval and the actual values -- we define such difference as Auto-scaling Demand Index. We also present a set of lessons from this study to help cloud providers build recommender systems for auto-scaling operations.
---
paper_title: On the Use of Fuzzy Modeling in Virtualized Data Center Management

paper_content:

One of the most important goals of data-center management is to reduce cost through efficient use of resources. Virtualization techniques provide the opportunity of carving individual physical servers into multiple virtual containers that can be run and managed separately. A key challenge that comes with virtualization is the simultaneous on-demand provisioning of shared resources to virtual containers and the management of their capacities to meet service quality targets at the least cost. This paper proposes a two-level resource management system with local controllers at the virtual-container level and a global controller at the resource-pool level. Autonomic resource allocation is realized through the interaction of the local and global controllers. A novelty of the controller designs is their use of fuzzy logic to efficiently and robustly deal with the complexity of the virtualized data center and the uncertainties of the dynamically changing workloads. Experimental results obtained through a prototype implementation demonstrate that, for the scenarios under consideration, the proposed resource management system can significantly reduce resource consumption while still achieving application performance targets.
---
paper_title: Autonomic Provisioning with Self-Adaptive Neural Fuzzy Control for End-to-end Delay Guarantee

paper_content:

Autonomic server provisioning for performance assurance is a critical issue in data centers. It is important but challenging to guarantee an important performance metric, percentile-based end-to-end delay of requests flowing through a virtualized multi-tier server cluster. It is mainly due to dynamically varying workload and the lack of an accurate system performance model. In this paper, we propose a novel autonomic server allocation approach based on a model-independent and self-adaptive neural fuzzy control. There are model-independent fuzzy controllers that utilize heuristic knowledge in the form of rule base for performance assurance. Those controllers are designed manually on trial and error basis, often not effective in the face of highly dynamic workloads. We design the neural fuzzy controller as a hybrid of control theoretical and machine learning techniques. It is capable of self-constructing its structure and adapting its parameters through fast online learning. Unlike other supervised machine learning techniques, it does not require off-line training. We further enhance the neural fuzzy controller to compensate for the effect of server switching delays. Extensive simulations demonstrate the effectiveness of our new approach in achieving the percentile-based end-to-end delay guarantees. Compared to a rule-based fuzzy controller enabled server allocation approach, the new approach delivers superior performance in the face of highly dynamic workloads. It is robust to workload variation, change in delay target and server switching delays.
---
paper_title: Cloud QoS Scaling by Fuzzy Logic

paper_content:

One of the biggest advantages of cloud infrastructures is the elasticity. Cloud services are monitored and based on the resource utilization and performance load, they get scaled up or down, by provision or de-provision of cloud resources. The goal is to guarantee the customers an acceptable performance with a minimum of resources. Such Quality of Service (QoS) characteristics are stated in a contract, called Service Level Agreement (SLA) negotiated between customer and provider. The approach of this paper shows that with additional imprecise information (e.g. expected daytime/week- time performance) modeled with fuzzy logic and used in a behavior, load and performance prediction model, the up and down scaling mechanism of a cloud service can be optimized. Evaluation results confirm, that using this approach, SLA violation can be minimized.
---
paper_title: Efficient server provisioning with end-to-end delay guarantee on multi-tier clusters

paper_content:

Dynamic server provisioning is critical to quality-of-service assurance for multi-tier Internet applications. In this paper, we address three important and challenging problems. First, we propose an efficient server provisioning approach on multi-tier clusters based on an end-to-end resource allocation optimization model. It is to minimize the number of servers allocated to the system while the average end-to-end delay guarantee is satisfied. Second, we design a model-independent fuzzy controller for bounding an important performance metric, the 90 th -percentile delay of requests flowing through the multi-tier architecture. Third, to compensate for the latency due to the dynamic addition of servers, we design a self-tuning component that adaptively adjusts the output scaling factor of the fuzzy controller according to the transient behavior of the end-to-end delay. Extensive simulation results, using one representative customer behavior model in a typical three-tier web cluster, demonstrate that the provisioning approach is able to significantly reduce the server utilization compared to an existing representative approach. The approach integrated with the model-independent self-tuning fuzzy controller can efficiently assure the average and the 90 th -percentile end-to-end delay guarantees on multi-tier server clusters.
---
paper_title: Managing Uncertainty in Autonomic Cloud Elasticity Controllers

paper_content:

Elasticity allows a cloud system to maintain an optimal user experience by automatically acquiring and releasing resources. Autoscaling-adding or removing resources automatically on the fly-involves specifying threshold-based rules to implement elasticity policies. However, the elasticity rules must be specified through quantitative values, which requires cloud resource management knowledge and expertise. Furthermore, existing approaches don't explicitly deal with uncertainty in cloud-based software, where noise and unexpected events are common. The authors propose a control-theoretic approach that manages the behavior of a cloud environment as a dynamic system. They integrate a fuzzy cloud controller with an online learning mechanism, putting forward a framework that takes the human out of the dynamic adaptation loop and can cope with various sources of uncertainty in the cloud.
---
paper_title: A Cost-Aware Elasticity Provisioning System for the Cloud

paper_content:

In this paper we present Kingfisher, a {\em cost-aware} system that provides efficient support for elasticity in the cloud by (i) leveraging multiple mechanisms to reduce the time to transition to new configurations, and (ii) optimizing the selection of a virtual server configuration that minimizes the cost. We have implemented a prototype of Kingfisher and have evaluated its efficacy on a laboratory cloud platform. Our experiments with varying application workloads demonstrate that Kingfisher is able to (i) decrease the cost of virtual server resources by as much as $24\%$ compared to the current cost-unaware approach, (ii) reduce by an order of magnitude the time to transition to a new configuration through multiple elasticity mechanisms in the cloud, and (iii), illustrate the opportunity for design alternatives which trade-off the cost of server resources with the time required to scale the application.
---
paper_title: AGILE: elastic distributed resource scaling for Infrastructure-as-a-Service

paper_content:

Dynamically adjusting the number of virtual machines (VMs) assigned to a cloud application to keep up with load changes and interference from other uses typically requires detailed application knowledge and an ability to know the future, neither of which are readily available to infrastructure service providers or application owners. The result is that systems need to be over-provisioned (costly), or risk missing their performance Service Level Objectives (SLOs) and have to pay penalties (also costly). AGILE deals with both issues: it uses wavelets to provide a medium-term resource demand prediction with enough lead time to start up new application server instances before performance falls short, and it uses dynamic VM cloning to reduce application startup times. Tests using RUBiS and Google cluster traces show that AGILE can predict varying resource demands over the medium-term with up to 3.42× better true positive rate and 0.34× the false positive rate than existing schemes. Given a target SLO violation rate, AGILE can efficiently handle dynamic application workloads, reducing both penalties and user dissatisfaction.
---
paper_title: A Reliable and Cost-Efficient Auto-Scaling System for Web Applications Using Heterogeneous Spot Instances

paper_content:

Cloud providers sell their idle capacity on markets through an auction-like mechanism to increase their return on investment. The instances sold in this way are called spot instances. In spite that spot instances are usually 90% cheaper than on-demand instances, they can be terminated by provider when their bidding prices are lower than market prices. Thus, they are largely used to provision fault-tolerant applications only. In this paper, we explore how to utilize spot instances to provision web applications, which are usually considered as availability-critical. The idea is to take advantage of differences in price among various types of spot instances to reach both high availability and significant cost saving. We first propose a fault-tolerant model for web applications provisioned by spot instances. Based on that, we devise novel cost-efficient auto-scaling polices that comply with the defined fault-tolerant semantics for hourly billed cloud markets. We implemented the proposed model and policies both on a simulation testbed for repeatable validation and Amazon EC2. The experiments on the simulation testbed and EC2 show that the proposed approach can greatly reduce resource cost and still achieve satisfactory Quality of Service (QoS) in terms of response time and availability.
---
paper_title: DejaVu: accelerating resource allocation in virtualized environments

paper_content:

Effective resource management of virtualized environments is a challenging task. State-of-the-art management systems either rely on analytical models or evaluate resource allocations by running actual experiments. However, both approaches incur a significant overhead once the workload changes. The former needs to re-calibrate and re-validate models, whereas the latter has to run a new set of experiments to select a new resource allocation. During the adaptation period, the system may run with an inefficient configuration. In this paper, we propose DejaVu - a framework that (1) minimizes the resource management overhead by identifying a small set of workload classes for which it needs to evaluate resource allocation decisions, (2) quickly adapts to workload changes by classifying workloads using signatures and caching their preferred resource allocations at runtime, and (3) deals with interference by estimating an "interference index". We evaluate DejaVu by running representative network services on Amazon EC2. DejaVu achieves more than 10x speedup in adaptation time for each workload change relative to the state-of-the-art. By enabling quick adaptation, DejaVu saves up to 60% of the service provisioning cost. Finally, DejaVu is easily deployable as it does not require any extensive instrumentation or human intervention.
---
paper_title: AutoScale: Dynamic, Robust Capacity Management for Multi-Tier Data Centers

paper_content:

Energy costs for data centers continue to rise, already exceeding $15 billion yearly. Sadly much of this power is wasted. Servers are only busy 10--30p of the time on average, but they are often left on, while idle, utilizing 60p or more of peak power when in the idle state. We introduce a dynamic capacity management policy, AutoScale, that greatly reduces the number of servers needed in data centers driven by unpredictable, time-varying load, while meeting response time SLAs. AutoScale scales the data center capacity, adding or removing servers as needed. AutoScale has two key features: (i) it autonomically maintains just the right amount of spare capacity to handle bursts in the request rate; and (ii) it is robust not just to changes in the request rate of real-world traces, but also request size and server efficiency. We evaluate our dynamic capacity management approach via implementation on a 38-server multi-tier data center, serving a web site of the type seen in Facebook or Amazon, with a key-value store workload. We demonstrate that AutoScale vastly improves upon existing dynamic capacity management policies with respect to meeting SLAs and robustness.
---
paper_title: Autoscaling Web Applications in Heterogeneous Cloud Infrastructures

paper_content:

Improving resource provisioning of heterogeneous cloud infrastructures is an important research challenge. The wide diversity of cloud-based applications and customers with different QoS requirements have recently exhibited the weaknesses of current provisioning systems. Today's cloud infrastructures provide provisioning systems that dynamically adapt the computational power of applications by adding or releasing resources. Unfortunately, these scaling systems are fairly limited:(i) They restrict themselves to a single type of resource, (ii)they are unable to fulfill QoS requirements in face of spiky workload, and (iii) they offer the same QoS level to all their customers, independent of customer preferences such as different levels of service availability and performance. In this paper, we present an autoscaling system that overcomes these limitations by exploiting heterogeneous types of resources, and by defining multiple levels of QoS requirements. The proposed system selects a resource scaling plan according to both workload and customer requirements. Our experiments conducted on both public and private infrastructures show significant reductions in QoS-level violations when faced with highly variable workloads.
---
paper_title: An adaptive hybrid elasticity controller for cloud infrastructures

paper_content:

Cloud elasticity is the ability of the cloud infrastructure to rapidly change the amount of resources allocated to a service in order to meet the actual varying demands on the service while enforcing SLAs. In this paper, we focus on horizontal elasticity, the ability of the infrastructure to add or remove virtual machines allocated to a service deployed in the cloud. We model a cloud service using queuing theory. Using that model we build two adaptive proactive controllers that estimate the future load on a service. We explore the different possible scenarios for deploying a proactive elasticity controller coupled with a reactive elasticity controller in the cloud. Using simulation with workload traces from the FIFA world-cup web servers, we show that a hybrid controller that incorporates a reactive controller for scale up coupled with our proactive controllers for scale down decisions reduces SLA violations by a factor of 2 to 10 compared to a regression based controller or a completely reactive controller.
---
paper_title: A Decentralized Autonomic Architecture for Performance Control in the Cloud

paper_content:

In this paper, we introduce a decentralized autonomic architecture for multi-tier applications deployed in cloud environments. The architecture maintains the application's service level objective at a predefined level and, implicitly, reduces the cost. The architecture uses a series of autonomic controllers, in which each controller independently regulates a tier of the application. The architecture utilizes feedback loops and implements Proportional, Integrative and Derivative control laws at each autonomic controller. A prototype is described and an initial set of experiments is conducted on a public commercial cloud. The experiments demonstrate the effectiveness of this approach at maintaining a service level objective through the decomposition of an application's aggregate performance into its set of discretely managed component tiers.
---
paper_title: Agile dynamic provisioning of multi-tier Internet applications

paper_content:

Dynamic capacity provisioning is a useful technique for handling the multi-time-scale variations seen in Internet workloads. In this article, we propose a novel dynamic provisioning technique for multi-tier Internet applications that employs (1) a flexible queuing model to determine how much of the resources to allocate to each tier of the application, and (2) a combination of predictive and reactive methods that determine when to provision these resources, both at large and small time scales. We propose a novel data center architecture based on virtual machine monitors to reduce provisioning overheads. Our experiments on a forty-machine Xen/Linux-based hosting platform demonstrate the responsiveness of our technique in handling dynamic workloads. In one scenario where a flash crowd caused the workload of a three-tier application to double, our technique was able to double the application capacity within five minutes, thus maintaining response-time targets. Our technique also reduced the overhead of switching servers across applications from several minutes to less than a second, while meeting the performance targets of residual sessions.
---
paper_title: A Regression-Based Analytic Model for Dynamic Resource Provisioning of Multi-Tier Applications

paper_content:

The multi-tier implementation has become the industry standard for developing scalable client-server enterprise applications. Since these applications are performance sensitive, effective models for dynamic resource provisioning and for delivering quality of service to these applications become critical. Workloads in such environments are characterized by client sessions of interdependent requests with changing transaction mix and load over time, making model adaptivity to the observed workload changes a critical requirement for model effectiveness. In this work, we apply a regression-based approximation of the CPU demand of client transactions on a given hardware. Then we use this approximation in an analytic model of a simple network of queues, each queue representing a tier, and show the approximation's effectiveness for modeling diverse workloads with a changing transaction mix over time. Using the TPC- W benchmark and its three different transaction mixes we investigate factors that impact the efficiency and accuracy of the proposed performance prediction models. Experimental results show that this regression-based approach provides a simple and powerful solution for efficient capacity planning and resource provisioning of multi-tier applications under changing workload conditions.
---
paper_title: Efficient Autoscaling in the Cloud Using Predictive Models for Workload Forecasting

paper_content:

Large-scale component-based enterprise applications that leverage Cloud resources expect Quality of Service(QoS) guarantees in accordance with service level agreements between the customer and service providers. In the context of Cloud computing, auto scaling mechanisms hold the promise of assuring QoS properties to the applications while simultaneously making efficient use of resources and keeping operational costs low for the service providers. Despite the perceived advantages of auto scaling, realizing the full potential of auto scaling is hard due to multiple challenges stemming from the need to precisely estimate resource usage in the face of significant variability in client workload patterns. This paper makes three contributions to overcome the general lack of effective techniques for workload forecasting and optimal resource allocation. First, it discusses the challenges involved in auto scaling in the cloud. Second, it develops a model-predictive algorithm for workload forecasting that is used for resource auto scaling. Finally, empirical results are provided that demonstrate that resources can be allocated and deal located by our algorithm in a way that satisfies both the application QoS while keeping operational costs low.
---
paper_title: Modeling the Impact of Workload on Cloud Resource Scaling

paper_content:

Cloud computing offers the flexibility to dynamically size the infrastructure in response to changes in workload demand. While both horizontal and vertical scaling of infrastructure is supported by major cloud providers, these scaling options differ significantly in terms of their cost, provisioning time, and their impact on workload performance. Importantly, the efficacy of horizontal and vertical scaling critically depends on the workload characteristics, such as the workload's parallelizability and its core scalability. In today's cloud systems, the scaling decision is left to the users, requiring them to fully understand the tradeoffs associated with the different scaling options. In this paper, we present our solution for optimizing the resource scaling of cloud deployments via implementation in OpenStack. The key component of our solution is the modelling engine that characterizes the workload and then quantitatively evaluates different scaling options for that workload. Our modelling engine leverages Amdahl's Law to model service time scaling in scaleup environments and queueing-theoretic concepts to model performance scaling in scale-out environments. We further employ Kalman filtering to account for inaccuracies in the model-based methodology, and to dynamically track changes in the workload and cloud environment.
---
paper_title: Adaptive, Model-driven Autoscaling for Cloud Applications

paper_content:

Applications with a dynamic workload demand need access to a flexible infrastructure to meet performance guarantees and minimize resource costs. While cloud computing provides the elasticity to scale the infrastructure on demand, cloud service providers lack control and visibility of user space applications, making it difficult to accurately scale the underlying infrastructure. Thus, the burden of scaling falls on the user. In this paper, we propose a new cloud service, Dependable Compute Cloud (DC2), that automatically scales the infrastructure to meet the user-specified performance requirements. DC2 employs Kalman filtering to automatically learn the (possibly changing) system parameters for each application, allowing it to proactively scale the infrastructure to meet performance guarantees. DC2 is designed for the cloud it is application-agnostic and does not require any offline application profiling or benchmarking. Our implementation results on OpenStack using a multi-tier application under a range of workload traces demonstrate the robustness and superiority of DC2 over existing rule-based approaches.
---
paper_title: Dynamic Provisioning Modeling for Virtualized Multi-tier Applications in Cloud Data Center

paper_content:

Dynamic provisioning is a useful technique for handling the virtualized multi-tier applications in cloud environment. Understanding the performance of virtualized multi-tier applications is crucial for efficient cloud infrastructure management. In this paper, we present a novel dynamic provisioning technique for a cluster-based virtualized multi-tier application that employ a flexible hybrid queueing model to determine the number of virtual machines at each tier in a virtualized application. We present a cloud data center based on virtual machine to optimize resources provisioning. Using simulation experiments of three-tier application, we adopt an optimization model to minimize the total number of virtual machines while satisfying the customer average response time constraint and the request arrival rate constraint. Our experiments show that cloud data center resources can be allocated accurately with these techniques, and the extra cost can be effectively reduced.
---
paper_title: Enabling cost-aware and adaptive elasticity of multi-tier cloud applications

paper_content:

Elasticity (on-demand scaling) of applications is one of the most important features of cloud computing. This elasticity is the ability to adaptively scale resources up and down in order to meet varying application demands. To date, most existing scaling techniques can maintain applications' Quality of Service (QoS) but do not adequately address issues relating to minimizing the costs of using the service. In this paper, we propose an elastic scaling approach that makes use of cost-aware criteria to detect and analyse the bottlenecks within multi-tier cloud-based applications. We present an adaptive scaling algorithm that reduces the costs incurred by users of cloud infrastructure services, allowing them to scale their applications only at bottleneck tiers, and present the design of an intelligent platform that automates the scaling process. Our approach is generic for a wide class of multi-tier applications, and we demonstrate its effectiveness against other approaches by studying the behaviour of an example e-commerce application using a standard workload benchmark.
---
paper_title: Runtime Vertical Scaling of Virtualized Applications via Online Model Estimation

paper_content:

Applications in virtualized data centers are often subject to Service Level Objectives (SLOs) regarding their performance (e.g., latency or throughput). In order to fulfill these SLOs, it is necessary to allocate sufficient resources of different types (CPU, memory, I/O, etc.) to an application. However, the relationship between the application performance and the resource allocation is complex and depends on multiple factors including application architecture, system configuration, and workload demands. In this paper, we present a model-based approach to ensure that the application performance meets the user-defined SLO efficiently by runtime "vertical scaling" (i.e., adding or removing resources) of individual virtual machines (VMs) running the application. A layered performance model describing the relationship between the resource allocation and the observed application performance is automatically extracted and updated online using resource demand estimation techniques. Such a model is then used in a feedback controller to dynamically adapt the number of virtual CPUs of individual VMs. We have implemented the controller on top of the VMware vSphere platform and evaluated it in a case study using a real-world email and groupware server. The experimental results show that our approach allows the managed application to achieve SLO satisfaction in spite of workload demand variation while avoiding oscillations commonly observed with state-of-the-art threshold-based controllers.
---
paper_title: Optimal autoscaling in a IaaS cloud

paper_content:

An application provider leases resources (i.e., virtual machine instances) of variable configurations from a IaaS provider over some lease duration (typically one hour). The application provider (i.e., consumer) would like to minimize their cost while meeting all service level obligations (SLOs). The mechanism of adding and removing resources at runtime is referred to as autoscaling. The process of autoscaling is automated through the use of a management component referred to as an autoscaler. This paper introduces a novel autoscaling approach in which both cloud and application dynamics are modeled in the context of a stochastic, model predictive control problem. The approach exploits trade-off between satisfying performance related objectives for the consumer's application while minimizing their cost. Simulation results are presented demonstrating the efficacy of this new approach.
---
paper_title: Optimal Cloud Resource Auto-Scaling for Web Applications

paper_content:

In the on-demand cloud environment, web application providers have the potential to scale virtual resources up or down to achieve cost-effective outcomes. True elasticity and cost-effectiveness in the pay-per-use cloud business model, however, have not yet been achieved. To address this challenge, we propose a novel cloud resource auto-scaling scheme at the virtual machine (VM) level for web application providers. The scheme automatically predicts the number of web requests and discovers an optimal cloud resource demand with cost-latency trade-off. Based on this demand, the scheme makes a resource scaling decision that is up or down or NOP (no operation) in each time-unit re-allocation. We have implemented the scheme on the Amazon cloud platform and evaluated it using three real-world web log datasets. Our experiment results demonstrate that the proposed scheme achieves resource auto-scaling with an optimal cost-latency trade-off, as well as low SLA violations.
---
paper_title: Provisioning multi-tier cloud applications using statistical bounds on sojourn time

paper_content:

In this paper we present a simple and effective approach for resource provisioning to achieve a percentile bound on the end to end response time of a multi-tier application. We, at first, model the multi-tier application as an open tandem network of M/G/1-PS queues and develop a method that produces a near optimal application configuration, i.e, number of servers at each tier, to meet the percentile bound in a homogeneous server environment -- using a single type of server. We then extend our solution to a K-server case and our technique demonstrates a good accuracy, independent of the variability of service-times. Our approach demonstrates a provisioning error of no more than 3% compared to a 140% worst case provisioning error obtained by techniques based on an M/M/1-FCFS queue model. In addition, we extend our approach to handle a heterogenous server environment, i.e., with multiple types of servers. We find that fewer high-capacity servers are preferable for high percentile provisioning. Finally, we extend our approach to account for the rental cost of each server-type and compute a cost efficient application configuration with savings of over 80%. We demonstrate the applicability of our approach in a real world system by employing it to provision the two tiers of the java implementation of TPC-W -- a multi-tier transactional web benchmark that represents an e-commerce web application, i.e. an online bookstore.
---
paper_title: An Analytical Model for Estimating Cloud Resources of Elastic Services

paper_content:

In the cloud, ensuring proper elasticity for hosted applications and services is a challenging problem and far from being solved. To achieve proper elasticity, the minimal number of cloud resources that are needed to satisfy a particular service level objective (SLO) requirement has to be determined. In this paper, we present an analytical model based on Markov chains to predict the number of cloud instances or virtual machines (VMs) needed to satisfy a given SLO performance requirement such as response time, throughput, or request loss probability. For the estimation of these SLO performance metrics, our analytical model takes the offered workload, the number of VM instances as an input, and the capacity of each VM instance. The correctness of the model has been verified using discrete-event simulation. Our model has also been validated using experimental measurements conducted on the Amazon Web Services cloud platform.
---
paper_title: A resource elasticity framework for QoS-aware execution of cloud applications

paper_content:

Abstract Cloud infrastructures consisting of heterogeneous resources are increasingly being utilized for hosting large-scale distributed applications from diverse users with discrete needs. The multifarious cloud applications impose varied demands for computational resources along with multitude of performance implications. Successful hosting of cloud applications necessitates service providers to take into account the heterogeneity existing in the behavior of users, applications and system resources while respecting the user’s agreed Quality of Service (QoS) criteria. In this work, we propose a QoS-Aware Resource Elasticity (QRE) framework that allows service providers to make an assessment of the application behavior and develop mechanisms that enable dynamic scalability of cloud resources hosting the application components. Experimental results conducted on the Amazon EC2 cloud clearly demonstrate the effectiveness of our approach while complying with the agreed QoS attributes of users.
---
paper_title: Autonomous resource provisioning for multi-service web applications

paper_content:

Dynamic resource provisioning aims at maintaining the end-to-end response time of a web application within a pre-defined SLA. Although the topic has been well studied for monolithic applications, provisioning resources for applications composed of multiple services remains a challenge. When the SLA is violated, one must decide which service(s) should be reprovisioned for optimal effect. We propose to assign an SLA only to the front-end service. Other services are not given any particular response time objectives. Services are autonomously responsible for their own provisioning operations and collaboratively negotiate performance objectives with each other to decide the provisioning service(s). We demonstrate through extensive experiments that our system can add/remove/shift both servers and caches within an entire multi-service application under varying workloads to meet the SLA target and improve resource utilization.
---
paper_title: Yaksha: a self-tuning controller for managing the performance of 3-tiered Web sites

paper_content:

Managing the performance of multiple-tiered Web sites under high client loads is a critical problem with the advent of dynamic content and database-driven servers on the Internet. This paper presents a control-theoretic approach for admission control in multitiered Web sites that both prevents overload and enforces absolute client response times, while still maintaining high throughput under load. We use classical control theoretic techniques to design a proportional integral (PI) controller for admission control of client HTTP requests. In addition, we present a processor-sharing model that is used to make the controller self-tuning, so that no parameter setting is required beyond a target response time. Our controller is implemented as a proxy, called Yaksha, which operates by taking simple external measurements of the client response times. Our design is noninvasive and requires minimal operator intervention. We evaluate our techniques experimentally using a 3-tiered dynamic content Web site as a testbed. Using the industry standard TPC-W client workload generator, we study the performance of the PI admission controller with extensive experiments. We show that the controller effectively bounds the response times of requests for dynamic content while still maintaining high throughput levels, even when the client request rate is many times that of the server's maximum processing rate. We demonstrate the effectiveness of our self-tuning mechanism, showing that it responds and adapts smoothly to changes in the workload.
---
paper_title: An Architecture for Automatic Scaling of Replicated Services

paper_content:

Replicated services that allow to scale dynamically can adapt to requests load. Choosing the right number of replicas is fundamental to avoid performance worsening when input spikes occur and to save resources when the load is low. Current mechanisms for automatic scaling are mostly based on fixed thresholds on CPU and memory usage, which are not sufficiently accurate and often entail late countermeasures. We propose Make Your Service Elastic (MYSE), an architecture for automatic scaling of generic replicated services based on queuing models for accurate response time estimation. Requests and service times patterns are analyzed to learn and predict over time their distribution so as to allow for early scaling. A novel heuristic is proposed to avoid the flipping phenomenon. We carried out simulations that show promising results for what concerns the effectiveness of our approach.
---
paper_title: Provisioning servers in the application tier for e-commerce systems

paper_content:

Server providers that support e-commerce applications as a service to multiple e-commerce websites traditionally use a tiered server architecture. This architecture includes an application tier to process requests that require dynamically generated content. How this tier is provisioned can significantly impact a provider's profit margin. We study methods to provision servers in the application serving tier to increase a server provider's profits. First, we examine actual traces of request arrivals to the application tier of e-commerce sites, and show that the arrival process is effectively Poisson. Next, we construct an optimization problem in the context of a set of application servers modeled as M/G/l/PS queueing systems, and derive three simple methods to approximate the allocation that maximizes profits. Simulation results demonstrate that our approximation methods achieve profits that are close to optimal and are significantly higher than those achieved via simple heuristics.
---
paper_title: Resource provisioning with budget constraints for adaptive applications in cloud environments

paper_content:

Utility computing was a vision stated more than 40 years ago. It refers to the idea that computing resources and services can be delivered, utilized, and paid for as utilities such as water or electricity. The recent emergence of cloud computing is making this vision realizable. Examples of efforts in this area include Infrastructure as a Service (IaaS) providers like Amazon EC2 and Software as a Service (SaaS) providers like Google AppEngine and Microsoft Azure. More recently, there is a growing interest in the use of cloud computing for scientific computing, as evidenced by a large-scale funded project like Magellan, and new workshops.
---
paper_title: Online Resource Allocation Using Decompositional Reinforcement Learning

paper_content:

This paper considers a novel application domain for reinforcement learning: that of "autonomic computing," i.e. selfmanaging computing systems. RL is applied to an online resource allocation task in a distributed multi-application computing environment with independent time-varying load in each application. The task is to allocate servers in real time so as to maximize the sum of performance-based expected utility in each application. This task may be treated as a composite MDP, and to exploit the problem structure, a simple localized RL approach is proposed, with better scalability than previous approaches. The RL approach is tested in a realistic prototype data center comprising real servers, real HTTP requests, and realistic time-varying demand. This domain poses a number of major challenges associated with live training in a real system, including: the need for rapid training, exploration that avoids excessive penalties, and handling complex, potentially non-Markovian system effects. The early results are encouraging: in overnight training, RL performs as well as or slightly better than heavily researched model-based approaches derived from queuing theory.
---
paper_title: Using reinforcement learning for controlling an elastic web application hosting platform

paper_content:

In this paper, we propose and implement a control mechanism that interfaces with Infrastructure as a Service (IaaS) or cloud providers to provision resources and manage instances of web applications in response to volatile and complex request patterns. We use reinforcement learning to orchestrate control actions such as provisioning servers and application placement to meet performance requirements and minimize ongoing costs. The mechanism is incorporated in a distributed, elastic hosting architecture that is evaluated using actual web applications running on resources from Amazon EC2.
---
paper_title: Using Reinforcement Learning for Autonomic Resource Allocation in Clouds: Towards a Fully Automated Workflow

paper_content:

Dynamic and appropriate resource dimensioning isa crucial issue in cloud computing. As applications go more andmore 24/7, online policies must be sought to balance performancewith the cost of allocated virtual machines. Most industrialapproaches to date use ad hoc manual policies, such as thresholdbasedones. Providing good thresholds proved to be tricky andhard to automatize to fit every application requirement. Researchis being done to apply automatic decision-making approaches,such as reinforcement learning. Yet, they face a lot of problemsto go to the field: having good policies in the early phasesof learning, time for the learning to converge to an optimalpolicy and coping with changes in the application performancebehavior over time. In this paper, we propose to deal with theseproblems using appropriate initialization for the early stages aswell as convergence speedups applied throughout the learningphases and we present our first experimental results for these.We also introduce a performance model change detection onwhich we are currently working to complete the learning processmanagement. Even though some of these proposals were knownin the reinforcement learning field, the key contribution of thispaper is to integrate them in a real cloud controller and toprogram them as an automated workflow.
---
paper_title: VScaler: Autonomic Virtual Machine Scaling

paper_content:

Recent research results in cloud community found that cloud users increasingly force providers to shift from fixed bundle instance types(e.g. Amazon instances) to flexible bundles and shrinked billing cycles. This means that cloud applications can dynamically provision the used amount of resources in a more fine-grained fashion. This observation calls for approaches which are able to automatically implement fine granular VM resource allocation with respect to user-provided SLAs. In this work we propose VScaler, a framework which implements autonomic resource allocation using a novel approach to reinforcement learning.
---
paper_title: Unsupervised Learning of Dynamic Resource Provisioning Policies for Cloud-Hosted Multitier Web Applications

paper_content:

Dynamic resource provisioning for Web applications allows for low operational costs while meeting service-level objectives (SLOs). However, the complexity of multitier Web applications makes it difficult to automatically provision resources for each tier without human supervision. In this paper, we introduce unsupervised machine learning methods to dynamically provision multitier Web applications, while observing user-defined performance goals. The proposed technique operates in real time and uses learning techniques to identify workload patterns from access logs, reactively identifies bottlenecks for specific workload patterns, and dynamically builds resource allocation policies for each particular workload. We demonstrate the effectiveness of the proposed approach in several experiments using synthetic workloads on the Amazon Elastic Compute Cloud (EC2) and compare it with industry-standard rule-based autoscale strategies. Our results show that the proposed techniques would enable cloud infrastructure providers or application owners to build systems that automatically manage multitier Web applications, while meeting SLOs, without any prior knowledge of the applications' resource utilization or workload patterns.
---
paper_title: An autonomic resource provisioning approach for service-based cloud applications: A hybrid approach

paper_content:

Abstract In cloud computing environment, resources can be dynamically provisioned on deman for cloud services The amount of the resources to be provisioned is determined during runtime according to the workload changes. Deciding the right amount of resources required to run the cloud services is not trivial, and it depends on the current workload of the cloud services. Therefore, it is necessary to predict the future demands to automatically provision resources in order to deal with fluctuating demands of the cloud services. In this paper, we propose a hybrid resource provisioning approach for cloud services that is based on a combination of the concept of the autonomic computing and the reinforcement learning (RL). Also, we present a framework for autonomic resource provisioning which is inspired by the cloud layer model. Finally, we evaluate the effectiveness of our approach under two real world workload traces. The experimental results show that the proposed approach reduces the total cost by up to 50%, and increases the resource utilization by up to 12% compared with the other approaches.
---
paper_title: NASLA: Novel Auto Scaling Approach based on Learning Automata for Web Application in Cloud Computing Environment

paper_content:

Considering the growing interest in using cloud services, the accessibility and the effective management of the required resources, irrespective of the time and place, seems to be of great importance both to the service providers and users. One of the best ways for increasing utilization and improving the performance of the cloud systems is the auto-scaling of the applications; this is because of the fact that, due to the scalability of cloud computing, on the one hand, cloud providers believe that sufficient resources have to be prepared for the users, and on the other, the users also have a tendency towards the “pay as you go” system of payment for the resources. This paper seeks to offer an approach, based on the learning automata, for the scalability of the web applications, which combines virtual machine clusters and the learning automata in order to provide the best possible way for the scaling up and scaling down of the virtual machines. The results of this study indicate that the proposed approach has decreased the number of SLA violations (in percentage), while it has a smaller load of scalability compared to the other approaches in this regard.
---
paper_title: Applying reinforcement learning towards automating resource allocation and application scalability in the cloud

paper_content:

SUMMARY ::: Public Infrastructure as a Service (IaaS) clouds such as Amazon, GoGrid and Rackspace deliver computational resources by means of virtualisation technologies. These technologies allow multiple independent virtual machines to reside in apparent isolation on the same physical host. Dynamically scaling applications running on IaaS clouds can lead to varied and unpredictable results because of the performance interference effects associated with co-located virtual machines. Determining appropriate scaling policies in a dynamic non-stationary environment is non-trivial. One principle advantage exhibited by IaaS clouds over their traditional hosting counterparts is the ability to scale resources on-demand. However, a problem arises concerning resource allocation as to which resources should be added and removed when the underlying performance of the resource is in a constant state of flux. Decision theoretic frameworks such as Markov Decision Processes are particularly suited to decision making under uncertainty. By applying a temporal difference, reinforcement learning algorithm known as Q-learning, optimal scaling policies can be determined. Additionally, reinforcement learning techniques typically suffer from curse of dimensionality problems, where the state space grows exponentially with each additional state variable. To address this challenge, we also present a novel parallel Q-learning approach aimed at reducing the time taken to determine optimal policies whilst learning online. Copyright © 2012 John Wiley & Sons, Ltd.
---
paper_title: A Review of Auto-scaling Techniques for Elastic Applications in Cloud Environments

paper_content:

Cloud computing environments allow customers to dynamically scale their applications. The key problem is how to lease the right amount of resources, on a pay-as-you-go basis. Application re-dimensioning can be implemented effortlessly, adapting the resources assigned to the application to the incoming user demand. However, the identification of the right amount of resources to lease in order to meet the required Service Level Agreement, while keeping the overall cost low, is not an easy task. Many techniques have been proposed for automating application scaling. We propose a classification of these techniques into five main categories: static threshold-based rules, control theory, reinforcement learning, queuing theory and time series analysis. Then we use this classification to carry out a literature review of proposals for auto-scaling in the cloud.
---
paper_title: Coordinated Self-Configuration of Virtual Machines and Appliances Using a Model-Free Learning Approach

paper_content:

Cloud computing has a key requirement for resource configuration in a real-time manner. In such virtualized environments, both virtual machines (VMs) and hosted applications need to be configured on-the-fly to adapt to system dynamics. The interplay between the layers of VMs and applications further complicates the problem of cloud configuration. Independent tuning of each aspect may not lead to optimal system wide performance. In this paper, we propose a framework, namely CoTuner, for coordinated configuration of VMs and resident applications. At the heart of the framework is a model-free hybrid reinforcement learning (RL) approach, which combines the advantages of Simplex method and RL method and is further enhanced by the use of system knowledge guided exploration policies. Experimental results on Xen-based virtualized environments with TPC-W and TPC-C benchmarks demonstrate that CoTuner is able to drive a virtual server cluster into an optimal or near-optimal configuration state on the fly, in response to the change of workload. It improves the systems throughput by more than 30 percent over independent tuning strategies. In comparison with the coordinated tuning strategies based on basic RL or Simplex algorithm, the hybrid RL algorithm gains 25 to 40 percent throughput improvement.
---
paper_title: From Data Center Resource Allocation to Control Theory and Back

paper_content:

Continuously adjusting the horizontal scaling of applications hosted by data centers appears as a good candidate to automatic control approaches allocating resources in closed-loop given their current workload. Despite several attempts, real applications of these techniques in cloud computing infrastructures face some difficulties. Some of them essentially turn back to the core concepts of automatic control: controllability, inertia of the controlled system, gain and stability. In this paper, considering our recent work to build a management framework dedicated to automatic resource allocation in virtualized applications, we attempt to identify from experiments the sources of instabilities in the controlled systems. As examples, we analyze two types of policies: threshold-based and reinforcement learning techniques to dynamically scale resources. The experiments show that both approaches are tricky and that trying to implement a controller without looking at the way the controlled system reacts to actions, both in time and in amplitude, is doomed to fail. We discuss both lessons learned from the experiments in terms of simple yet key points to build good resource management policies, and longer term issues on which we are currently working to manage contracts and reinforcement learning efficiently in cloud controllers.
---
paper_title: A Feedback-Control Approach for Resource Management in Public Clouds

paper_content:

Nowadays, more and more the industry and market depend on cloud-based infrastructures for delivering IT services. To this aim cloud-based infrastructures are changing continuously, increasing their complexity especially for the management of cloud resources. Control and management of resources (e.g., virtual machines, VMs) are of paramount importance to adjust resources automatically allocated to an application and for delivering quality-assured services to final users. In this paper, we propose a feedback-based control approach for the management of VMs in the AWS EC2 public cloud. First, we evaluate the proposed Gain Scheduling policy against different workloads. Second, we provide results on the robustness of the proposed Gain Scheduling policy in presence of failures. Finally, we compare our approach to state-of-the-art control approaches for cloud resources. Our results indicate that the proposed control strategy guarantees, without the need of a priori information on system dynamics or complex estimations of the operating conditions, high performance with respect to both constant and time- varying workloads as well as in spite of sudden VM failures.
---
paper_title: Energy-Aware Server Provisioning and Load Dispatching for Connection-Intensive Internet Services

paper_content:

Energy consumption in hosting Internet services is becoming a pressing issue as these services scale up. Dynamic server provisioning techniques are effective in turning off unnecessary servers to save energy. Such techniques, mostly studied for request-response services, face challenges in the context of connection servers that host a large number of long-lived TCP connections. In this paper, we characterize unique properties, performance, and power models of connection servers, based on a real data trace collected from the deployed Windows Live Messenger. Using the models, we design server provisioning and load dispatching algorithms and study subtle interactions between them. We show that our algorithms can save a significant amount of energy without sacrificing user experiences.
---
paper_title: Kriging Controllers for Cloud Applications

paper_content:

The infrastructure-as-a-service paradigm for cloud computing lets service providers execute applications on third-party infrastructures with a pay-as-you-go billing model. Providers can balance operational costs and quality of service by monitoring application behavior and changing the deployed configuration at runtime as operating conditions change. Current approaches for automatically scaling cloud applications exploit user-defined rules that respond well to predictable events but don't react adequately to unexpected execution conditions. The authors' autonomic controllers, designed using Kriging models, automatically adapt to unpredicted conditions by dynamically updating a model of the system's behavior.
---
paper_title: Predicting service metrics for cluster-based services using real-time analytics

paper_content:

Predicting the performance of cloud services is intrinsically hard. In this work, we pursue an approach based upon statistical learning, whereby the behaviour of a system is learned from observations. Specifically, our testbed implementation collects device statistics from a server cluster and uses a regression method that accurately predicts, in real-time, client-side service metrics for a video streaming service running on the cluster. The method is service-agnostic in the sense that it takes as input operating-systems statistics instead of service-level metrics. We show that feature set reduction significantly improves prediction accuracy in our case, while simultaneously reducing model computation time. We also discuss design and implementation of a real-time analytics engine, which processes streams of device statistics and service metrics from testbed sensors and produces model predictions through online learning.
---
paper_title: Dynamic Provisioning Modeling for Virtualized Multi-tier Applications in Cloud Data Center

paper_content:

Dynamic provisioning is a useful technique for handling the virtualized multi-tier applications in cloud environment. Understanding the performance of virtualized multi-tier applications is crucial for efficient cloud infrastructure management. In this paper, we present a novel dynamic provisioning technique for a cluster-based virtualized multi-tier application that employ a flexible hybrid queueing model to determine the number of virtual machines at each tier in a virtualized application. We present a cloud data center based on virtual machine to optimize resources provisioning. Using simulation experiments of three-tier application, we adopt an optimization model to minimize the total number of virtual machines while satisfying the customer average response time constraint and the request arrival rate constraint. Our experiments show that cloud data center resources can be allocated accurately with these techniques, and the extra cost can be effectively reduced.
---
paper_title: Automated control of multiple virtualized resources

paper_content:

Virtualized data centers enable sharing of resources among hosted applications. However, it is difficult to satisfy service-level objectives(SLOs) of applications on shared infrastructure, as application workloads and resource consumption patterns change over time. In this paper, we present AutoControl, a resource control system that automatically adapts to dynamic workload changes to achieve application SLOs. AutoControl is a combination of an online model estimator and a novel multi-input, multi-output (MIMO) resource controller. The model estimator captures the complex relationship between application performance and resource allocations, while the MIMO controller allocates the right amount of multiple virtualized resources to achieve application SLOs. Our experimental evaluation with RUBiS and TPC-W benchmarks along with production-trace-driven workloads indicates that AutoControl can detect and mitigate CPU and disk I/O bottlenecks that occur over time and across multiple nodes by allocating each resource accordingly. We also show that AutoControl can be used to provide service differentiation according to the application priorities during resource contention.
---
paper_title: On the use of hybrid reinforcement learning for autonomic resource allocation

paper_content:

Reinforcement Learning (RL) provides a promising new approach to systems performance management that differs radically from standard queuing-theoretic approaches making use of explicit system performance models. In principle, RL can automatically learn high-quality management policies without an explicit performance model or traffic model, and with little or no built-in system specific knowledge. In our original work (Das, R., Tesauro, G., Walsh, W.E.: IBM Research, Tech. Rep. RC23802 (2005), Tesauro, G.: In: Proc. of AAAI-05, pp. 886---891 (2005), Tesauro, G., Das, R., Walsh, W.E., Kephart, J.O.: In: Proc. of ICAC-05, pp. 342---343 (2005)) we showed the feasibility of using online RL to learn resource valuation estimates (in lookup table form) which can be used to make high-quality server allocation decisions in a multi-application prototype Data Center scenario. The present work shows how to combine the strengths of both RL and queuing models in a hybrid approach, in which RL trains offline on data collected while a queuing model policy controls the system. By training offline we avoid suffering potentially poor performance in live online training. We also now use RL to train nonlinear function approximators (e.g. multi-layer perceptrons) instead of lookup tables; this enables scaling to substantially larger state spaces. Our results now show that, in both open-loop and closed-loop traffic, hybrid RL training can achieve significant performance improvements over a variety of initial model-based policies. We also find that, as expected, RL can deal effectively with both transients and switching delays, which lie outside the scope of traditional steady-state queuing theory.
---
paper_title: On the Use of Fuzzy Modeling in Virtualized Data Center Management

paper_content:

One of the most important goals of data-center management is to reduce cost through efficient use of resources. Virtualization techniques provide the opportunity of carving individual physical servers into multiple virtual containers that can be run and managed separately. A key challenge that comes with virtualization is the simultaneous on-demand provisioning of shared resources to virtual containers and the management of their capacities to meet service quality targets at the least cost. This paper proposes a two-level resource management system with local controllers at the virtual-container level and a global controller at the resource-pool level. Autonomic resource allocation is realized through the interaction of the local and global controllers. A novelty of the controller designs is their use of fuzzy logic to efficiently and robustly deal with the complexity of the virtualized data center and the uncertainties of the dynamically changing workloads. Experimental results obtained through a prototype implementation demonstrate that, for the scenarios under consideration, the proposed resource management system can significantly reduce resource consumption while still achieving application performance targets.
---
paper_title: Autonomic Provisioning with Self-Adaptive Neural Fuzzy Control for End-to-end Delay Guarantee

paper_content:

Autonomic server provisioning for performance assurance is a critical issue in data centers. It is important but challenging to guarantee an important performance metric, percentile-based end-to-end delay of requests flowing through a virtualized multi-tier server cluster. It is mainly due to dynamically varying workload and the lack of an accurate system performance model. In this paper, we propose a novel autonomic server allocation approach based on a model-independent and self-adaptive neural fuzzy control. There are model-independent fuzzy controllers that utilize heuristic knowledge in the form of rule base for performance assurance. Those controllers are designed manually on trial and error basis, often not effective in the face of highly dynamic workloads. We design the neural fuzzy controller as a hybrid of control theoretical and machine learning techniques. It is capable of self-constructing its structure and adapting its parameters through fast online learning. Unlike other supervised machine learning techniques, it does not require off-line training. We further enhance the neural fuzzy controller to compensate for the effect of server switching delays. Extensive simulations demonstrate the effectiveness of our new approach in achieving the percentile-based end-to-end delay guarantees. Compared to a rule-based fuzzy controller enabled server allocation approach, the new approach delivers superior performance in the face of highly dynamic workloads. It is robust to workload variation, change in delay target and server switching delays.
---
paper_title: Kriging-Based Self-Adaptive Cloud Controllers

paper_content:

Cloud technology is rapidly substituting classic computing solutions, and challenges the community with new problems. In this paper we focus on controllers for cloud application elasticity, and propose a novel solution for self-adaptive cloud controllers based on Kriging models. Cloud controllers are application specific schedulers that allocate resources to applications running in the cloud, aiming to meet the quality of service requirements while optimizing the execution costs. General-purpose cloud resource schedulers provide sub-optimal solutions to the problem with respect to application-specific solutions that we call cloud controllers. In this paper we discuss a general way to design self-adaptive cloud controllers based on Kriging models. We present Kriging models, and show how they can be used for building efficient controllers thanks to their unique characteristics. We report experimental data that confirm the suitability of Kriging models to support efficient cloud control and open the way to the development of a new generation of cloud controllers.
---
paper_title: A Regression-Based Analytic Model for Dynamic Resource Provisioning of Multi-Tier Applications

paper_content:

The multi-tier implementation has become the industry standard for developing scalable client-server enterprise applications. Since these applications are performance sensitive, effective models for dynamic resource provisioning and for delivering quality of service to these applications become critical. Workloads in such environments are characterized by client sessions of interdependent requests with changing transaction mix and load over time, making model adaptivity to the observed workload changes a critical requirement for model effectiveness. In this work, we apply a regression-based approximation of the CPU demand of client transactions on a given hardware. Then we use this approximation in an analytic model of a simple network of queues, each queue representing a tier, and show the approximation's effectiveness for modeling diverse workloads with a changing transaction mix over time. Using the TPC- W benchmark and its three different transaction mixes we investigate factors that impact the efficiency and accuracy of the proposed performance prediction models. Experimental results show that this regression-based approach provides a simple and powerful solution for efficient capacity planning and resource provisioning of multi-tier applications under changing workload conditions.
---
paper_title: Adaptive, Model-driven Autoscaling for Cloud Applications

paper_content:

Applications with a dynamic workload demand need access to a flexible infrastructure to meet performance guarantees and minimize resource costs. While cloud computing provides the elasticity to scale the infrastructure on demand, cloud service providers lack control and visibility of user space applications, making it difficult to accurately scale the underlying infrastructure. Thus, the burden of scaling falls on the user. In this paper, we propose a new cloud service, Dependable Compute Cloud (DC2), that automatically scales the infrastructure to meet the user-specified performance requirements. DC2 employs Kalman filtering to automatically learn the (possibly changing) system parameters for each application, allowing it to proactively scale the infrastructure to meet performance guarantees. DC2 is designed for the cloud it is application-agnostic and does not require any offline application profiling or benchmarking. Our implementation results on OpenStack using a multi-tier application under a range of workload traces demonstrate the robustness and superiority of DC2 over existing rule-based approaches.
---
paper_title: Self-Adaptive and Online QoS Modeling for Cloud-Based Software Services

paper_content:

In the presence of scale, dynamism, uncertainty and elasticity, cloud software engineers faces several challenges when modeling Quality of Service (QoS) for cloud-based software services. These challenges can be best managed through self-adaptivity because engineers’ intervention is difficult, if not impossible, given the dynamic and uncertain QoS sensitivity to the environment and control knobs in the cloud. This is especially true for the shared infrastructure of cloud, where unexpected interference can be caused by co-located software services running on the same virtual machine; and co-hosted virtual machines within the same physical machine. In this paper, we describe the related challenges and present a fully dynamic, self-adaptive and online QoS modeling approach, which grounds on sound information theory and machine learning algorithms, to create QoS model that is capable to predict the QoS value as output over time by using the information on environmental conditions, control knobs and interference as inputs. In particular, we report on in-depth analysis on the correlations of selected inputs to the accuracy of QoS model in cloud. To dynamically selects inputs to the models at runtime and tune accuracy, we design self-adaptive hybrid dual-learners that partition the possible inputs space into two sub-spaces, each of which applies different symmetric uncertainty based selection techniques; the results of sub-spaces are then combined. Subsequently, we propose the use of adaptive multi-learners for building the model. These learners simultaneously allow several learning algorithms to model the QoS function, permitting the capability for dynamically selecting the best model for prediction on the fly. We experimentally evaluate our models in the cloud environment using RUBiS benchmark and realistic FIFA 98 workload. The results show that our approach is more accurate and effective than state-of-the-art modelings.
---
paper_title: Efficient server provisioning with end-to-end delay guarantee on multi-tier clusters

paper_content:

Dynamic server provisioning is critical to quality-of-service assurance for multi-tier Internet applications. In this paper, we address three important and challenging problems. First, we propose an efficient server provisioning approach on multi-tier clusters based on an end-to-end resource allocation optimization model. It is to minimize the number of servers allocated to the system while the average end-to-end delay guarantee is satisfied. Second, we design a model-independent fuzzy controller for bounding an important performance metric, the 90 th -percentile delay of requests flowing through the multi-tier architecture. Third, to compensate for the latency due to the dynamic addition of servers, we design a self-tuning component that adaptively adjusts the output scaling factor of the fuzzy controller according to the transient behavior of the end-to-end delay. Extensive simulation results, using one representative customer behavior model in a typical three-tier web cluster, demonstrate that the provisioning approach is able to significantly reduce the server utilization compared to an existing representative approach. The approach integrated with the model-independent self-tuning fuzzy controller can efficiently assure the average and the 90 th -percentile end-to-end delay guarantees on multi-tier server clusters.
---
paper_title: Managing Uncertainty in Autonomic Cloud Elasticity Controllers

paper_content:

Elasticity allows a cloud system to maintain an optimal user experience by automatically acquiring and releasing resources. Autoscaling-adding or removing resources automatically on the fly-involves specifying threshold-based rules to implement elasticity policies. However, the elasticity rules must be specified through quantitative values, which requires cloud resource management knowledge and expertise. Furthermore, existing approaches don't explicitly deal with uncertainty in cloud-based software, where noise and unexpected events are common. The authors propose a control-theoretic approach that manages the behavior of a cloud environment as a dynamic system. They integrate a fuzzy cloud controller with an online learning mechanism, putting forward a framework that takes the human out of the dynamic adaptation loop and can cope with various sources of uncertainty in the cloud.
---
paper_title: An adaptive hybrid elasticity controller for cloud infrastructures

paper_content:

Cloud elasticity is the ability of the cloud infrastructure to rapidly change the amount of resources allocated to a service in order to meet the actual varying demands on the service while enforcing SLAs. In this paper, we focus on horizontal elasticity, the ability of the infrastructure to add or remove virtual machines allocated to a service deployed in the cloud. We model a cloud service using queuing theory. Using that model we build two adaptive proactive controllers that estimate the future load on a service. We explore the different possible scenarios for deploying a proactive elasticity controller coupled with a reactive elasticity controller in the cloud. Using simulation with workload traces from the FIFA world-cup web servers, we show that a hybrid controller that incorporates a reactive controller for scale up coupled with our proactive controllers for scale down decisions reduces SLA violations by a factor of 2 to 10 compared to a regression based controller or a completely reactive controller.
---
paper_title: Automated control in cloud computing: challenges and opportunities

paper_content:

With advances in virtualization technology, virtual machine services offered by cloud utility providers are becoming increasingly powerful, anchoring the ecosystem of cloud services. Virtual computing services are attractive in part because they enable customers to acquire and release computing resources for guest applications adaptively in response to load surges and other dynamic behaviors. ``Elastic'' cloud computing APIs present a natural opportunity for feedback controllers to automate this adaptive resource provisioning, and many recent works have explored feedback control policies for a variety of network services under various assumptions. This paper addresses the challenge of building an effective controller as a customer add-on outside of the cloud utility service itself. Such external controllers must function within the constraints of the utility service APIs. It is important to consider techniques for effective feedback control using cloud APIs, as well as how to design those APIs to enable more effective control. As one example, we explore proportional thresholding, a policy enhancement for feedback controllers that enables stable control across a wide range of guest cluster sizes using the coarse-grained control offered by popular virtual compute cloud services.
---
paper_title: Automated control for elastic storage

paper_content:

Elasticity - where systems acquire and release resources in response to dynamic workloads, while paying only for what they need - is a driving property of cloud computing. At the core of any elastic system is an automated controller. This paper addresses elastic control for multi-tier application services that allocate and release resources in discrete units, such as virtual server instances of predetermined sizes. It focuses on elastic control of the storage tier, in which adding or removing a storage node or "brick" requires rebalancing stored data across the nodes. The storage tier presents new challenges for elastic control: actuator delays (lag) due to rebalancing, interference with applications and sensor measurements, and the need to synchronize the multiple control elements, including rebalancing. We have designed and implemented a new controller for elastic storage systems to address these challenges. Using a popular distributed storage system - the Hadoop Distributed File System (HDFS) - under dynamic Web 2.0 workloads, we show how the controller adapts to workload changes to maintain performance objectives efficiently in a pay-as-you-go cloud computing environment.
---
paper_title: Efficient server provisioning with end-to-end delay guarantee on multi-tier clusters

paper_content:

Dynamic server provisioning is critical to quality-of-service assurance for multi-tier Internet applications. In this paper, we address three important and challenging problems. First, we propose an efficient server provisioning approach on multi-tier clusters based on an end-to-end resource allocation optimization model. It is to minimize the number of servers allocated to the system while the average end-to-end delay guarantee is satisfied. Second, we design a model-independent fuzzy controller for bounding an important performance metric, the 90 th -percentile delay of requests flowing through the multi-tier architecture. Third, to compensate for the latency due to the dynamic addition of servers, we design a self-tuning component that adaptively adjusts the output scaling factor of the fuzzy controller according to the transient behavior of the end-to-end delay. Extensive simulation results, using one representative customer behavior model in a typical three-tier web cluster, demonstrate that the provisioning approach is able to significantly reduce the server utilization compared to an existing representative approach. The approach integrated with the model-independent self-tuning fuzzy controller can efficiently assure the average and the 90 th -percentile end-to-end delay guarantees on multi-tier server clusters.
---
paper_title: Automated control of multiple virtualized resources

paper_content:

Virtualized data centers enable sharing of resources among hosted applications. However, it is difficult to satisfy service-level objectives(SLOs) of applications on shared infrastructure, as application workloads and resource consumption patterns change over time. In this paper, we present AutoControl, a resource control system that automatically adapts to dynamic workload changes to achieve application SLOs. AutoControl is a combination of an online model estimator and a novel multi-input, multi-output (MIMO) resource controller. The model estimator captures the complex relationship between application performance and resource allocations, while the MIMO controller allocates the right amount of multiple virtualized resources to achieve application SLOs. Our experimental evaluation with RUBiS and TPC-W benchmarks along with production-trace-driven workloads indicates that AutoControl can detect and mitigate CPU and disk I/O bottlenecks that occur over time and across multiple nodes by allocating each resource accordingly. We also show that AutoControl can be used to provide service differentiation according to the application priorities during resource contention.
---
paper_title: Efficient provisioning of bursty scientific workloads on the cloud using adaptive elasticity control

paper_content:

Elasticity is the ability of a cloud infrastructure to dynamically change the amount of resources allocated to a running service as load changes. We build an autonomous elasticity controller that changes the number of virtual machines allocated to a service based on both monitored load changes and predictions of future load. The cloud infrastructure is modeled as a G/G/N queue. This model is used to construct a hybrid reactive-adaptive controller that quickly reacts to sudden load changes, prevents premature release of resources, takes into account the heterogeneity of the workload, and avoids oscillations. Using simulations with Web and cluster workload traces, we show that our proposed controller lowers the number of delayed requests by a factor of 70 for the Web traces and 3 for the cluster traces when compared to a reactive controller. Our controller also decreases the average number of queued requests by a factor of 3 for both traces, and reduces oscillations by a factor of 7 for the Web traces and 3 for the cluster traces. This comes at the expense of between 20% and 30% over-provisioning, as compared to a few percent for the reactive controller.
---
paper_title: Exploiting User Patience for Scaling Resource Capacity in Cloud Services

paper_content:

An important feature of cloud computing is its elasticity, that is, the ability to have resource capacity dynamically modified according to the current system load. Auto-scaling is challenging because it must account for two conflicting objectives: minimising system capacity available to users and maximising QoS, which typically translates to short response times. Current auto-scaling techniques are based solely on load forecasts and ignore the perception that users have from cloud services. As a consequence, providers tend to provision a volume of resources that is significantly larger than necessary to keep users satisfied. In this article, we propose a scheduling algorithm and an auto-scaling triggering technique that explore user patience in order to identify critical times when auto-scaling is needed and the appropriate volume of capacity by which the cloud platform should either extend or shrink. The proposed technique assists service providers in reducing costs related to resource allocation while keeping the same QoS to users. Our experiments show that it is possible to reduce resource-hour by up to approximately 8% compared to auto-scaling based on system utilisation.
---
paper_title: Evaluating Auto-scaling Strategies for Cloud Computing Environments

paper_content:

Auto-scaling is a key feature in clouds responsible for adjusting the number of available resources to meet service demand. Resource pool modifications are necessary to keep performance indicators, such as utilisation level, between user-defined lower and upper bounds. Auto-scaling strategies that are not properly configured according to user workload characteristics may lead to unacceptable QoS and large resource waste. As a consequence, there is a need for a deeper understanding of auto-scaling strategies and how they should be configured to minimise these problems. In this work, we evaluate various auto-scaling strategies using log traces from a production Google data centre cluster comprising millions of jobs. Using utilisation level as performance indicator, our results show that proper management of auto-scaling parameters reduces the difference between the target utilisation interval and the actual values -- we define such difference as Auto-scaling Demand Index. We also present a set of lessons from this study to help cloud providers build recommender systems for auto-scaling operations.
---
paper_title: GEAM : a general and event-related aspects model for Twitter event detection

paper_content:

Event detection on Twitter has become a promising research direction due to Twitter’s popularity, up-to-date feature, free writing style and so on. Unfortunately, it’s a challenge to analyze Twitter dataset for event detection, since the informal expressions of short messages comprise many abbreviations, Internet buzzwords, spelling mistakes, meaningless contents etc. Previous techniques proposed for Twitter event detection mainly focus on clustering bursty words related to the events, while ignoring that these words may not be easily interpreted to clear event descriptions. In this paper, we propose a General and Event-related Aspects Model (GEAM), a new topic model for event detection from Twitter that associates General topics and Event-related Aspects with events. We then introduce a collapsed Gibbs sampling algorithm to estimate the word distributions of General topics and Event-related Aspects in GEAM. Our experiments based on over 7 million tweets demonstrate that GEAM outperforms the state-of-the-art topic model in terms of both Precision and DERate (measuring Duplicated Events Rate detected). Particularly, GEAM can get better event representation by providing a 4-tuple (Time, Locations, Entities, Keywords) structure of the detected events. We show that GEAM not only can be used to effectively detect events but also can be used to analyze event trends.
---
paper_title: Cloud QoS Scaling by Fuzzy Logic

paper_content:

One of the biggest advantages of cloud infrastructures is the elasticity. Cloud services are monitored and based on the resource utilization and performance load, they get scaled up or down, by provision or de-provision of cloud resources. The goal is to guarantee the customers an acceptable performance with a minimum of resources. Such Quality of Service (QoS) characteristics are stated in a contract, called Service Level Agreement (SLA) negotiated between customer and provider. The approach of this paper shows that with additional imprecise information (e.g. expected daytime/week- time performance) modeled with fuzzy logic and used in a behavior, load and performance prediction model, the up and down scaling mechanism of a cloud service can be optimized. Evaluation results confirm, that using this approach, SLA violation can be minimized.
---
paper_title: Replica Placement in Cloud through Simple Stochastic Model Predictive Control

paper_content:

This paper presents a model and an algorithm for optimal service placement (OSP) of a set of N-tier software systems, subject to dynamic changes in the workload, Service Level Agreements (SLA), and administrator preferences. The objective function models the resources' cost, the service level agreements and the trashing cost. The optimization algorithm is predictive: its allocation or reallocation decisions are based not only on the current metrics but also on predicted evolution of the system. The solution of the optimization, in each step, is a set some service replicas to be added or removed from the available hosts. These deployment changes are optimal with regards to overall objectives defined over time. In addition, the optimization considers the restrictions imposed on the number of possible service migrations at each time interval. We present experimental results that show the effectiveness of our approach.
---
paper_title: Dynamic Service Placement in Geographically Distributed Clouds

paper_content:

Large-scale online service providers have been increasingly relying on geographically distributed cloud infrastructures for service hosting and delivery. In this context, a key challenge faced by service providers is to determine the locations where service applications should be placed such that the hosting cost is minimized while key performance requirements (e.g., response time) are ensured. Furthermore, the dynamic nature of both demand pattern and infrastructure cost favors a dynamic solution to this problem. Currently most of the existing solutions for service placement have either ignored dynamics, or provided solutions inadequate to achieve this objective. In this paper, we present a framework for dynamic service placement problems based on control- and game-theoretic models. In particular, we present a solution that optimizes the hosting cost dynamically over time according to both demand and resource price fluctuations. We further consider the case where multiple service providers compete for resources in a dynamic manner. This paper extends our previous work [1] by analyzing the outcome of the competition in terms of both price of stability and price of anarchy. Our analysis suggests that in an uncoordinated scenario where service providers behave in a selfish manner, the resulting Nash equilibrium can be arbitrarily worse than the optimal centralized solution in terms of social welfare. Based on this observation, we present a coordination mechanism that can be employed by the infrastructure provider to maximize the social welfare of the system. Finally, we demonstrate the effectiveness of our solutions using realistic simulations.
---
paper_title: Optimal autoscaling in a IaaS cloud

paper_content:

An application provider leases resources (i.e., virtual machine instances) of variable configurations from a IaaS provider over some lease duration (typically one hour). The application provider (i.e., consumer) would like to minimize their cost while meeting all service level obligations (SLOs). The mechanism of adding and removing resources at runtime is referred to as autoscaling. The process of autoscaling is automated through the use of a management component referred to as an autoscaler. This paper introduces a novel autoscaling approach in which both cloud and application dynamics are modeled in the context of a stochastic, model predictive control problem. The approach exploits trade-off between satisfying performance related objectives for the consumer's application while minimizing their cost. Simulation results are presented demonstrating the efficacy of this new approach.
---
paper_title: Energy-Aware Server Provisioning and Load Dispatching for Connection-Intensive Internet Services

paper_content:

Energy consumption in hosting Internet services is becoming a pressing issue as these services scale up. Dynamic server provisioning techniques are effective in turning off unnecessary servers to save energy. Such techniques, mostly studied for request-response services, face challenges in the context of connection servers that host a large number of long-lived TCP connections. In this paper, we characterize unique properties, performance, and power models of connection servers, based on a real data trace collected from the deployed Windows Live Messenger. Using the models, we design server provisioning and load dispatching algorithms and study subtle interactions between them. We show that our algorithms can save a significant amount of energy without sacrificing user experiences.
---
paper_title: AGILE: elastic distributed resource scaling for Infrastructure-as-a-Service

paper_content:

Dynamically adjusting the number of virtual machines (VMs) assigned to a cloud application to keep up with load changes and interference from other uses typically requires detailed application knowledge and an ability to know the future, neither of which are readily available to infrastructure service providers or application owners. The result is that systems need to be over-provisioned (costly), or risk missing their performance Service Level Objectives (SLOs) and have to pay penalties (also costly). AGILE deals with both issues: it uses wavelets to provide a medium-term resource demand prediction with enough lead time to start up new application server instances before performance falls short, and it uses dynamic VM cloning to reduce application startup times. Tests using RUBiS and Google cluster traces show that AGILE can predict varying resource demands over the medium-term with up to 3.42× better true positive rate and 0.34× the false positive rate than existing schemes. Given a target SLO violation rate, AGILE can efficiently handle dynamic application workloads, reducing both penalties and user dissatisfaction.
---
paper_title: Agile dynamic provisioning of multi-tier Internet applications

paper_content:

Dynamic capacity provisioning is a useful technique for handling the multi-time-scale variations seen in Internet workloads. In this article, we propose a novel dynamic provisioning technique for multi-tier Internet applications that employs (1) a flexible queuing model to determine how much of the resources to allocate to each tier of the application, and (2) a combination of predictive and reactive methods that determine when to provision these resources, both at large and small time scales. We propose a novel data center architecture based on virtual machine monitors to reduce provisioning overheads. Our experiments on a forty-machine Xen/Linux-based hosting platform demonstrate the responsiveness of our technique in handling dynamic workloads. In one scenario where a flash crowd caused the workload of a three-tier application to double, our technique was able to double the application capacity within five minutes, thus maintaining response-time targets. Our technique also reduced the overhead of switching servers across applications from several minutes to less than a second, while meeting the performance targets of residual sessions.
---
paper_title: Efficient Autoscaling in the Cloud Using Predictive Models for Workload Forecasting

paper_content:

Large-scale component-based enterprise applications that leverage Cloud resources expect Quality of Service(QoS) guarantees in accordance with service level agreements between the customer and service providers. In the context of Cloud computing, auto scaling mechanisms hold the promise of assuring QoS properties to the applications while simultaneously making efficient use of resources and keeping operational costs low for the service providers. Despite the perceived advantages of auto scaling, realizing the full potential of auto scaling is hard due to multiple challenges stemming from the need to precisely estimate resource usage in the face of significant variability in client workload patterns. This paper makes three contributions to overcome the general lack of effective techniques for workload forecasting and optimal resource allocation. First, it discusses the challenges involved in auto scaling in the cloud. Second, it develops a model-predictive algorithm for workload forecasting that is used for resource auto scaling. Finally, empirical results are provided that demonstrate that resources can be allocated and deal located by our algorithm in a way that satisfies both the application QoS while keeping operational costs low.
---
paper_title: Towards an Autonomic Auto-scaling Prediction System for Cloud Resource Provisioning

paper_content:

This paper investigates the accuracy of predictive auto-scaling systems in the Infrastructure as a Service (IaaS) layer of cloud computing. The hypothesis in this research is that prediction accuracy of auto-scaling systems can be increased by choosing an appropriate time-series prediction algorithm based on the performance pattern over time. To prove this hypothesis, an experiment has been conducted to compare the accuracy of time-series prediction algorithms for different performance patterns. In the experiment, workload was considered as the performance metric, and Support Vector Machine (SVM) and Neural Networks (NN) were utilized as time-series prediction techniques. In addition, we used Amazon EC2 as the experimental infrastructure and TPC-W as the benchmark to generate different workload patterns. The results of the experiment show that prediction accuracy of SVM and NN depends on the incoming workload pattern of the system under study. Specifically, the results show that SVM has better prediction accuracy in the environments with periodic and growing workload patterns, while NN outperforms SVM in forecasting unpredicted workload pattern. Based on these experimental results, this paper proposes an architecture for a self-adaptive prediction suite using an autonomic system approach. This suite can choose the most suitable prediction technique based on the performance pattern, which leads to more accurate prediction results.
---
paper_title: VScaler: Autonomic Virtual Machine Scaling

paper_content:

Recent research results in cloud community found that cloud users increasingly force providers to shift from fixed bundle instance types(e.g. Amazon instances) to flexible bundles and shrinked billing cycles. This means that cloud applications can dynamically provision the used amount of resources in a more fine-grained fashion. This observation calls for approaches which are able to automatically implement fine granular VM resource allocation with respect to user-provided SLAs. In this work we propose VScaler, a framework which implements autonomic resource allocation using a novel approach to reinforcement learning.
---
paper_title: A cost-aware auto-scaling approach using the workload prediction in service clouds

paper_content:

Service clouds are distributed infrastructures which deploys communication services in clouds. The scalability is an important characteristic of service clouds. With the scalability, the service cloud can offer on-demand computing power and storage capacities to different services. In order to achieve the scalability, we need to know when and how to scale virtual resources assigned to different services. In this paper, a novel service cloud architecture is presented, and a linear regression model is used to predict the workload. Based on this predicted workload, an auto-scaling mechanism is proposed to scale virtual resources at different resource levels in service clouds. The auto-scaling mechanism combines the real-time scaling and the pre-scaling. Finally experimental results are provided to demonstrate that our approach can satisfy the user Service Level Agreement (SLA) while keeping scaling costs low.
---
paper_title: Prediction-based real-time resource provisioning for massively multiplayer online games

paper_content:

Massively Multiplayer Online Games (MMOGs) are a class of computationally intensive client-server applications with severe real-time Quality of Service (QoS) requirements, such as the number of updates per second each client needs to receive from the servers for a fluent and realistic experience. To guarantee the QoS requirements, game providers currently over-provision a large amount of their resources, which makes the overall efficiency of provisioning and utilization of resources rather low and prohibits any but the largest providers from joining the market. To address this deficiency, we propose a new prediction-based method for dynamic resource provisioning and scaling of MMOGs in distributed Grid environments. Firstly, a load prediction service anticipates the future game world entity distribution from historical trace data using a fast and flexible neural network-based method. On top of it, we developed generic analytical game load models used to foresee future hot-spots that congest the game servers and make the overall environment fragmented and unplayable. Finally, a resource allocation service performs dynamic load distribution, balancing, and migration of entities that keep the game servers reasonably loaded such that the real-time QoS requirements are maintained. Experimental results based on a realistic simulation environment demonstrate the advantages of our prediction service compared to other conventional methods, especially due to its ability to adapt to different user load patterns, and a reduction of the average over-allocation from 250% (in the case of static over-provisioning) to around 25% using our dynamic provisioning method.
---
paper_title: Optimal Cloud Resource Auto-Scaling for Web Applications

paper_content:

In the on-demand cloud environment, web application providers have the potential to scale virtual resources up or down to achieve cost-effective outcomes. True elasticity and cost-effectiveness in the pay-per-use cloud business model, however, have not yet been achieved. To address this challenge, we propose a novel cloud resource auto-scaling scheme at the virtual machine (VM) level for web application providers. The scheme automatically predicts the number of web requests and discovers an optimal cloud resource demand with cost-latency trade-off. Based on this demand, the scheme makes a resource scaling decision that is up or down or NOP (no operation) in each time-unit re-allocation. We have implemented the scheme on the Amazon cloud platform and evaluated it using three real-world web log datasets. Our experiment results demonstrate that the proposed scheme achieves resource auto-scaling with an optimal cost-latency trade-off, as well as low SLA violations.
---
paper_title: RPPS: A Novel Resource Prediction and Provisioning Scheme in Cloud Data Center

paper_content:

Cloud data centers and virtualization are being highly considered for enterprises and industries. However, elastic fine-grained resource provision while ensuring performance and SLA guarantees for applications requires careful consideration of important and extremely challenging tradeoffs. In this paper, we present RPPS (Cloud Resource Prediction and Provisioning scheme), a scheme that automatically predict future demand and perform proactive resource provisioning for cloud applications. RPPS employs the ARIMA model to predict the workloads in the future, combines both coarse-grained and fine-grained resource scaling under different situations, and adopts a VM-complementary migration strategy. RPPS can resolve predictive resource provisioning problem when enterprises confront demand fluctuations in cloud data center. We evaluate a prototype of RPPS with traces collected by ourselves using typical CPU intensive applications and as well as workloads from a real data center. The results show that it not only has high prediction accuracy (about 90% match in most time) but also scales the resource well.
---
paper_title: An autonomic resource provisioning approach for service-based cloud applications: A hybrid approach

paper_content:

Abstract In cloud computing environment, resources can be dynamically provisioned on deman for cloud services The amount of the resources to be provisioned is determined during runtime according to the workload changes. Deciding the right amount of resources required to run the cloud services is not trivial, and it depends on the current workload of the cloud services. Therefore, it is necessary to predict the future demands to automatically provision resources in order to deal with fluctuating demands of the cloud services. In this paper, we propose a hybrid resource provisioning approach for cloud services that is based on a combination of the concept of the autonomic computing and the reinforcement learning (RL). Also, we present a framework for autonomic resource provisioning which is inspired by the cloud layer model. Finally, we evaluate the effectiveness of our approach under two real world workload traces. The experimental results show that the proposed approach reduces the total cost by up to 50%, and increases the resource utilization by up to 12% compared with the other approaches.
---
paper_title: Self-adaptive workload classification and forecasting for proactive resource provisioning

paper_content:

As modern enterprise software systems become increasingly dynamic, workload forecasting techniques are gaining an importance as a foundation for online capacity planning and resource management. Time series analysis offers a broad spectrum of methods to calculate workload forecasts based on history monitoring data. Related work in the field of workload forecasting mostly concentrates on evaluating specific methods and their individual optimisation potential or on predicting QoS metrics directly. As a basis, we present a survey on established forecasting methods of the time series analysis concerning their benefits and drawbacks and group them according to their computational overheads. In this paper, we propose a novel self-adaptive approach that selects suitable forecasting methods for a given context based on a decision tree and direct feedback cycles together with a corresponding implementation. The user needs to provide only his general forecasting objectives. In several experiments and case studies based on real-world workload traces, we show that our implementation of the approach provides continuous and reliable forecast results at run-time. The results of this extensive evaluation show that the relative error of the individual forecast points is significantly reduced compared with statically applied forecasting methods, for example, in an exemplary scenario on average by 37%. In a case study, between 55 and 75% of the violations of a given service level objective can be prevented by applying proactive resource provisioning based on the forecast results of our implementation. Copyright © 2014 John Wiley & Sons, Ltd.
---
paper_title: An Architecture for Automatic Scaling of Replicated Services

paper_content:

Replicated services that allow to scale dynamically can adapt to requests load. Choosing the right number of replicas is fundamental to avoid performance worsening when input spikes occur and to save resources when the load is low. Current mechanisms for automatic scaling are mostly based on fixed thresholds on CPU and memory usage, which are not sufficiently accurate and often entail late countermeasures. We propose Make Your Service Elastic (MYSE), an architecture for automatic scaling of generic replicated services based on queuing models for accurate response time estimation. Requests and service times patterns are analyzed to learn and predict over time their distribution so as to allow for early scaling. A novel heuristic is proposed to avoid the flipping phenomenon. We carried out simulations that show promising results for what concerns the effectiveness of our approach.
---
paper_title: Providing IaaS resources automatically through prediction and monitoring approaches

paper_content:

A cloud computing infrastructure management is proposed in this paper, which consists of two approaches that facilitate the provisioning of computing resources in a self-adaptive virtualized environment. Resource allocation is employed to predict the future of workload management and to employ a self-adaptive approach by using computational agents to monitor the Virtual Machines (VMs). The paper also includes the Return on Investment (ROI) formula that deals with the relationship between the prices for the Infrastructure-as-a-Service (IaaS) contracted by the customer and the effective use of this service. The experimental results show a significant improvement when self-configuration is used with agent-based computational modeling in contrast with the self-configuration based on prediction for future workload.
---
paper_title: Vadara: Predictive Elasticity for Cloud Applications

paper_content:

Elasticity is a key feature in cloud computing, and perhaps what distinguishes it from other computing paradigms. Despite the advantages of elasticity, realizing its full potential is hard due to multiple challenges stemming from the need to estimate workload demand. A desirable solution would require predicting system workload and allocating resources a priori, i.e., A predictive approach. Instead, what is mainly available are reactive solutions, requiring difficult parameter tuning. Since each Cloud Provider (CP) has its own implementation idiosyncrasies, it's impossible for developers to: (i) learn only about one platform and re-use that knowledge in others, (ii) migrate developed elasticity solutions between different CPs, and (iii) to develop reusable predictive elasticity rules or algorithms. This paper makes three contributions to provide an effective elasticity environment. First, Vadara, a totally generic elasticity framework, that transparently connects and abstracts several CPs API behaviour, and enables the use of pluggable CP-agnostic elasticity strategies. Second, it presents a predictive workload forecasting approach, which ensembles several individual forecasting methods, and introduces a padding system based on the most recent prediction errors for both under- and over-provisioning. Finally, results show (1) Vadara's successful connection to well-known CPs, (2) the improvements made regarding under- and over-provisioning due to our padding system, and (3) the effectiveness of our ensemble forecasting technique.
---
paper_title: An Empirical Study into Adaptive Resource Provisioning in the Cloud

paper_content:

Cloud computing allows dynamic resource scaling for enterprise online transaction systems, one of the key characteristics that differentiates cloud from the traditional computing paradigm. However, initializing a new virtual instance in cloud is not instantaneous; the cloud hosting platforms introduce significant delay in the hardware resource allocation. In this paper, we develop prediction-based resource measurement and provisioning strategies using Neural Network and Linear Regression to satisfy upcoming resource demands. Experimental results demonstrate that the proposed technique offers more adaptive resource management for applications hosted in cloud environment, an important mechanism to achieve on-demand resource allocation in the cloud.
---
paper_title: PRESS: PRedictive Elastic ReSource Scaling for cloud systems

paper_content:

Cloud systems require elastic resource allocation to minimize resource provisioning costs while meeting service level objectives (SLOs). In this paper, we present a novel PRedictive Elastic reSource Scaling (PRESS) scheme for cloud systems. PRESS unobtrusively extracts fine-grained dynamic patterns in application resource demands and adjust their resource allocations automatically. Our approach leverages light-weight signal processing and statistical learning algorithms to achieve online predictions of dynamic application resource requirements. We have implemented the PRESS system on Xen and tested it using RUBiS and an application load trace from Google. Our experiments show that we can achieve good resource prediction accuracy with less than 5% over-estimation error and near zero under-estimation error, and elastic resource scaling can both significantly reduce resource waste and SLO violations.
---
paper_title: Vertical Scaling for Prioritized VMs Provisioning

paper_content:

The elastic scaling of services permits us (1) to meet service provisioning requirements (i.e., not to violate its SLAs), and (2) to efficiently use available resources (i.e., to minimize the cost of running the application). In this paper, we describe a novel online controller design built on top of the Xen hyper visor. The controller facilitates the elastic provisioning of applications using a combination of adapting the resource constraints of VMs and dynamically plugging in new virtual CPUs. We have implemented our design and evaluated our approach using real world workload traces from the RUBiS online auction benchmark. The results show that our controller allows to reduce total CPU time in comparison to statically allocated CPU with of minimum SLA violation rate and provides stable response time for high priority VM, when we apply it to provision prioritized VMs.
---
paper_title: Pattern Matching Based Forecast of Non-periodic Repetitive Behavior for Cloud Clients

paper_content:

The Cloud phenomenon brings along the cost-saving benefit of dynamic scaling. As a result, the question of efficient resource scaling arises. Prediction is necessary as the virtual resources that Cloud computing uses have a setup time that is not negligible. We propose an approach to the problem of workload prediction based on identifying similar past occurrences of the current short-term workload history. We present in detail the Cloud client resource auto-scaling algorithm that uses the above approach to help when scaling decisions are made, as well as experimental results by using real-world Cloud client application traces. We also present an overall evaluation of this approach, its potential and usefulness for enabling efficient auto-scaling of Cloud user resources.
---
paper_title: Autoflex: Service Agnostic Auto-scaling Framework for IaaS Deployment Models

paper_content:

Elasticity is a key property to reduce the costs associated with running services in cloud systems that employ an infrastructure-as-a-service (IaaS) deployment model. However, to be able to exploit this property, users of IaaS systems need to be able to anticipate the short-term future demand of their own services, so that only the required infrastructure is requested at any instant in time. This guarantees that service level objectives (SLOs) are always honored by the users of IaaS systems, while over provisioning is avoided. The process of automatically change the amount of resources used to run a service in an IaaS system is named auto-scaling, and the state-of-the-practice uses simple reactive approaches. Although these approaches can successfully reduce the costs due to over provisioning, they are frequently insufficient to minimize the costs due to SLO violations. For that, proactive approaches are required. In this paper we propose a framework for the implementation of auto-scaling services that follows both reactive and proactive approaches. The latter is based on the use of a set of predictors of the future demand of services deployed over IaaS resources, and a selection mechanism that chooses, over time, what is the best predictor to be used. We have also proposed a correction method that uses historical data on the predictors' errors to reduce the probability of under provisioning the services, thus further diminishing the number of SLO violations. We have evaluated the performance of the proposed approach using production traces of HP customers. Our results show that costs savings of as much as 37% can be achieved, while the probability of an SLO violation can be kept, on average, as small as 0.008%, and never larger than 0.036%.
---
paper_title: Resource provisioning with budget constraints for adaptive applications in cloud environments

paper_content:

Utility computing was a vision stated more than 40 years ago. It refers to the idea that computing resources and services can be delivered, utilized, and paid for as utilities such as water or electricity. The recent emergence of cloud computing is making this vision realizable. Examples of efforts in this area include Infrastructure as a Service (IaaS) providers like Amazon EC2 and Software as a Service (SaaS) providers like Google AppEngine and Microsoft Azure. More recently, there is a growing interest in the use of cloud computing for scientific computing, as evidenced by a large-scale funded project like Magellan, and new workshops.
---
paper_title: Self-adaptive and self-configured CPU resource provisioning for virtualized servers using Kalman filters

paper_content:

Data center virtualization allows cost-effective server consolidation which can increase system throughput and reduce power consumption. Resource management of virtualized servers is an important and challenging task, especially when dealing with fluctuating workloads and complex multi-tier server applications. Recent results in control theory-based resource management have shown the potential benefits of adjusting allocations to match changing workloads. This paper presents a new resource management scheme that integrates the Kalman filter into feedback controllers to dynamically allocate CPU resources to virtual machines hosting server applications. The novelty of our approach is the use of the Kalman filter-the optimal filtering technique for state estimation in the sum of squares sense-to track the CPU utilizations and update the allocations accordingly. Our basic controllers continuously detect and self-adapt to unforeseen workload intensity changes. Our more advanced controller self-configures itself to any workload condition without any a priori information. Indicatively, it results in within 4.8% of the performance of workload-aware controllers under high intensity workload changes, and performs equally well under medium intensity traffic. In addition, our controllers are enhanced to deal with multi-tier server applications: by using the pair-wise resource coupling between application components, they provide a 3% on average server performance improvement when facing large unexpected workload increases when compared to controllers with no such resource-coupling mechanism. We evaluate our techniques by controlling a 3-tier Rubis benchmark web site deployed on a prototype Xen-virtualized cluster.
---
paper_title: On the Use of Fuzzy Modeling in Virtualized Data Center Management

paper_content:

One of the most important goals of data-center management is to reduce cost through efficient use of resources. Virtualization techniques provide the opportunity of carving individual physical servers into multiple virtual containers that can be run and managed separately. A key challenge that comes with virtualization is the simultaneous on-demand provisioning of shared resources to virtual containers and the management of their capacities to meet service quality targets at the least cost. This paper proposes a two-level resource management system with local controllers at the virtual-container level and a global controller at the resource-pool level. Autonomic resource allocation is realized through the interaction of the local and global controllers. A novelty of the controller designs is their use of fuzzy logic to efficiently and robustly deal with the complexity of the virtualized data center and the uncertainties of the dynamically changing workloads. Experimental results obtained through a prototype implementation demonstrate that, for the scenarios under consideration, the proposed resource management system can significantly reduce resource consumption while still achieving application performance targets.
---
paper_title: Fine-Grained Resource Scaling in a Public Cloud: A Tenant's Perspective

paper_content:

Growing tenant workload needs and an increasingly competitive market will force cloud providers to operate their data centers at significantly higher utilization levels than seen today. We argue that a key enabler of such cloud ecosystems would be facilities for tenants to engage in fine-grained resource scaling in addition to those offered by current providers. The basic unit of resource scaling exposed by current cloud providers is the canonical interface of virtual machines (VMs) with relatively static resource capacities. This paper describes opportunities and challenges in augmenting this interface to also include fine-grained scaling of CPU and memory within an already procured VM. Qualitative arguments for why this would offer cost benefits for both the provider and its tenants are presented. We focus on the cost-effective operation of a tenant in such an environment via the design of a feedback controller. The efficacy of our ideas is illustrated by implementing a case study in a Memcached tenant workload. Our results are promising and point to an interesting and broad area for further research - e.g., with the real-world workload in our evaluation, up to 50% utility improvement can be achieved by just applying memory scaling, a further 66% improvement can be achieved by coordinating fine-grained CPU and memory scaling.
---
paper_title: Elastic Virtual Machine for Fine-Grained Cloud Resource Provisioning

paper_content:

Elasticity is one of the distinguishing characteristics associated with Cloud computing emergence. It enables cloud resources to auto-scale to cope with workload demand. Multi-instances horizontal scaling is the common scalability architecture in Cloud; however, its current implementation is coarse-grained, while it considers Virtual Machine (VM) as a scaling unit, this implies additional scaling-out overhead and limits it to specific applications. To overcome these limitations, we propose Elastic VM as a fine-grained vertical scaling architecture. Our results proved that Elastic VM architecture implies less consumption of resources, mitigates Service Level Objectives (SLOs) violation, and avoids scaling-up overhead. Furthermore, it scales broader range of applications including databases.
---
paper_title: PRESS: PRedictive Elastic ReSource Scaling for cloud systems

paper_content:

Cloud systems require elastic resource allocation to minimize resource provisioning costs while meeting service level objectives (SLOs). In this paper, we present a novel PRedictive Elastic reSource Scaling (PRESS) scheme for cloud systems. PRESS unobtrusively extracts fine-grained dynamic patterns in application resource demands and adjust their resource allocations automatically. Our approach leverages light-weight signal processing and statistical learning algorithms to achieve online predictions of dynamic application resource requirements. We have implemented the PRESS system on Xen and tested it using RUBiS and an application load trace from Google. Our experiments show that we can achieve good resource prediction accuracy with less than 5% over-estimation error and near zero under-estimation error, and elastic resource scaling can both significantly reduce resource waste and SLO violations.
---
paper_title: VScaler: Autonomic Virtual Machine Scaling

paper_content:

Recent research results in cloud community found that cloud users increasingly force providers to shift from fixed bundle instance types(e.g. Amazon instances) to flexible bundles and shrinked billing cycles. This means that cloud applications can dynamically provision the used amount of resources in a more fine-grained fashion. This observation calls for approaches which are able to automatically implement fine granular VM resource allocation with respect to user-provided SLAs. In this work we propose VScaler, a framework which implements autonomic resource allocation using a novel approach to reinforcement learning.
---
paper_title: Runtime Vertical Scaling of Virtualized Applications via Online Model Estimation

paper_content:

Applications in virtualized data centers are often subject to Service Level Objectives (SLOs) regarding their performance (e.g., latency or throughput). In order to fulfill these SLOs, it is necessary to allocate sufficient resources of different types (CPU, memory, I/O, etc.) to an application. However, the relationship between the application performance and the resource allocation is complex and depends on multiple factors including application architecture, system configuration, and workload demands. In this paper, we present a model-based approach to ensure that the application performance meets the user-defined SLO efficiently by runtime "vertical scaling" (i.e., adding or removing resources) of individual virtual machines (VMs) running the application. A layered performance model describing the relationship between the resource allocation and the observed application performance is automatically extracted and updated online using resource demand estimation techniques. Such a model is then used in a feedback controller to dynamically adapt the number of virtual CPUs of individual VMs. We have implemented the controller on top of the VMware vSphere platform and evaluated it in a case study using a real-world email and groupware server. The experimental results show that our approach allows the managed application to achieve SLO satisfaction in spite of workload demand variation while avoiding oscillations commonly observed with state-of-the-art threshold-based controllers.
---
paper_title: Vertical Scaling for Prioritized VMs Provisioning

paper_content:

The elastic scaling of services permits us (1) to meet service provisioning requirements (i.e., not to violate its SLAs), and (2) to efficiently use available resources (i.e., to minimize the cost of running the application). In this paper, we describe a novel online controller design built on top of the Xen hyper visor. The controller facilitates the elastic provisioning of applications using a combination of adapting the resource constraints of VMs and dynamically plugging in new virtual CPUs. We have implemented our design and evaluated our approach using real world workload traces from the RUBiS online auction benchmark. The results show that our controller allows to reduce total CPU time in comparison to statically allocated CPU with of minimum SLA violation rate and provides stable response time for high priority VM, when we apply it to provision prioritized VMs.
---
paper_title: Self-Adaptive Trade-off Decision Making for Autoscaling Cloud-Based Services

paper_content:

Elasticity in the cloud is often achieved by on-demand autoscaling. In such context, the goal is to optimize the Quality of Service (QoS) and cost objectives for the cloud-based services. However, the difficulty lies in the facts that these objectives, e.g., throughput and cost, can be naturally conflicted; and the QoS of cloud-based services often interfere due to the shared infrastructure in cloud. Consequently, dynamic and effective trade-off decision making of autoscaling in the cloud is necessary, yet challenging. In particular, it is even harder to achieve well-compromised trade-offs, where the decision largely improves the majority of the objectives; while causing relatively small degradations to others. In this paper, we present a self-adaptive decision making approach for autoscaling in the cloud. It is capable to adaptively produce autoscaling decisions that lead to well-compromised trade-offs without heavy human intervention. We leverage on ant colony inspired multi-objective optimization for searching and optimizing the trade-offs decisions, the result is then filtered by compromise-dominance, a mechanism that extracts the decisions with balanced improvements in the trade-offs. We experimentally compare our approach to four state-of-the-arts autoscaling approaches: rule, heuristic, randomized and multi-objective genetic algorithm based solutions. The results reveal the effectiveness of our approach over the others, including better quality of trade-offs and significantly smaller violation of the requirements.
---
paper_title: CloudScale: elastic resource scaling for multi-tenant cloud systems

paper_content:

Elastic resource scaling lets cloud systems meet application service level objectives (SLOs) with minimum resource provisioning costs. In this paper, we present CloudScale, a system that automates fine-grained elastic resource scaling for multi-tenant cloud computing infrastructures. CloudScale employs online resource demand prediction and prediction error handling to achieve adaptive resource allocation without assuming any prior knowledge about the applications running inside the cloud. CloudScale can resolve scaling conflicts between applications using migration, and integrates dynamic CPU voltage/frequency scaling to achieve energy savings with minimal effect on application SLOs. We have implemented CloudScale on top of Xen and conducted extensive experiments using a set of CPU and memory intensive applications (RUBiS, Hadoop, IBM System S). The results show that CloudScale can achieve significantly higher SLO conformance than other alternatives with low resource and energy cost. CloudScale is non-intrusive and light-weight, and imposes negligible overhead (
---
paper_title: Coordinated Self-Configuration of Virtual Machines and Appliances Using a Model-Free Learning Approach

paper_content:

Cloud computing has a key requirement for resource configuration in a real-time manner. In such virtualized environments, both virtual machines (VMs) and hosted applications need to be configured on-the-fly to adapt to system dynamics. The interplay between the layers of VMs and applications further complicates the problem of cloud configuration. Independent tuning of each aspect may not lead to optimal system wide performance. In this paper, we propose a framework, namely CoTuner, for coordinated configuration of VMs and resident applications. At the heart of the framework is a model-free hybrid reinforcement learning (RL) approach, which combines the advantages of Simplex method and RL method and is further enhanced by the use of system knowledge guided exploration policies. Experimental results on Xen-based virtualized environments with TPC-W and TPC-C benchmarks demonstrate that CoTuner is able to drive a virtual server cluster into an optimal or near-optimal configuration state on the fly, in response to the change of workload. It improves the systems throughput by more than 30 percent over independent tuning strategies. In comparison with the coordinated tuning strategies based on basic RL or Simplex algorithm, the hybrid RL algorithm gains 25 to 40 percent throughput improvement.
---
paper_title: Automated control of multiple virtualized resources

paper_content:

Virtualized data centers enable sharing of resources among hosted applications. However, it is difficult to satisfy service-level objectives(SLOs) of applications on shared infrastructure, as application workloads and resource consumption patterns change over time. In this paper, we present AutoControl, a resource control system that automatically adapts to dynamic workload changes to achieve application SLOs. AutoControl is a combination of an online model estimator and a novel multi-input, multi-output (MIMO) resource controller. The model estimator captures the complex relationship between application performance and resource allocations, while the MIMO controller allocates the right amount of multiple virtualized resources to achieve application SLOs. Our experimental evaluation with RUBiS and TPC-W benchmarks along with production-trace-driven workloads indicates that AutoControl can detect and mitigate CPU and disk I/O bottlenecks that occur over time and across multiple nodes by allocating each resource accordingly. We also show that AutoControl can be used to provide service differentiation according to the application priorities during resource contention.
---
paper_title: A Cost-Aware Elasticity Provisioning System for the Cloud

paper_content:

In this paper we present Kingfisher, a {\em cost-aware} system that provides efficient support for elasticity in the cloud by (i) leveraging multiple mechanisms to reduce the time to transition to new configurations, and (ii) optimizing the selection of a virtual server configuration that minimizes the cost. We have implemented a prototype of Kingfisher and have evaluated its efficacy on a laboratory cloud platform. Our experiments with varying application workloads demonstrate that Kingfisher is able to (i) decrease the cost of virtual server resources by as much as $24\%$ compared to the current cost-unaware approach, (ii) reduce by an order of magnitude the time to transition to a new configuration through multiple elasticity mechanisms in the cloud, and (iii), illustrate the opportunity for design alternatives which trade-off the cost of server resources with the time required to scale the application.
---
paper_title: Optimal Resource Provisioning for Scaling Enterprise Applications on the Cloud

paper_content:

Over the past years organizations have been moving their enterprise applications to the cloud to take advantage of cloud's utility computing and elasticity. However, in enterprise applications or workflows, generally, different components/tasks will have different scaling requirements and finding an ideal deployment configuration and having the application to scale up and down based on the incoming requests is a difficult task. This paper presents a novel resource provisioning policy that can find the most cost optimal setup of variety of instances of cloud that can fulfill incoming workload. All major factors involved in resource amount estimation such as processing power, periodic cost and configuration cost of each instance type and capacity of clouds are considered in the model. Additionally, the model takes lifetime of each running instance into account while trying to find the optimal setup. Benchmark experiments were conducted on Amazon cloud, using a real load trace and through two main control flow components of enterprise applications, AND and XOR. In these experiments, our model could find the most cost-optimal setup for each component/task of the application within reasonable time, making it plausible for auto-scaling any web/services based enterprise workflow/application on the cloud.
---
paper_title: A Reliable and Cost-Efficient Auto-Scaling System for Web Applications Using Heterogeneous Spot Instances

paper_content:

Cloud providers sell their idle capacity on markets through an auction-like mechanism to increase their return on investment. The instances sold in this way are called spot instances. In spite that spot instances are usually 90% cheaper than on-demand instances, they can be terminated by provider when their bidding prices are lower than market prices. Thus, they are largely used to provision fault-tolerant applications only. In this paper, we explore how to utilize spot instances to provision web applications, which are usually considered as availability-critical. The idea is to take advantage of differences in price among various types of spot instances to reach both high availability and significant cost saving. We first propose a fault-tolerant model for web applications provisioned by spot instances. Based on that, we devise novel cost-efficient auto-scaling polices that comply with the defined fault-tolerant semantics for hourly billed cloud markets. We implemented the proposed model and policies both on a simulation testbed for repeatable validation and Amazon EC2. The experiments on the simulation testbed and EC2 show that the proposed approach can greatly reduce resource cost and still achieve satisfactory Quality of Service (QoS) in terms of response time and availability.
---
paper_title: Dynamic Selection of Virtual Machines for Application Servers in Cloud Environments

paper_content:

Autoscaling is a hallmark of cloud computing as it allows flexible just-in-time allocation and release of computational resources in response to dynamic and often unpredictable workloads. This is especially important for web applications, whose workload is time dependent and prone to flash crowds. Most of them follow the 3-tier architectural pattern, and are divided into presentation, application/domain and data layers. In this work, we focus on the application layer. Reactive autoscaling policies of the type “Instantiate a new Virtual Machine (VM) when the average server CPU utilisation reaches X%” have been used successfully since the dawn of cloud computing. But which VM type is the most suitable for the specific application at the moment remains an open question. In this work, we propose an approach for dynamic VM type selection. It uses a combination of online machine learning techniques, works in real time and adapts to changes in the users’ workload patterns, application changes as well as middleware upgrades and reconfigurations. We have developed a prototype, which we tested with the CloudStone benchmark deployed on AWS EC2. Results show that our method quickly adapts to workload changes and reduces the total cost compared to the industry standard approach.
---
paper_title: A virtual machine re-packing approach to the horizontal vs. vertical elasticity trade-off for cloud autoscaling

paper_content:

An automated solution to horizontal vs. vertical elasticity problem is central to make cloud autoscalers truly autonomous. Today's cloud autoscalers are typically varying the capacity allocated by increasing and decreasing the number of virtual machines (VMs) of a predefined size (horizontal elasticity), not taking into account that as load varies it may be advantageous not only to vary the number but also the size of VMs (vertical elasticity). We analyze the price/performance effects achieved by different strategies for selecting VM-sizes for handling increasing load and we propose a cost-benefit based approach to determine when to (partly) replace a current set of VMs with a different set. We evaluate our repacking approach in combination with different auto-scaling strategies. Our results show a range of 7% up to 60% cost saving in total resource utilization cost of our sample applications and workloads.
---
paper_title: Provisioning multi-tier cloud applications using statistical bounds on sojourn time

paper_content:

In this paper we present a simple and effective approach for resource provisioning to achieve a percentile bound on the end to end response time of a multi-tier application. We, at first, model the multi-tier application as an open tandem network of M/G/1-PS queues and develop a method that produces a near optimal application configuration, i.e, number of servers at each tier, to meet the percentile bound in a homogeneous server environment -- using a single type of server. We then extend our solution to a K-server case and our technique demonstrates a good accuracy, independent of the variability of service-times. Our approach demonstrates a provisioning error of no more than 3% compared to a 140% worst case provisioning error obtained by techniques based on an M/M/1-FCFS queue model. In addition, we extend our approach to handle a heterogenous server environment, i.e., with multiple types of servers. We find that fewer high-capacity servers are preferable for high percentile provisioning. Finally, we extend our approach to account for the rental cost of each server-type and compute a cost efficient application configuration with savings of over 80%. We demonstrate the applicability of our approach in a real world system by employing it to provision the two tiers of the java implementation of TPC-W -- a multi-tier transactional web benchmark that represents an e-commerce web application, i.e. an online bookstore.
---
paper_title: Autoscaling Web Applications in Heterogeneous Cloud Infrastructures

paper_content:

Improving resource provisioning of heterogeneous cloud infrastructures is an important research challenge. The wide diversity of cloud-based applications and customers with different QoS requirements have recently exhibited the weaknesses of current provisioning systems. Today's cloud infrastructures provide provisioning systems that dynamically adapt the computational power of applications by adding or releasing resources. Unfortunately, these scaling systems are fairly limited:(i) They restrict themselves to a single type of resource, (ii)they are unable to fulfill QoS requirements in face of spiky workload, and (iii) they offer the same QoS level to all their customers, independent of customer preferences such as different levels of service availability and performance. In this paper, we present an autoscaling system that overcomes these limitations by exploiting heterogeneous types of resources, and by defining multiple levels of QoS requirements. The proposed system selects a resource scaling plan according to both workload and customer requirements. Our experiments conducted on both public and private infrastructures show significant reductions in QoS-level violations when faced with highly variable workloads.
---
paper_title: A Reliable and Cost-Efficient Auto-Scaling System for Web Applications Using Heterogeneous Spot Instances

paper_content:

Cloud providers sell their idle capacity on markets through an auction-like mechanism to increase their return on investment. The instances sold in this way are called spot instances. In spite that spot instances are usually 90% cheaper than on-demand instances, they can be terminated by provider when their bidding prices are lower than market prices. Thus, they are largely used to provision fault-tolerant applications only. In this paper, we explore how to utilize spot instances to provision web applications, which are usually considered as availability-critical. The idea is to take advantage of differences in price among various types of spot instances to reach both high availability and significant cost saving. We first propose a fault-tolerant model for web applications provisioned by spot instances. Based on that, we devise novel cost-efficient auto-scaling polices that comply with the defined fault-tolerant semantics for hourly billed cloud markets. We implemented the proposed model and policies both on a simulation testbed for repeatable validation and Amazon EC2. The experiments on the simulation testbed and EC2 show that the proposed approach can greatly reduce resource cost and still achieve satisfactory Quality of Service (QoS) in terms of response time and availability.
---
paper_title: SpotCheck: designing a derivative IaaS cloud on the spot market

paper_content:

Infrastructure-as-a-Service (IaaS) cloud platforms rent resources, in the form of virtual machines (VMs), under a variety of contract terms that offer different levels of risk and cost. For example, users may acquire VMs in the spot market that are often cheap but entail significant risk, since their price varies over time based on market supply and demand and they may terminate at any time if the price rises too high. Currently, users must manage all the risks associated with using spot servers. As a result, conventional wisdom holds that spot servers are only appropriate for delay-tolerant batch applications. In this paper, we propose a derivative cloud platform, called SpotCheck, that transparently manages the risks associated with using spot servers for users. SpotCheck provides the illusion of an IaaS platform that offers always-available VMs on demand for a cost near that of spot servers, and supports all types of applications, including interactive ones. SpotCheck's design combines the use of nested VMs with live bounded-time migration and novel server pool management policies to maximize availability, while balancing risk and cost. We implement SpotCheck on Amazon's EC2 and show that it i) provides nested VMs to users that are 99.9989% available, ii) achieves nearly 5x cost savings compared to using equivalent types of on-demand VMs, and iii) eliminates any risk of losing VM state.
---
paper_title: Cutting the Cost of Hosting Online Services Using Cloud Spot Markets

paper_content:

The use of cloud servers to host modern Internet-based services is becoming increasingly common. Today's cloud platforms offer a choice of server types, including non-revocable on-demand servers and cheaper but revocable spot servers. A service provider requiring servers can bid in the spot market where the price of a spot server changes dynamically according to the current supply and demand for cloud resources. Spot servers are usually cheap, but can be revoked by the cloud provider when the cloud resources are scarce. While it is well-known that spot servers can reduce the cost of performing time-flexible interruption-tolerant tasks, we explore the novel possibility of using spot servers for reducing the cost of hosting an Internet-based service such as an e-commerce site that must {\em always} be on and the penalty for service unavailability is high. By using the spot markets, we show that it is feasible to host an always-on Internet-based service at one-third to one-fifth the cost of hosting the same service in the traditional fashion using dedicated non-revocable servers. To achieve these savings, we devise a cloud scheduler that reduces the cost by intelligently bidding for spot servers. Further, the scheduler uses novel VM migration mechanisms to quickly migrate the service between spot servers and on-demand servers to avoid potential service disruptions due to spot server revocations by the cloud provider. Our work provides the first feasibility study of using cloud spot markets to significantly reduce the cost of hosting always-on Internet-based services without sacrificing service availability.
---
paper_title: Agile dynamic provisioning of multi-tier Internet applications

paper_content:

Dynamic capacity provisioning is a useful technique for handling the multi-time-scale variations seen in Internet workloads. In this article, we propose a novel dynamic provisioning technique for multi-tier Internet applications that employs (1) a flexible queuing model to determine how much of the resources to allocate to each tier of the application, and (2) a combination of predictive and reactive methods that determine when to provision these resources, both at large and small time scales. We propose a novel data center architecture based on virtual machine monitors to reduce provisioning overheads. Our experiments on a forty-machine Xen/Linux-based hosting platform demonstrate the responsiveness of our technique in handling dynamic workloads. In one scenario where a flash crowd caused the workload of a three-tier application to double, our technique was able to double the application capacity within five minutes, thus maintaining response-time targets. Our technique also reduced the overhead of switching servers across applications from several minutes to less than a second, while meeting the performance targets of residual sessions.
---
paper_title: SmartScale: Automatic Application Scaling in Enterprise Clouds

paper_content:

Enterprise clouds today support an on demand resource allocation model and can provide resources requested by applications in a near online manner using virtual machine resizing or cloning. However, in order to take advantage of an on demand resource model, enterprise applications need to be automatically scaled in a way that makes the most efficient use of resources. In this work, we present the SmartScale automated scaling framework. SmartScale uses a combination of vertical (adding more resources to existing VM instances) and horizontal (adding more VM instances) scaling to ensure that the application is scaled in a manner that optimizes both resource usage and the reconfiguration cost incurred due to scaling. The SmartScale methodology is proactive and ensures that the application converges quickly to the desired scaling level even when the workload intensity changes significantly. We evaluate SmartScale using real production traces on Olio, an emerging cloud benchmark, running on a ???-based cloud testbed. We present both theoretical and experimental evidence that comprehensively establish the effectiveness of SmartScale.
---
paper_title: Lightweight Resource Scaling for Cloud Applications

paper_content:

Elastic resource provisioning is a key feature of cloud computing, allowing users to scale up or down resource allocation for their applications at run-time. To date, most practical approaches to managing elasticity are based on allocation/de-allocation of the virtual machine (VM) instances to the application. This VM-level elasticity typically incurs both considerable overhead and extra costs, especially for applications with rapidly fluctuating demands. In this paper, we propose a lightweight approach to enable cost-effective elasticity for cloud applications. Our approach operates fine-grained scaling at the resource level itself (CPUs, memory, I/O, etc) in addition to VM-level scaling. We also present the design and implementation of an intelligent platform for light-weight resource management of cloud applications. We describe our algorithms for light-weight scaling and VM-level scaling and show their interaction. We then use an industry standard benchmark to evaluate the effectiveness of our approach and compare its performance against traditional approaches.
---
paper_title: Modeling the Impact of Workload on Cloud Resource Scaling

paper_content:

Cloud computing offers the flexibility to dynamically size the infrastructure in response to changes in workload demand. While both horizontal and vertical scaling of infrastructure is supported by major cloud providers, these scaling options differ significantly in terms of their cost, provisioning time, and their impact on workload performance. Importantly, the efficacy of horizontal and vertical scaling critically depends on the workload characteristics, such as the workload's parallelizability and its core scalability. In today's cloud systems, the scaling decision is left to the users, requiring them to fully understand the tradeoffs associated with the different scaling options. In this paper, we present our solution for optimizing the resource scaling of cloud deployments via implementation in OpenStack. The key component of our solution is the modelling engine that characterizes the workload and then quantitatively evaluates different scaling options for that workload. Our modelling engine leverages Amdahl's Law to model service time scaling in scaleup environments and queueing-theoretic concepts to model performance scaling in scale-out environments. We further employ Kalman filtering to account for inaccuracies in the model-based methodology, and to dynamically track changes in the workload and cloud environment.
---
paper_title: A cost-aware auto-scaling approach using the workload prediction in service clouds

paper_content:

Service clouds are distributed infrastructures which deploys communication services in clouds. The scalability is an important characteristic of service clouds. With the scalability, the service cloud can offer on-demand computing power and storage capacities to different services. In order to achieve the scalability, we need to know when and how to scale virtual resources assigned to different services. In this paper, a novel service cloud architecture is presented, and a linear regression model is used to predict the workload. Based on this predicted workload, an auto-scaling mechanism is proposed to scale virtual resources at different resource levels in service clouds. The auto-scaling mechanism combines the real-time scaling and the pre-scaling. Finally experimental results are provided to demonstrate that our approach can satisfy the user Service Level Agreement (SLA) while keeping scaling costs low.
---
paper_title: Model-based self-adaptive resource allocation in virtualized environments

paper_content:

The adoption of virtualization and Cloud Computing technologies promises a number of benefits such as increased flexibility, better energy efficiency and lower operating costs for IT systems. However, highly variable workloads make it challenging to provide quality-of-service guarantees while at the same time ensuring efficient resource utilization. To avoid violations of service-level agreements (SLAs) or inefficient resource usage, resource allocations have to be adapted continuously during operation to reflect changes in application workloads. In this paper, we present a novel approach to self-adaptive resource allocation in virtualized environments based on online architecture-level performance models. We present a detailed case study of a representative enterprise application, the new SPECjEnterprise2010 benchmark, deployed in a virtualized cluster environment. The case study serves as a proof-of-concept demonstrating the effectiveness and practical applicability of our approach.
---
paper_title: DEPAS: a decentralized probabilistic algorithm for auto-scaling

paper_content:

The dynamic provisioning of virtualized resources offered by cloud computing infrastructures allows applications deployed in a cloud environment to automatically increase and decrease the amount of used resources. This capability is called auto-scaling and its main purpose is to automatically adjust the scale of the system that is running the application to satisfy the varying workload with minimum resource utilization. The need for auto-scaling is particularly important during workload peaks, in which applications may need to scale up to extremely large-scale systems. Both the research community and the main cloud providers have already developed auto-scaling solutions. However, most research solutions are centralized and not suitable for managing large-scale systems, moreover cloud providers' solutions are bound to the limitations of a specific provider in terms of resource prices, availability, reliability, and connectivity. In this paper we propose DEPAS, a decentralized probabilistic auto-scaling algorithm integrated into a P2P architecture that is cloud provider independent, thus allowing the auto-scaling of services over multiple cloud infrastructures at the same time. Our simulations, which are based on real service traces, show that our approach is capable of: (i) keeping the overall utilization of all the instantiated cloud resources in a target range, (ii) maintaining service response times close to the ones obtained using optimal centralized auto-scaling approaches.
---
paper_title: GeoScale: Providing Geo-Elasticity in Distributed Clouds

paper_content:

Distributed cloud platforms are well suited for serving a geographically diverse user base. However traditional cloud provisioning mechanisms that make local scaling decisions are not well suited for temporal and spatial workload fluctuations seen by modern web applications. In this paper, we argue the need of geo-elasticity and present GeoScale, a system to provide geo-elasticity in distributed clouds. We describe GeoScale's model-driven proactive provisioning approach and conduct an initial evaluation of GeoScale on Amazon's distributed EC2 cloud. Our results show up to 31% improvement in the 95th percentile response time when compared to traditional elasticity techniques.
---
paper_title: Efficient resource placement in cloud computing and network applications

paper_content:

We address the problem of resource placement in general networking applications, in particular cloud computing. We consider a large-scale service faced by regionally distributed demands for various resources. The service aims at placing the resources across regions to maximize profit, accounting for demand granting revenues minus resource placement costs. Cloud computing and online services, utilizing regional datacenters and facing the problem of where and how much to place various servers, naturally fall under this paradigm. The main challenge posed by this setting is the need to deal with arbitrary multi-dimensional stochastic demands. We show that, despite the challenging stochastic combinatorial complexity, one can optimize the system operation using fairly efficient algorithms.
---
paper_title: Kriging-Based Self-Adaptive Cloud Controllers

paper_content:

Cloud technology is rapidly substituting classic computing solutions, and challenges the community with new problems. In this paper we focus on controllers for cloud application elasticity, and propose a novel solution for self-adaptive cloud controllers based on Kriging models. Cloud controllers are application specific schedulers that allocate resources to applications running in the cloud, aiming to meet the quality of service requirements while optimizing the execution costs. General-purpose cloud resource schedulers provide sub-optimal solutions to the problem with respect to application-specific solutions that we call cloud controllers. In this paper we discuss a general way to design self-adaptive cloud controllers based on Kriging models. We present Kriging models, and show how they can be used for building efficient controllers thanks to their unique characteristics. We report experimental data that confirm the suitability of Kriging models to support efficient cloud control and open the way to the development of a new generation of cloud controllers.
---
paper_title: Using reinforcement learning for controlling an elastic web application hosting platform

paper_content:

In this paper, we propose and implement a control mechanism that interfaces with Infrastructure as a Service (IaaS) or cloud providers to provision resources and manage instances of web applications in response to volatile and complex request patterns. We use reinforcement learning to orchestrate control actions such as provisioning servers and application placement to meet performance requirements and minimize ongoing costs. The mechanism is incorporated in a distributed, elastic hosting architecture that is evaluated using actual web applications running on resources from Amazon EC2.
---
paper_title: Using Reinforcement Learning for Autonomic Resource Allocation in Clouds: Towards a Fully Automated Workflow

paper_content:

Dynamic and appropriate resource dimensioning isa crucial issue in cloud computing. As applications go more andmore 24/7, online policies must be sought to balance performancewith the cost of allocated virtual machines. Most industrialapproaches to date use ad hoc manual policies, such as thresholdbasedones. Providing good thresholds proved to be tricky andhard to automatize to fit every application requirement. Researchis being done to apply automatic decision-making approaches,such as reinforcement learning. Yet, they face a lot of problemsto go to the field: having good policies in the early phasesof learning, time for the learning to converge to an optimalpolicy and coping with changes in the application performancebehavior over time. In this paper, we propose to deal with theseproblems using appropriate initialization for the early stages aswell as convergence speedups applied throughout the learningphases and we present our first experimental results for these.We also introduce a performance model change detection onwhich we are currently working to complete the learning processmanagement. Even though some of these proposals were knownin the reinforcement learning field, the key contribution of thispaper is to integrate them in a real cloud controller and toprogram them as an automated workflow.
---
paper_title: Inter-Cloud Architectures and Application Brokering: Taxonomy and Survey

paper_content:

Although Cloud computing itself has many open problems, researchers in the field have already made the leap to envision Inter-Cloud computing. Their goal is to achieve better overall Quality of Service QoS, reliability and cost efficiency by utilizing multiple clouds. Inter-Cloud research is still in its infancy, and the body of knowledge in the area has not been well defined yet. In this work, we propose and motivate taxonomies for Inter-Cloud architectures and application brokering mechanisms. We present a detailed survey of the state of the art in terms of both academic and industry developments 20 projects, and we fit each project onto the discussed taxonomies. We discuss how the current Inter-Cloud environments facilitate brokering of distributed applications across clouds considering their nonfunctional requirements. Finally, we analyse the existing works and identify open challenges and trends in the area of Inter-Cloud application brokering. Copyright © 2012 John Wiley & Sons, Ltd.
---
paper_title: Dynamic Service Placement in Geographically Distributed Clouds

paper_content:

Large-scale online service providers have been increasingly relying on geographically distributed cloud infrastructures for service hosting and delivery. In this context, a key challenge faced by service providers is to determine the locations where service applications should be placed such that the hosting cost is minimized while key performance requirements (e.g., response time) are ensured. Furthermore, the dynamic nature of both demand pattern and infrastructure cost favors a dynamic solution to this problem. Currently most of the existing solutions for service placement have either ignored dynamics, or provided solutions inadequate to achieve this objective. In this paper, we present a framework for dynamic service placement problems based on control- and game-theoretic models. In particular, we present a solution that optimizes the hosting cost dynamically over time according to both demand and resource price fluctuations. We further consider the case where multiple service providers compete for resources in a dynamic manner. This paper extends our previous work [1] by analyzing the outcome of the competition in terms of both price of stability and price of anarchy. Our analysis suggests that in an uncoordinated scenario where service providers behave in a selfish manner, the resulting Nash equilibrium can be arbitrarily worse than the optimal centralized solution in terms of social welfare. Based on this observation, we present a coordination mechanism that can be employed by the infrastructure provider to maximize the social welfare of the system. Finally, we demonstrate the effectiveness of our solutions using realistic simulations.
---
paper_title: Pattern Matching Based Forecast of Non-periodic Repetitive Behavior for Cloud Clients

paper_content:

The Cloud phenomenon brings along the cost-saving benefit of dynamic scaling. As a result, the question of efficient resource scaling arises. Prediction is necessary as the virtual resources that Cloud computing uses have a setup time that is not negligible. We propose an approach to the problem of workload prediction based on identifying similar past occurrences of the current short-term workload history. We present in detail the Cloud client resource auto-scaling algorithm that uses the above approach to help when scaling decisions are made, as well as experimental results by using real-world Cloud client application traces. We also present an overall evaluation of this approach, its potential and usefulness for enabling efficient auto-scaling of Cloud user resources.
---
paper_title: Business-Driven Service Placement for Highly Dynamic and Distributed Cloud Systems

paper_content:

The emergence of large-scale Cloud computing environments characterized by dynamic resource pricing schemes enables valuable cost saving opportunities for service providers that could dynamically decide to change the placement of their IT service components in order to reduce their bills. However, that requires new management solutions to dynamically reconfigure IT service components placement, in order to respond to pricing changes and to control and guarantee the high-level business objectives defined by service providers. This paper proposes a novel approach based on Genetic Algorithm (GA) optimization techniques for adaptive business-driven IT service component reconfiguration. Our proposal allows to evaluate the performance of complex IT services deployments over large-scale Cloud systems in a wide range of alternative configurations, by granting prompt transitions to more convenient placements as business values and costs change dynamically. We deeply assessed our framework in a realistic scenario that consists of 2-tier service architectures with real-world pricing schemes. Collected results show the effectiveness and quantify the overhead of our solution. The results also demonstrate the suitability of business-driven IT management techniques for service components placement and reconfiguration in highly dynamic and distributed Cloud systems.
---
paper_title: Business-driven optimization of component placement for complex services in federated Clouds

paper_content:

With the advent of connected services ecosystems, new generations of services and systems are being conceived, responding to the ever growing demands of the market place. In parallel, the effective adoption of the Cloud computing paradigm is becoming an essential enabler for business enterprises. With such importance placed on the services ecosystem, the design and management of services becomes a key issue both for the providers and the users. One of the main challenges for service providers lies in the complexity of the services, comprising of a multiplicity of technologies and competing and cooperating providers, which is difficult to address through current technology-centric service design approaches, in particular for the deployment infrastructure. The work described in this paper lays a foundation for business driven service design by proposing a business goals driven model of resource allocation in the Cloud. We define a goal/loss/processing cost function for resource allocation that we optimize, while taking into account the dynamic and varying nature of requests load.
---
paper_title: Scaling social media applications into geo-distributed clouds

paper_content:

Federation of geo-distributed cloud services is a trend in cloud computing which, by spanning multiple data centers at different geographical locations, can provide a cloud platform with much larger capacities. Such a geo-distributed cloud is ideal for supporting large-scale social media streaming applications (e.g., YouTube-like sites) with dynamic contents and demands, owing to its abundant on-demand storage/bandwidth capacities and geographical proximity to different groups of users. Although promising, its realization presents challenges on how to efficiently store and migrate contents among different cloud sites (i.e. data centers), and to distribute user requests to the appropriate sites for timely responses at modest costs. These challenges escalate when we consider the persistently increasing contents and volatile user behaviors in a social media application. By exploiting social influences among users, this paper proposes efficient proactive algorithms for dynamic, optimal scaling of a social media application in a geo-distributed cloud. Our key contribution is an online content migration and request distribution algorithm with the following features: (1) future demand prediction by novelly characterizing social influences among the users in a simple but effective epidemic model; (2) oneshot optimal content migration and request distribution based on efficient optimization algorithms to address the predicted demand, and (3) a Δ(t)-step look-ahead mechanism to adjust the one-shot optimization results towards the offline optimum. We verify the effectiveness of our algorithm using solid theoretical analysis, as well as large-scale experiments under dynamic realistic settings on a home-built cloud platform.
---
paper_title: Providing IaaS resources automatically through prediction and monitoring approaches

paper_content:

A cloud computing infrastructure management is proposed in this paper, which consists of two approaches that facilitate the provisioning of computing resources in a self-adaptive virtualized environment. Resource allocation is employed to predict the future of workload management and to employ a self-adaptive approach by using computational agents to monitor the Virtual Machines (VMs). The paper also includes the Return on Investment (ROI) formula that deals with the relationship between the prices for the Infrastructure-as-a-Service (IaaS) contracted by the customer and the effective use of this service. The experimental results show a significant improvement when self-configuration is used with agent-based computational modeling in contrast with the self-configuration based on prediction for future workload.
---
paper_title: Replicated Server Placement with QoS Constraints

paper_content:

The network planning problem of placing replicated servers with QoS constraints is considered. Each server site may consist of multiple server types with varying capacities and each site can be placed in any location among those belonging to a given set. Each client can be served by more than one location as long as the round-trip delay of data requests satisfies predetermined upper bounds. Our main focus is to minimize the cost of using the servers and utilizing the link bandwidth, while serving requests according to their delay constraint. This is an NP-hard problem. A pseudopolynomial and a polynomial algorithm that provide guaranteed approximation factors with respect to the optimal for the problem at hand are presented
---

Title: Auto-Scaling Web Applications in Clouds: A Taxonomy and Survey

Section 1: INTRODUCTION
Description 1: Introduces the concept of cloud computing, elasticity, and the necessity of auto-scaling for efficient resource management in web applications, outlining the significance and challenges.

Section 2: TAXONOMY
Description 2: Presents a taxonomy for auto-scaling web applications in clouds, classifying existing works based on identified challenges and targeted environments.

Section 3: APPLICATION ARCHITECTURES
Description 3: Describes different types of web application architectures (single-tier, multitier, service-oriented) and discusses how they impact auto-scaling strategies.

Section 4: SESSION STICKINESS
Description 4: Discusses the implications of session stickiness on auto-scaling, including strategies to manage stateful and stateless sessions in scalable environments.

Section 5: ADAPTIVITY
Description 5: Explores different levels of adaptivity in auto-scalers (nonadaptive, self-adaptive, self-adaptive switching) and their impact on performance and maintenance.

Section 6: SCALING INDICATORS
Description 6: Examines the various performance metrics (low-level, high-level, hybrid metrics) used by auto-scalers to make scaling decisions.

Section 7: RESOURCE ESTIMATION
Description 7: Details different approaches to resource estimation (rule-based, fuzzy inference, application profiling, analytical modeling, machine learning, hybrid) and their respective advantages and challenges.

Section 8: OSCILLATION MITIGATION
Description 8: Discusses techniques to mitigate oscillation in auto-scaling, such as cooling time and dynamic parameter adjustment, to avoid resource wastage and SLA violations.

Section 9: SCALING TIMING
Description 9: Differentiates between reactive and proactive scaling approaches, emphasizing the importance of accurate workload prediction for proactive scaling.

Section 10: SCALING METHODS
Description 10: Investigates the key factors and challenges in vertical and horizontal scaling methods and the benefits of employing hybrid scaling strategies.

Section 11: ENVIRONMENT
Description 11: Summarizes the design considerations for auto-scalers in single-cloud environments and expands the discussion to the complexities and strategies for multi-cloud deployments.

Section 12: DISCUSSION AND FUTURE DIRECTIONS
Description 12: Identifies gaps between current solutions and an ideal auto-scaler, providing insights and potential future directions for research and development in auto-scaling technologies.

Section 13: SUMMARY AND CONCLUSIONS
Description 13: Concludes the survey by summarizing the main findings and contributions, highlighting the importance and advancements of auto-scaling techniques for web applications in cloud environments.