An overview of energy efficiency techniques in cluster computing systems
---
paper_title: The Green500 List: Encouraging Sustainable Supercomputing

paper_content:

The performance-at-any-cost design mentality ignores supercomputers' excessive power consumption and need for heat dissipation and will ultimately limit their performance. Without fundamental change in the design of supercomputing systems, the performance advances common over the past two decades won't continue. The HPC community needs a Green500 List to rank supercomputers on speed and power requirements and to supplement the TOP500 List. Vendors and system architects worldwide take substantial pride and invest tremendous effort toward making the biannual TOP500 List. We anticipate that the Green500 List effort will do the same and encourage the HPC community and operators of Internet data centers to design more power-efficient supercomputers and large-scale data centers.
---
paper_title: High Performance Cluster Computing: Architectures and Systems

paper_content:

From the Publisher: ::: Rapid improvements in network and processor performance are revolutionizing high performance computing, transforming clustered commodity workstations into the supercomputing solution of choice. This book brings together contributions from more than 100 leading practitioners, offering a single source for up-to-the-minute information on virtually every key system-related issue in high performance cluster computing. ::: The book contains expert coverage of "commodity supercomputing" systems and architectures; Internet-based wide area "metacomputing" systems; the role of Java; new applications and algorithms; advanced techniques for enhancing availability and throughput, and much more.
---
paper_title: Performance-constrained Distributed DVS Scheduling for Scientific Applications on Power-aware Clusters

paper_content:

Left unchecked, the fundamental drive to increase peak performance using tens of thousands of power hungry components will lead to intolerable operating costs and failure rates. High-performance, power-aware distributed computing reduces power and energy consumption of distributed applications and systems without sacrificing performance. Generally, we use DVS (Dynamic Voltage Scaling) technology now available in high-performance microprocessors to reduce power consumption during parallel application runs when peak CPU performance is not necessary due to load imbalance, communication delays, etc. We propose distributed performance-directed DVS scheduling strategies for use in scalable power-aware HPC clusters. By varying scheduling granularity we can obtain significant energy savings without increasing execution time (36% for FT from NAS PB). We created a software framework to implement and evaluate our various techniques and show performance-directed scheduling consistently saves more energy (nearly 25% for several codes) than comparable approaches with less impact on execution time (< 5%). Additionally, we illustrate the use of energy-delay products to automatically select distributed DVS schedules that meet users' needs.
---
paper_title: High-performance, power-aware distributed computing for scientific applications

paper_content:

The PowerPack framework enables distributed systems to profile, analyze, and conserve energy in scientific applications using dynamic voltage scaling. For one common benchmark, the framework achieves more than 30 percent energy savings with minimal performance impact.
---
paper_title: Power Aware Scheduling of Bag-of-Tasks Applications with Deadline Constraints on DVS-enabled Clusters

paper_content:

Power-aware scheduling problem has been a recent issue in cluster systems not only for operational cost due to electricity cost, but also for system reliability. As recent commodity processors support multiple operating points under various supply voltage levels, Dynamic Voltage Scaling (DVS) scheduling algorithms can reduce power consumption by controlling appropriate voltage levels. In this paper, we provide power-aware scheduling algorithms for bag-of-tasks applications with deadline constraints on DVS-enabled cluster systems in order to minimize power consumption as well as to meet the deadlines specified by application users. A bag-of-tasks application should finish all the sub-tasks before the deadline, so that the DVS scheduling scheme should consider the deadline as well. We provide the DVS scheduling algorithms for both time-shared and space-shared resource sharing policies. The simulation results show that the proposed algorithms reduce much power consumption compared to static voltage schemes.
---
paper_title: Making cluster applications energy-aware

paper_content:

Power consumption has become a critical issue in large scale clusters. Existing solutions for addressing the servers' energy consumption suggest "shrinking" the set of active machines, at least until the more power-proportional hardware devices become available. This paper demonstrates that leveraging the sleeping state, however, may lead to unacceptably poor performance and low data availability if the distributed services are not aware of the power management's actions. Therefore, we present an architecture for cluster services in which the deployed services overcome this problem by actively participating in any action taken by the power management. We propose, implement, and evaluate modifications for the Hadoop Distributed File System and the MapReduce clone that make them capable of operating efficiently under limited power budgets.
---
paper_title: A taxonomy of market-based resource management systems for utility-driven cluster computing

paper_content:

In utility-driven cluster computing, cluster Resource Management Systems (RMSs) need to know the specific needs of different users in order to allocate resources according to their needs. This in turn is vital to achieve service-oriented Grid computing that harnesses resources distributed worldwide based on users' objectives. Recently, numerous market-based RMSs have been proposed to make use of real-world market concepts and behavior to assign resources to users for various computing platforms. The aim of this paper is to develop a taxonomy that characterizes and classifies how market-based RMSs can support utility-driven cluster computing in practice. The taxonomy is then mapped to existing market-based RMSs designed for both cluster and other computing platforms to survey current research developments and identify outstanding issues. Copyright © 2006 John Wiley & Sons, Ltd.
---
paper_title: A Taxonomy and Survey of Energy-Efficient Data Centers and Cloud Computing Systems

paper_content:

Abstract Traditionally, the development of computing systems has been focused on performance improvements driven by the demand of applications from consumer, scientific, and business domains. However, the ever-increasing energy consumption of computing systems has started to limit further performance growth due to overwhelming electricity bills and carbon dioxide footprints. Therefore, the goal of the computer system design has been shifted to power and energy efficiency. To identify open challenges in the area and facilitate future advancements, it is essential to synthesize and classify the research on power- and energy-efficient design conducted to date. In this study, we discuss causes and problems of high power/energy consumption, and present a taxonomy of energy-efficient design of computing systems covering the hardware, operating system, virtualization, and data center levels. We survey various key works in the area and map them onto our taxonomy to guide future design and development efforts. This chapter concludes with a discussion on advancements identified in energy-efficient computing and our vision for future research directions.
---
paper_title: Single System Image

paper_content:

PCT No. PCT/US93/09895 Sec. 371 Date Apr. 14, 1995 Sec. 102(e) Date Apr. 14, 1995 PCT Filed Oct. 14, 1993 PCT Pub. No. WO94/08601 PCT Pub. Date Apr. 28, 1994B cell lymphoma tumor-associated antigen or a fragment thereof containing an epitope are linked to an immune-enhancing cytokine, such as GM-CSF, IL-2, or IL-4 to form an immuno-complex. This immuno-complex elicits immune responses which are protective with respect to tumor proliferation. The linkers may be simple chemical bifunctional moieties introduced through chemical synthetic techniques or peptides introduce through recombinant methodologies. Antibodies immunoreactive with these immunocomplexes are also useful as passive vaccines and as analytical tools.
---
paper_title: Improvement of power-performance efficiency for high-end computing

paper_content:

Left unchecked, the fundamental drive to increase peak performance using tens of thousands of power hungry components will lead to intolerable operating costs and failure rates. Recent work has shown application characteristics of single-processor, memory-bound non-interactive codes and distributed, interactive Web services can be exploited to conserve power and energy with minimal performance impact. Our novel approach is to exploit parallel performance inefficiencies characteristic of non-interactive, distributed scientific applications, conserving energy using DVS (dynamic voltage scaling) without impacting time-to-solution (ITS) significantly, reducing cost and improving reliability. We present a software framework to analyze and optimize distributed power-performance using DVS implemented on a 16-node Centrino-based cluster. Using various DVS strategies we achieve application-dependent overall system energy savings as large as 25% with as little as 2% performance impact.
---
paper_title: Determining the Minimum Energy Consumption using Dynamic Voltage and Frequency Scaling

paper_content:

While improving raw performance is of primary interest to most users of high-performance computers, energy consumption also is a critical concern. Some microprocessors allow voltage and frequency scaling, which enables a system to reduce CPU power and performance when the CPU is not on the critical path. When properly directed, such dynamic voltage and frequency scaling can produce significant energy savings with little performance penalty. Various DVFS scaling algorithms have been proposed. However, the benefit is application-dependent. We cannot see if they achieve the energy consumption as minimum as possible. So, it is important to establish the baseline of the DVFS scheduling for any application. This paper determines minimum energy consumption in voltage and frequency scaling systems for a given time delay. We assume we have a set of fixed points where scaling can occur. A brute-force solution is intractable even for a moderately sized set (although all programs presented in this paper can be solved with the brute-force). Our algorithm efficiently chooses the exact optimal schedule satisfying the given time constraint by estimation. Besides, our time and energy estimations from the optimal schedule have reasonable accuracy with 1.48% of differences at maximum.
---
paper_title: High-Density Computing: A 240-Processor Beowulf in One Cubic Meter

paper_content:

We present results from computations on Green Destiny, a 240-processor Beowulf cluster which is contained entirely within a single 19-inch wide 42U rack. The cluster consists of 240 Transmeta TM5600 667-MHz CPUs mounted on RLX Technologies motherboard blades. The blades are mounted side-by-side in an RLX 3U rack-mount chassis, which holds 24 blades. The overall cluster contains 10 chassis and associated Fast and Gigabit Ethernet switches. The system has a footprint of 0.5 meter 2 (6 square feet), a volume of 0.85 meter 3 (30 cubic feet) and a measured power dissipation under load of 5200 watts (including network switches). We have measured the performance of the cluster using a gravitational treecode N-body simulation of galaxy formation using 200 million particles, which sustained an average of 38.9 Gflops on 212 nodes of the system. We also present results from a three-dimensional hydrodynamic simulation of a core-collapse supernova.
---
paper_title: Performance-constrained Distributed DVS Scheduling for Scientific Applications on Power-aware Clusters

paper_content:

Left unchecked, the fundamental drive to increase peak performance using tens of thousands of power hungry components will lead to intolerable operating costs and failure rates. High-performance, power-aware distributed computing reduces power and energy consumption of distributed applications and systems without sacrificing performance. Generally, we use DVS (Dynamic Voltage Scaling) technology now available in high-performance microprocessors to reduce power consumption during parallel application runs when peak CPU performance is not necessary due to load imbalance, communication delays, etc. We propose distributed performance-directed DVS scheduling strategies for use in scalable power-aware HPC clusters. By varying scheduling granularity we can obtain significant energy savings without increasing execution time (36% for FT from NAS PB). We created a software framework to implement and evaluate our various techniques and show performance-directed scheduling consistently saves more energy (nearly 25% for several codes) than comparable approaches with less impact on execution time (< 5%). Additionally, we illustrate the use of energy-delay products to automatically select distributed DVS schedules that meet users' needs.
---
paper_title: State of the Art of Power Saving in Clusters and Results from the EDF Case Study

paper_content:

Energy management for mobile devices has been traditionally a well studied topic during the last two decades, as these devices usually do not have a permanent connection to the power grid and thus solely rely on the limited battery charge. However, this trend has been mostly disregarded in the context of HPC systems as the main focus mainly relied on improving the performance at any cost. Therefore, energy costs for operating and cooling the equipment of current data centers have increased significantly up to a point where they are able to surpass the hardware acquisition costs. In this work we survey the current energy conservation efforts in distributed systems. Hence, we start our work with the introduction of the basic principles of power and energy management. This involves the distinguishing between the two often misleading terms power and energy management. Furthermore, we present several approaches on how to measure the power consumption at system and component level. Moreover, in order to get a deeper understanding on where the most of the power is spent within a server, we take a closer look at its components (i.e. CPU, RAM, etc.) and outline the currently available energy-saving mechanisms. Given that energy consumption strongly depends on the workload characteristics we also discuss the different types of workloads (i.e. mobile, commercial and scientific). Afterwards, we continue with the current approaches for energy savings in distributed systems by focusing on both node and cluster-level efforts. Finally, we finish this survey with an outline of some open challenges and the introduction of the EcoGrappe project. Thereby, we detail its objectives and present some of the work done at our project partner EDF.
---
paper_title: Energy-efficient cluster computing with FAWN: Workloads and implications

paper_content:

This paper presents the architecture and motivation for a cluster-based, many-core computing architecture for energy-efficient, data-intensive computing. FAWN, a Fast Array of Wimpy Nodes, consists of a large number of slower but efficient nodes coupled with low-power storage. We present the computing trends that motivate a FAWN-like approach, for CPU, memory, and storage. We follow with a set of microbenchmarks to explore under what workloads these "wimpy nodes" perform well (or perform poorly). We conclude with an outline of the longer-term implications of FAWN that lead us to select a tightly integrated stacked chip-and-memory architecture for future FAWN development.
---
paper_title: Memory-miser: a performance-constrained runtime system for power-scalable clusters

paper_content:

Main memory in clusters may dominate total system power. The resulting energy consumption increases system operating cost and the heat produced reduces reliability. Emergent memory technology will provide servers with the ability to dynamically turn-on (online) and turn-off (offline) memory devices at runtime. This technology, coupled with slack in memory demand, offers the potential for significant energy savings in clusters of servers. Enabling power-aware memory and conserving energy in clusters are non-trivial. First, power-aware memory techniques must be scalable to thousands of devices. Second, techniques must not negatively impact the performance of parallel scientific applications. Third, techniques must be transparent to the user to be practical. We propose a Memory Management Infra-Structure for Energy Reduction (Memory MISER). Memory MISER is transparent, performance-neutral, and scalable. It consists of a prototype Linux kernel that manages memory at device granularity and a userspace daemon that monitors memory demand systemically to control devices and implement energy- and performance-constrained policies. Experiments on an 8-node cluster show our control daemon reduces memory energy up to 56.8% with
---
paper_title: FAWN: a fast array of wimpy nodes

paper_content:

This paper presents a new cluster architecture for low-power data-intensive computing. FAWN couples low-power embedded CPUs to small amounts of local flash storage, and balances computation and I/O capabilities to enable efficient, massively parallel access to data. The key contributions of this paper are the principles of the FAWN architecture and the design and implementation of FAWN-KV--a consistent, replicated, highly available, and high-performance key-value storage system built on a FAWN prototype. Our design centers around purely log-structured datastores that provide the basis for high performance on flash storage, as well as for replication and consistency obtained using chain replication on a consistent hashing ring. Our evaluation demonstrates that FAWN clusters can handle roughly 350 key-value queries per Joule of energy--two orders of magnitude more than a disk-based system.
---
paper_title: A Feasibility Analysis of Power Awareness in Commodity-Based High-Performance Clusters

paper_content:

We present a feasibility study of a power-reduction scheme that reduces the thermal power of processors by lowering frequency and voltage in the context of high-performance computing. The study revolves around a 16-processor Opteron-based Beowulf cluster, configured as four nodes of quad-processors, and shows that one can easily reduce a significant amount of CPU and system power dissipation and its associated energy costs while still maintaining high performance. Specifically, our study shows that a 5% performance slowdown can be traded off for an average of 19% system energy savings and 24% system power reduction. These preliminary empirical results, via real measurements, are encouraging because hardware failures often occur when the cluster is running hot, i.e, when the workload is heavy, and the new power-reduction scheme can effectively reduce a cluster's power demands during these busy periods
---
paper_title: Gordon: using flash memory to build fast, power-efficient clusters for data-intensive applications

paper_content:

As our society becomes more information-driven, we have begun to amass data at an astounding and accelerating rate. At the same time, power concerns have made it difficult to bring the necessary processing power to bear on querying, processing, and understanding this data. We describe Gordon, a system architecture for data-centric applications that combines low-power processors, flash memory, and data-centric programming systems to improve performance for data-centric applications while reducing power consumption. The paper presents an exhaustive analysis of the design space of Gordon systems, focusing on the trade-offs between power, energy, and performance that Gordon must make. It analyzes the impact of flash-storage and the Gordon architecture on the performance and power efficiency of data-centric applications. It also describes a novel flash translation layer tailored to data intensive workloads and large flash storage arrays. Our data show that, using technologies available in the near future, Gordon systems can out-perform disk-based clusters by 1.5× and deliver up to 2.5× more performance per Watt.
---
paper_title: Dynamic cluster reconfiguration for power and performance

paper_content:

In this chapter we address power conservation for clusters of workstations or PCs. Our approach is to develop systems that dynamically turn cluster nodes on - to be able to handle the load imposed on the system efficiently - and off - to save power under lighter load. The key component of our systems is an algorithm that makes cluster reconfiguration decisions by considering the total load imposed on the system and the power and performance implications of changing the current configuration. The algorithm is implemented in two common cluster-based systems: a network server and an operating system for clustered cycle servers. Our experimental results are very favorable, showing that our systems conserve both power and energy in comparison to traditional systems.
---
paper_title: Load Balancing and Unbalancing for Power and Performance in Cluster-Based Systems

paper_content:

In this paper we address power conservation for clusters of workstations or PCs. Our approach is to develop systems that dynamically turn cluster nodes on – to be able to handle the load imposed on the system efficiently – and off – to save power under lighter load. The key component of our systems is an algorithm that makes load balancing and unbalancing decisions by considering both the total load imposed on the cluster and the power and performance implications of turning nodes off. The algorithm is implemented in two different ways: (1) at the application level for a cluster-based, localityconscious network server; and (2) at the operating system level for an operating system for clustered cycle servers. Our experimental results are very favorable, showing that our systems conserve both power and energy in comparison to traditional systems.
---
paper_title: A Taxonomy and Survey of Energy-Efficient Data Centers and Cloud Computing Systems

paper_content:

Abstract Traditionally, the development of computing systems has been focused on performance improvements driven by the demand of applications from consumer, scientific, and business domains. However, the ever-increasing energy consumption of computing systems has started to limit further performance growth due to overwhelming electricity bills and carbon dioxide footprints. Therefore, the goal of the computer system design has been shifted to power and energy efficiency. To identify open challenges in the area and facilitate future advancements, it is essential to synthesize and classify the research on power- and energy-efficient design conducted to date. In this study, we discuss causes and problems of high power/energy consumption, and present a taxonomy of energy-efficient design of computing systems covering the hardware, operating system, virtualization, and data center levels. We survey various key works in the area and map them onto our taxonomy to guide future design and development efforts. This chapter concludes with a discussion on advancements identified in energy-efficient computing and our vision for future research directions.
---
paper_title: Improvement of power-performance efficiency for high-end computing

paper_content:

Left unchecked, the fundamental drive to increase peak performance using tens of thousands of power hungry components will lead to intolerable operating costs and failure rates. Recent work has shown application characteristics of single-processor, memory-bound non-interactive codes and distributed, interactive Web services can be exploited to conserve power and energy with minimal performance impact. Our novel approach is to exploit parallel performance inefficiencies characteristic of non-interactive, distributed scientific applications, conserving energy using DVS (dynamic voltage scaling) without impacting time-to-solution (ITS) significantly, reducing cost and improving reliability. We present a software framework to analyze and optimize distributed power-performance using DVS implemented on a 16-node Centrino-based cluster. Using various DVS strategies we achieve application-dependent overall system energy savings as large as 25% with as little as 2% performance impact.
---
paper_title: Performance-constrained Distributed DVS Scheduling for Scientific Applications on Power-aware Clusters

paper_content:

Left unchecked, the fundamental drive to increase peak performance using tens of thousands of power hungry components will lead to intolerable operating costs and failure rates. High-performance, power-aware distributed computing reduces power and energy consumption of distributed applications and systems without sacrificing performance. Generally, we use DVS (Dynamic Voltage Scaling) technology now available in high-performance microprocessors to reduce power consumption during parallel application runs when peak CPU performance is not necessary due to load imbalance, communication delays, etc. We propose distributed performance-directed DVS scheduling strategies for use in scalable power-aware HPC clusters. By varying scheduling granularity we can obtain significant energy savings without increasing execution time (36% for FT from NAS PB). We created a software framework to implement and evaluate our various techniques and show performance-directed scheduling consistently saves more energy (nearly 25% for several codes) than comparable approaches with less impact on execution time (< 5%). Additionally, we illustrate the use of energy-delay products to automatically select distributed DVS schedules that meet users' needs.
---
paper_title: Memory-miser: a performance-constrained runtime system for power-scalable clusters

paper_content:

Main memory in clusters may dominate total system power. The resulting energy consumption increases system operating cost and the heat produced reduces reliability. Emergent memory technology will provide servers with the ability to dynamically turn-on (online) and turn-off (offline) memory devices at runtime. This technology, coupled with slack in memory demand, offers the potential for significant energy savings in clusters of servers. Enabling power-aware memory and conserving energy in clusters are non-trivial. First, power-aware memory techniques must be scalable to thousands of devices. Second, techniques must not negatively impact the performance of parallel scientific applications. Third, techniques must be transparent to the user to be practical. We propose a Memory Management Infra-Structure for Energy Reduction (Memory MISER). Memory MISER is transparent, performance-neutral, and scalable. It consists of a prototype Linux kernel that manages memory at device granularity and a userspace daemon that monitors memory demand systemically to control devices and implement energy- and performance-constrained policies. Experiments on an 8-node cluster show our control daemon reduces memory energy up to 56.8% with
---
paper_title: Automatic performance setting for dynamic voltage scaling

paper_content:

The emphasis on processors that are both low power and high performance has resulted in the incorporation of dynamic voltage scaling into processor designs. This feature allows one to make fine granularity trade-offs between power use and performance, provided there is a mechanism in the OS to control that trade-off. In this paper, we describe a novel software approach to automatically controlling dynamic voltage scaling in order to optimize energy use. Our mechanism is implemented in the Linux kernel and requires no modification of user programs. Unlike previous automated approaches, our method works equally well with irregular and multiprogrammed workloads. Moreover, it has the ability to ensure that the quality of interactive performance is within user specified parameters. Our experiments show that as a result of our algorithm, processor energy savings of as much as 75% can be achieved with only a minimal impact on the user experience.
---
paper_title: Improvement of power-performance efficiency for high-end computing

paper_content:

Left unchecked, the fundamental drive to increase peak performance using tens of thousands of power hungry components will lead to intolerable operating costs and failure rates. Recent work has shown application characteristics of single-processor, memory-bound non-interactive codes and distributed, interactive Web services can be exploited to conserve power and energy with minimal performance impact. Our novel approach is to exploit parallel performance inefficiencies characteristic of non-interactive, distributed scientific applications, conserving energy using DVS (dynamic voltage scaling) without impacting time-to-solution (ITS) significantly, reducing cost and improving reliability. We present a software framework to analyze and optimize distributed power-performance using DVS implemented on a 16-node Centrino-based cluster. Using various DVS strategies we achieve application-dependent overall system energy savings as large as 25% with as little as 2% performance impact.
---
paper_title: An Energy-Efficient Scheduling Algorithm Using Dynamic Voltage Scaling for Parallel Applications on Clusters

paper_content:

In the past decade cluster computing platforms have been widely applied to support a variety of scientific and commercial applications, many of which are parallel in nature. However, scheduling parallel applications on large scale clusters is technically challenging due to significant communication latencies and high energy consumption. As such, shortening schedule length and conserving energy consumption are two major concerns in designing economical and environmentally friendly clusters. In this paper, we propose an energy-efficient scheduling algorithm (TDVAS) using the dynamic voltage scaling technique to provide significant energy savings for clusters. The TDVAS algorithm aims at judiciously leveraging processor idle times to lower processor voltages (i.e., the dynamic voltage scaling technique or DVS), thereby reducing energy consumption experienced by parallel applications running on clusters. Reducing processor voltages, however, can inevitably lead to increased execution times of parallel task. The salient feature of the TDVAS algorithm is to tackle this problem by exploiting tasks precedence constraints. Thus, TDVAS applies the DVS technique to parallel tasks followed by idle processor times to conserve energy consumption without increasing schedule lengths of parallel applications. Experimental results clearly show that the TDVAS algorithm is conducive to reducing energy dissipation in large-scale clusters without adversely affecting system performance.
---
paper_title: Adaptive, transparent frequency and voltage scaling of communication phases in MPI programs

paper_content:

Although users of high-performance computing are most interested in raw performance, both energy and power consumption have become critical concerns. Some microprocessors allow frequency and voltage scaling, which enables a system to reduce CPU performance and power when the CPU is not on the critical path. When properly directed, such dynamic frequency and voltage scaling can produce significant energy savings with little performance penalty. This paper presents an MPI runtime system that dynamically reduces CPU performance during communication phases in MPI programs. It dynamically identifies such phases and, without profiling or training, selects the CPU frequency in order to minimize energy-delay product. All analysis and subsequent frequency and voltage scaling is within MPI and so is entirely transparent to the application. This means that the large number of existing MPI programs, as well as new ones being developed, can use our system without modification. Results show that the average reduction in energy-delay product over the NAS benchmark suite is 10% - the average energy reduction is 12% while the average execution time increase is only 2.1%
---
paper_title: Reducing power with performance constraints for parallel sparse applications

paper_content:

Sparse and irregular computations constitute a large fraction of applications in the data-intensive scientific domain. While every effort is made to balance the computational workload in such computations across parallel processors, achieving sustained near machine-peak performance with close-to-ideal load balanced computation-to-processor mapping is inherently difficult. As a result, most of the time, the loads assigned to parallel processors can exhibit significant variations. While there have been numerous past efforts that study this imbalance from the performance viewpoint, to our knowledge, no prior study has considered exploiting the imbalance for reducing power consumption during execution. Power consumption in large-scale clusters of workstations is becoming a critical issue as noted by several recent research papers from both industry and academia. Focusing on sparse matrix computations in which underlying parallel computations and data dependencies can be represented by trees, this paper proposes schemes that save power through voltage/frequency scaling. Our goal is to reduce overall energy consumption by scaling the voltages/frequencies of those processors that are not in the critical path; i.e., our approach is oriented towards saving power without incurring performance penalties.
---
paper_title: Performance-constrained Distributed DVS Scheduling for Scientific Applications on Power-aware Clusters

paper_content:

Left unchecked, the fundamental drive to increase peak performance using tens of thousands of power hungry components will lead to intolerable operating costs and failure rates. High-performance, power-aware distributed computing reduces power and energy consumption of distributed applications and systems without sacrificing performance. Generally, we use DVS (Dynamic Voltage Scaling) technology now available in high-performance microprocessors to reduce power consumption during parallel application runs when peak CPU performance is not necessary due to load imbalance, communication delays, etc. We propose distributed performance-directed DVS scheduling strategies for use in scalable power-aware HPC clusters. By varying scheduling granularity we can obtain significant energy savings without increasing execution time (36% for FT from NAS PB). We created a software framework to implement and evaluate our various techniques and show performance-directed scheduling consistently saves more energy (nearly 25% for several codes) than comparable approaches with less impact on execution time (< 5%). Additionally, we illustrate the use of energy-delay products to automatically select distributed DVS schedules that meet users' needs.
---
paper_title: Energy-Efficient Cluster Computing via Accurate Workload Characterization

paper_content:

This paper presents an eco-friendly daemon that reduces power and energy consumption while better maintaining high performance via an accurate workload characterization that infers “processor stall cycles due to off-chip activities.” The eco-friendly daemon is an interval-based, run-time algorithm that uses the workload characterization to dynamically adjust a processor’s frequency and voltage to reduce power and energy consumption with little impact on application performance. Using the NAS Parallel Benchmarks as our workload, we then evaluate our eco-friendly daemon on a cluster computer. The results indicate that our workload characterization allows the power-aware daemon to more tightly control performance (5% loss instead of 11%) while delivering substantial energy savings (11% instead of 8%).
---
paper_title: Profile-based optimization of power performance by using dynamic voltage scaling on a PC cluster

paper_content:

Currently, several of the high performance processors used in a PC cluster have a DVS (dynamic voltage scaling) architecture that can dynamically scale processor voltage and frequency. Adaptive scheduling of the voltage and frequency enables us to reduce power dissipation without a performance slowdown during communication and memory access. In this paper, we propose a method of profiled-based power-performance optimization by DVS scheduling in a high-performance PC cluster. We divide the program execution into several regions and select the best gear for power efficiency. Selecting the best gear is not straightforward since the overhead of DVS transition is not free. We propose an optimization algorithm to select a gear using the execution and power profile by taking the transition overhead into account. We have built and designed a power-profiling system, PowerWatch. With this system we examined the effectiveness of our optimization algorithm on two types of power-scalable clusters (Crusoe and Turion). According to the results of benchmark tests, we achieved almost 40% reduction in terms of EDP (energy-delay product) without performance impact (less than 5%) compared to results using the standard clock frequency.
---
paper_title: Exploring the energy-time tradeoff in MPI programs on a power-scalable cluster

paper_content:

Recently, energy has become an important issue in high-performance computing. For example, supercomputers that have energy in mind, such as BlueGene/L, have been built; the idea is to improve the energy efficiency of nodes. Our approach, which uses off-the-shelf, high-performance cluster nodes that are frequency scalable, allows energy saving by scaling down the CPU. This paper investigates the energy consumption and execution time of applications from a standard benchmark suite (NAS) on a power-scalable cluster. We study via direct measurement and simulation both intra-node and inter-node effects of memory and communication bottlenecks, respectively. Additionally, we compare energy consumption and execution time across different numbers of nodes. Our results show that a power-scalable cluster has the potential to save energy by scaling the processor down to lower energy levels. Furthermore, we found that for some programs, it is possible to both consume less energy and execute in less time when using a larger number of nodes, each at reduced energy. Additionally, we developed and validated a model that enables us to predict the energy-time tradeoff of larger clusters.
---
paper_title: CPU MISER: A Performance-Directed, Run-Time System for Power-Aware Clusters

paper_content:

Performance and power are critical design constraints in today's high-end computing systems. Reducing power consumption without impacting system performance is a challenge for the HPC community. We present a runtime system (CPU MISER) and an integrated performance model for performance-directed, power-aware cluster computing. CPU MISER supports system-wide, application-independent, fine-grain, dynamic voltage and frequency scaling (DVFS) based power management for a generic power-aware cluster. Experimental results show that CPU MISER can achieve as much as 20% energy savings for the NAS parallel benchmarks. In addition to energy savings, CPU MISER is able to constrain performance loss for most applications within user-specified limits. These constraints are achieved through accurate performance modeling and prediction, coupled with advanced control techniques.
---
paper_title: Using multiple energy gears in MPI programs on a power-scalable cluster

paper_content:

Recently, system architects have built low-power, high-performance clusters, such as Green Destiny. The idea behind these clusters is to improve the energy efficiency of nodes. However, these clusters save power at the expense of performance. Our approach is instead to use high-performance cluster nodes that are frequency- and voltage-scalable; energy can than be saved by scaling down the CPU. Our prior work has examined the costs and benefits of executing an entire application at a single reduced frequency.This paper presents a framework for executing a single application in several frequency-voltage settings. The basic idea is to first divide programs into phases and then execute a series of experiments, with each phase assigned a prescribed frequency. During each experiment, we measure energy consumption and time and then use a heuristic to choose the assignment of frequency to phase for the next experiment.Our results show that significant energy can be saved without an undue performance penalty; particularly, our heuristic finds assignments of frequency to phase that is superior to any fixed-frequency solution. Specifically, this paper shows that more than half of the NAS benchmarks exhibit a better energy-time tradeoff using multiple gears than using a single gear. For example, IS using multiple gears uses 9% less energy and executes in 1% less time than the closest single-gear solution. Compared to no frequency scaling, multiple gear IS uses 16% less energy while executing only 1% longer.
---
paper_title: Dynamic cluster reconfiguration for power and performance

paper_content:

In this chapter we address power conservation for clusters of workstations or PCs. Our approach is to develop systems that dynamically turn cluster nodes on - to be able to handle the load imposed on the system efficiently - and off - to save power under lighter load. The key component of our systems is an algorithm that makes cluster reconfiguration decisions by considering the total load imposed on the system and the power and performance implications of changing the current configuration. The algorithm is implemented in two common cluster-based systems: a network server and an operating system for clustered cycle servers. Our experimental results are very favorable, showing that our systems conserve both power and energy in comparison to traditional systems.
---
paper_title: Load Balancing and Unbalancing for Power and Performance in Cluster-Based Systems

paper_content:

In this paper we address power conservation for clusters of workstations or PCs. Our approach is to develop systems that dynamically turn cluster nodes on – to be able to handle the load imposed on the system efficiently – and off – to save power under lighter load. The key component of our systems is an algorithm that makes load balancing and unbalancing decisions by considering both the total load imposed on the cluster and the power and performance implications of turning nodes off. The algorithm is implemented in two different ways: (1) at the application level for a cluster-based, localityconscious network server; and (2) at the operating system level for an operating system for clustered cycle servers. Our experimental results are very favorable, showing that our systems conserve both power and energy in comparison to traditional systems.
---
paper_title: Adaptive, transparent frequency and voltage scaling of communication phases in MPI programs

paper_content:

Although users of high-performance computing are most interested in raw performance, both energy and power consumption have become critical concerns. Some microprocessors allow frequency and voltage scaling, which enables a system to reduce CPU performance and power when the CPU is not on the critical path. When properly directed, such dynamic frequency and voltage scaling can produce significant energy savings with little performance penalty. This paper presents an MPI runtime system that dynamically reduces CPU performance during communication phases in MPI programs. It dynamically identifies such phases and, without profiling or training, selects the CPU frequency in order to minimize energy-delay product. All analysis and subsequent frequency and voltage scaling is within MPI and so is entirely transparent to the application. This means that the large number of existing MPI programs, as well as new ones being developed, can use our system without modification. Results show that the average reduction in energy-delay product over the NAS benchmark suite is 10% - the average energy reduction is 12% while the average execution time increase is only 2.1%
---
paper_title: Performance-constrained Distributed DVS Scheduling for Scientific Applications on Power-aware Clusters

paper_content:

Left unchecked, the fundamental drive to increase peak performance using tens of thousands of power hungry components will lead to intolerable operating costs and failure rates. High-performance, power-aware distributed computing reduces power and energy consumption of distributed applications and systems without sacrificing performance. Generally, we use DVS (Dynamic Voltage Scaling) technology now available in high-performance microprocessors to reduce power consumption during parallel application runs when peak CPU performance is not necessary due to load imbalance, communication delays, etc. We propose distributed performance-directed DVS scheduling strategies for use in scalable power-aware HPC clusters. By varying scheduling granularity we can obtain significant energy savings without increasing execution time (36% for FT from NAS PB). We created a software framework to implement and evaluate our various techniques and show performance-directed scheduling consistently saves more energy (nearly 25% for several codes) than comparable approaches with less impact on execution time (< 5%). Additionally, we illustrate the use of energy-delay products to automatically select distributed DVS schedules that meet users' needs.
---

Title: An Overview of Energy Efficiency Techniques in Cluster Computing Systems

Section 1: Introduction and motivation
Description 1: This section provides the background, historical context, and the importance of energy efficiency in cluster computing systems.

Section 2: Static power management
Description 2: This section discusses various static power management (SPM) techniques, including the use of low-power components and examples of successful implementations.

Section 3: Dynamic power management
Description 3: This section explores dynamic power management (DPM) techniques, focusing on real-time adaptation based on resource utilization and workload.

Section 4: Software and power-scalable components
Description 4: This section elaborates on the role of software and power-scalable components in DPM, including dynamic voltage and frequency scaling (DVFS) modules and their related scheduling strategies.

Section 5: Power-scalable memory
Description 5: This section covers dynamic power-scalable memory management systems and their effectiveness in reducing energy consumption in cluster computing.

Section 6: Power-scalable processors
Description 6: This section examines power-scalable processors, such as those using DVFS, and their impact on energy savings and performance.

Section 7: Load balancing
Description 7: This section discusses load balancing techniques aimed at distributing workload to achieve optimal resource utilization and energy efficiency.

Section 8: Discussion
Description 8: This section summarizes the findings, highlights the challenges of achieving energy efficiency in cluster computing, and suggests future directions in terms of power management techniques.

Section 9: Conclusion
Description 9: This section concludes the paper by summarizing the key points discussed and their implications for energy efficiency in cluster computing systems.