A Survey of Techniques for Architecting and Managing GPU Register File
---
paper_title: GPUWattch: enabling energy optimizations in GPGPUs

paper_content:

General-purpose GPUs (GPGPUs) are becoming prevalent in mainstream computing, and performance per watt has emerged as a more crucial evaluation metric than peak performance. As such, GPU architects require robust tools that will enable them to quickly explore new ways to optimize GPGPUs for energy efficiency. We propose a new GPGPU power model that is configurable, capable of cycle-level calculations, and carefully validated against real hardware measurements. To achieve configurability, we use a bottom-up methodology and abstract parameters from the microarchitectural components as the model's inputs. We developed a rigorous suite of 80 microbenchmarks that we use to bound any modeling uncertainties and inaccuracies. The power model is comprehensively validated against measurements of two commercially available GPUs, and the measured error is within 9.9% and 13.4% for the two target GPUs (GTX 480 and Quadro FX5600). The model also accurately tracks the power consumption trend over time. We integrated the power model with the cycle-level simulator GPGPU-Sim and demonstrate the energy savings by utilizing dynamic voltage and frequency scaling (DVFS) and clock gating. Traditional DVFS reduces GPU energy consumption by 14.4% by leveraging within-kernel runtime variations. More finer-grained SM cluster-level DVFS improves the energy savings from 6.6% to 13.6% for those benchmarks that show clustered execution behavior. We also show that clock gating inactive lanes during divergence reduces dynamic power by 11.2%.
---
paper_title: Architecture supported register stash for GPGPU

paper_content:

GPGPU provides abundant hardware resources to support a large number of light-weighted threads. They are organized into blocks and run in warps. All threads of a block must be dispatched to one stream multiprocessor (SM) of GPGPU together. When the remaining resources of an SM cannot support one more block, all threads of the block are held back until former blocks retire from the SM. We found that the register file is prone to be the most limited one among all the resources, especially for SMs with less registers. Meanwhile, we revealed the dynamics of a thread's register requirement: only part of its pre-allocated registers are used for different instructions at run time. This results in considerable register underutilization.We proposed the architecture supported register stash (ASRS). It removes the limitation of registers when dispatching blocks. The hardware registers are allocated at run time according to each instruction's live registers, which can be analyzed statically by a compiler. When the hardware registers cannot meet the requirements of all running warps, some warps are suspended and their registers are reclaimed temporarily. The data in these registers are stashed to memory. On the other hand, if there are spare hardware registers, it will start a new warp or resume a suspended warp after all the warp's stashed register data are loaded from memory. The intra-block synchronization is also taken care of when some of the warps of the same block are not schedulable due to the ASRS.The ASRS alleviates the register underutilization and improves performance without modifying the current programming model or demanding extra effort from the programmers. It also enables an SM with limited registers that cannot even support a single block to execute it. Besides, it helps lower the register file energy consumption and increase the power efficiency. The ASRS achieved speedups of 1.59 and 1.14 when the registers of each SM are limited to 8K and 16K respectively with an insignificant overhead. The speedups compared with the infinite register files are 0.84 and 0.98 with 8K and 16K registers respectively. Compared with the baseline 32K register file, the ASRS decreases the 8K and 16K register file energy consumption to 66.5% and 75.8% respectively. Their power efficiencies (in ratio of performance and power) are increased to 1.29x?and 1.31x?respectively. Register requirement of GPGPU varies among different kernels and during run time.Register file (RF) capacity limits the schedulable warps and performance.Reducing RF capacity lowers energy consumption and area.We proposed a method to support more warps with limited registers.It gains significant speedup and a higher energy efficiency with a smaller RF.
---
paper_title: Demystifying GPU microarchitecture through microbenchmarking

paper_content:

Graphics processors (GPU) offer the promise of more than an order of magnitude speedup over conventional processors for certain non-graphics computations. Because the GPU is often presented as a C-like abstraction (e.g., Nvidia's CUDA), little is known about the characteristics of the GPU's architecture beyond what the manufacturer has documented. This work develops a microbechmark suite and measures the CUDA-visible architectural characteristics of the Nvidia GT200 (GTX280) GPU. Various undisclosed characteristics of the processing elements and the memory hierarchies are measured. This analysis exposes undocumented features that impact program performance and correctness. These measurements can be useful for improving performance optimization, analysis, and modeling on this architecture and offer additional insight on the decisions made in developing this GPU.
---
paper_title: Power-performance co-optimization of throughput core architecture using resistive memory

paper_content:

Massively parallel computing on throughput computers such as GPUs requires myriad memory accesses to register files, on-chip scratchpad, caches, and off-chip DRAM. Unlike CPUs, these processors have a large register file and on-chip scratchpad memory, which consume a significant portion of compute core power (35%-45%). In this paper, we introduce novel throughput architecture by integrating resistive memory (Spin Transfer Torque RAM) inside the compute core, which reduces leakage significantly, but introduces write power overhead and longer write latencies in GPU shared memory and register file accesses. We enhance the compute core by introducing register file organization with differential memory update mechanism to remove update redundancy during write operations. Furthermore, using merged register-write-mechanism and write-back buffer, we coalesce multithreaded GPU register write accesses to save write energy. In addition, we introduce hybrid shared memory design using SRAM and STT-MRAM that provides significant leakage/dynamic power savings without affecting performance. On average, across 23 GPGPU/graphics workloads, our schemes save 46% dynamic power due to register access (83% leakage power saving) with negligible performance degradation. On average, hybrid shared memory provides 10% reduction in dynamic power with maximum 1.6× performance improvement for the current workloads at no additional area overhead.
---
paper_title: How a single chip causes massive power bills GPUSimPow: A GPGPU power simulator

paper_content:

Modern GPUs are true power houses in every meaning of the word: While they offer general-purpose (GPGPU) compute performance an order of magnitude higher than that of conventional CPUs, they have also been rapidly approaching the infamous “power wall”, as a single chip sometimes consumes more than 300W. Thus, the design space of GPGPU microarchitecture has been extended by another dimension: power. While GPU researchers have previously relied on cycle-accurate simulators for estimating performance during design cycles, there are no simulation tools that include power as well. To mitigate this issue, we introduce the GPUSimPow power estimation framework for GPGPUs consisting of both analytical and empirical models for regular and irregular hardware components. To validate this framework, we build a custom measurement setup to obtain power numbers from real graphics cards. An evaluation on a set of well-known benchmarks reveals an average relative error of 11.7% between simulated and hardware power for GT240 and an average relative error of 10.8% for GTX580. The simulator has been made available to the public [1].
---
paper_title: Power Modeling for GPU Architectures Using McPAT

paper_content:

Graphics Processing Units (GPUs) are very popular for both graphics and general-purpose applications. Since GPUs operate many processing units and manage multiple levels of memory hierarchy, they consume a significant amount of power. Although several power models for CPUs are available, the power consumption of GPUs has not been studied much yet. In this article we develop a new power model for GPUs by utilizing McPAT, a CPU power tool. We generate initial power model data from McPAT with a detailed GPU configuration, and then adjust the models by comparing them with empirical data. We use the NVIDIA's Fermi architecture for building the power model, and our model estimates the GPU power consumption with an average error of 7.7p and 12.8p for the microbenchmarks and Merge benchmarks, respectively.
---
paper_title: A Survey of CPU-GPU Heterogeneous Computing Techniques

paper_content:

As both CPUs and GPUs become employed in a wide range of applications, it has been acknowledged that both of these Processing Units (PUs) have their unique features and strengths and hence, CPU-GPU collaboration is inevitable to achieve high-performance computing. This has motivated a significant amount of research on heterogeneous computing techniques, along with the design of CPU-GPU fused chips and petascale heterogeneous supercomputers. In this article, we survey Heterogeneous Computing Techniques (HCTs) such as workload partitioning that enable utilizing both CPUs and GPUs to improve performance and/or energy efficiency. We review heterogeneous computing approaches at runtime, algorithm, programming, compiler, and application levels. Further, we review both discrete and fused CPU-GPU systems and discuss benchmark suites designed for evaluating Heterogeneous Computing Systems (HCSs). We believe that this article will provide insights into the workings and scope of applications of HCTs to researchers and motivate them to further harness the computational powers of CPUs and GPUs to achieve the goal of exascale performance.
---
paper_title: Reducing shift penalty in domain wall memory through register locality

paper_content:

General-purpose graphics processing units (GPGPUs) have the ability to execute hundreds to thousands of threads simultaneously. Extreme multithreading requires a large register file to hold state of executing threads and facilitate context switching. As feature size reduces, power consumption in the large register file becomes a major concern. In this work, we exploit Domain Wall Memory (DWM) which is a spin-based memory to reduce power consumption in register file. DWM is a promising technology and offers non-volatility, high energy efficiency, and high density by storing several bits into the domains of a ferromagnetic wire. However, despite of favourable properties of DMW over SRAM technology, DWM poses a unique challenge that the bits must be accessed serially through shift operations, leading to variable and potentially higher access latencies. To address this challenge, we propose a new predictive shift policy. In this policy, we exploit register locality across threads and predict source and destination operands of instructions. We record history of registers accessed by instructions and shift magnetic domains of DWM tracks for subsequent instructions, speculatively. Over a wide range of applications from NVIDIA CUDA SDK, ISPASS, and Rodinia, our predictive scheme achieves dramatic energy saving over an SRAM register file while changing performance negligibly.
---
paper_title: An Adaptive Thread Scheduling Mechanism With Low-Power Register File for Mobile GPUs

paper_content:

In response to the remarkable increase in 3D applications in consumer electronics devices in recent years, graphics processing units (GPUs) have become widely available on mobile devices. These GPUs typically use hardware multithreaded shaders to improve their throughputs for real-time rendering, but they depend on duplicate register files to maintain the context of each hardware thread, increasing power consumption. However, the register usage of shading programs is often relatively low, which causes many registers to remain unused, thus wasting power. Long latency memory operations can also consume unnecessary power to activate registers. This study proposes a low-power register file with multiple power modes to reduce the power consumption of the register file. This study also presents an adaptive thread scheduling mechanism to achieve a tradeoff between the power consumption of the register file and frames per second (FPS). Results show that the average performance degradation from the proposed low-power register file is only 0.62%. The proposed adaptive thread scheduling has average under prediction ratio of 3.32%. The leakage reduction of the proposed low-power register file is 74.80%. This reduction can be improved to 81.49%, 82.22%, and 84.28% with adaptive thread scheduling at frame rates of 30, 25, and 20, respectively.
---
paper_title: Exploration of GPGPU register file architecture using domain-wall-shift-write based racetrack memory

paper_content:

SRAM based register file (RF) is one of the major factors limiting the scaling of GPGPU. In this work, we propose to use the emerging nonvolatile domain-wall-shift-write based racetrack memory (DWSW-RM) to implement a power-efficient GPGPU RF, of which the power consumption is substantially reduced. A holistic technology set is developed to minimize the high access cost of DWSW-RW caused by the sequential access mechanism. Experiment results show that our proposed techniques can improve the GPGPU performance by 4.6% compared to the baseline with SRAM based RF. The RF energy efficiency is also significantly improved by 2.45×.
---
paper_title: Power-performance co-optimization of throughput core architecture using resistive memory

paper_content:

Massively parallel computing on throughput computers such as GPUs requires myriad memory accesses to register files, on-chip scratchpad, caches, and off-chip DRAM. Unlike CPUs, these processors have a large register file and on-chip scratchpad memory, which consume a significant portion of compute core power (35%-45%). In this paper, we introduce novel throughput architecture by integrating resistive memory (Spin Transfer Torque RAM) inside the compute core, which reduces leakage significantly, but introduces write power overhead and longer write latencies in GPU shared memory and register file accesses. We enhance the compute core by introducing register file organization with differential memory update mechanism to remove update redundancy during write operations. Furthermore, using merged register-write-mechanism and write-back buffer, we coalesce multithreaded GPU register write accesses to save write energy. In addition, we introduce hybrid shared memory design using SRAM and STT-MRAM that provides significant leakage/dynamic power savings without affecting performance. On average, across 23 GPGPU/graphics workloads, our schemes save 46% dynamic power due to register access (83% leakage power saving) with negligible performance degradation. On average, hybrid shared memory provides 10% reduction in dynamic power with maximum 1.6× performance improvement for the current workloads at no additional area overhead.
---
paper_title: How a single chip causes massive power bills GPUSimPow: A GPGPU power simulator

paper_content:

Modern GPUs are true power houses in every meaning of the word: While they offer general-purpose (GPGPU) compute performance an order of magnitude higher than that of conventional CPUs, they have also been rapidly approaching the infamous “power wall”, as a single chip sometimes consumes more than 300W. Thus, the design space of GPGPU microarchitecture has been extended by another dimension: power. While GPU researchers have previously relied on cycle-accurate simulators for estimating performance during design cycles, there are no simulation tools that include power as well. To mitigate this issue, we introduce the GPUSimPow power estimation framework for GPGPUs consisting of both analytical and empirical models for regular and irregular hardware components. To validate this framework, we build a custom measurement setup to obtain power numbers from real graphics cards. An evaluation on a set of well-known benchmarks reveals an average relative error of 11.7% between simulated and hardware power for GT240 and an average relative error of 10.8% for GTX580. The simulator has been made available to the public [1].
---
paper_title: Power Modeling for GPU Architectures Using McPAT

paper_content:

Graphics Processing Units (GPUs) are very popular for both graphics and general-purpose applications. Since GPUs operate many processing units and manage multiple levels of memory hierarchy, they consume a significant amount of power. Although several power models for CPUs are available, the power consumption of GPUs has not been studied much yet. In this article we develop a new power model for GPUs by utilizing McPAT, a CPU power tool. We generate initial power model data from McPAT with a detailed GPU configuration, and then adjust the models by comparing them with empirical data. We use the NVIDIA's Fermi architecture for building the power model, and our model estimates the GPU power consumption with an average error of 7.7p and 12.8p for the microbenchmarks and Merge benchmarks, respectively.
---
paper_title: GPUWattch: enabling energy optimizations in GPGPUs

paper_content:

General-purpose GPUs (GPGPUs) are becoming prevalent in mainstream computing, and performance per watt has emerged as a more crucial evaluation metric than peak performance. As such, GPU architects require robust tools that will enable them to quickly explore new ways to optimize GPGPUs for energy efficiency. We propose a new GPGPU power model that is configurable, capable of cycle-level calculations, and carefully validated against real hardware measurements. To achieve configurability, we use a bottom-up methodology and abstract parameters from the microarchitectural components as the model's inputs. We developed a rigorous suite of 80 microbenchmarks that we use to bound any modeling uncertainties and inaccuracies. The power model is comprehensively validated against measurements of two commercially available GPUs, and the measured error is within 9.9% and 13.4% for the two target GPUs (GTX 480 and Quadro FX5600). The model also accurately tracks the power consumption trend over time. We integrated the power model with the cycle-level simulator GPGPU-Sim and demonstrate the energy savings by utilizing dynamic voltage and frequency scaling (DVFS) and clock gating. Traditional DVFS reduces GPU energy consumption by 14.4% by leveraging within-kernel runtime variations. More finer-grained SM cluster-level DVFS improves the energy savings from 6.6% to 13.6% for those benchmarks that show clustered execution behavior. We also show that clock gating inactive lanes during divergence reduces dynamic power by 11.2%.
---
paper_title: Demystifying GPU microarchitecture through microbenchmarking

paper_content:

Graphics processors (GPU) offer the promise of more than an order of magnitude speedup over conventional processors for certain non-graphics computations. Because the GPU is often presented as a C-like abstraction (e.g., Nvidia's CUDA), little is known about the characteristics of the GPU's architecture beyond what the manufacturer has documented. This work develops a microbechmark suite and measures the CUDA-visible architectural characteristics of the Nvidia GT200 (GTX280) GPU. Various undisclosed characteristics of the processing elements and the memory hierarchies are measured. This analysis exposes undocumented features that impact program performance and correctness. These measurements can be useful for improving performance optimization, analysis, and modeling on this architecture and offer additional insight on the decisions made in developing this GPU.
---
paper_title: Bank stealing for conflict mitigation in GPGPU Register File

paper_content:

Modern General Purpose Graphic Processing Unit (GPGPU) demands a large Register File (RF), which is typically organized into multiple banks to support the massive parallelism. Although heavy banking benefits RF throughput, its associated area and energy costs with diminishing performance gains greatly limit future RF s-caling. In this paper, we propose an improved RF design with a bank stealing technique, which enables a high RF throughput with compact area. By deeply investigating the GPGPU microarchitecture, we identify the deficiency in the state-of-the-art RF designs as the bank conflict problem, while the majority of conflicts can be eliminated leveraging the fact that the highly-banked RF oftentimes experiences under-utilization. This is especially true in GPGPU where multiple ready warps are available at the scheduling stage with their operands to be wisely coordinated. Our lightweight bank stealing technique can opportunistically fill the idle banks for better operand service, and the average GPGPU performance can be improved under smaller energy budget with significant area saving, which makes it promising for sustainable RF scaling.
---
paper_title: SRAM-DRAM hybrid memory with applications to efficient register files in fine-grained multi-threading

paper_content:

Large register files are common in highly multi-threaded architectures such as GPUs. This paper presents a hybrid memory design that tightly integrates embedded DRAM into SRAM cells with a main application to reducing area and power consumption of multi-threaded register files. In the hybrid memory, each SRAM cell is augmented with multiple DRAM cells so that multiple bits can be stored in each cell. This configuration results in significant area and energy savings compared to the SRAM array with the same capacity due to compact DRAM cells. On other hand, the hybrid memory requires explicit data movements in order to access DRAM contexts. In order to minimize context switching impact, we introduce write-back buffers, background context switching, and context-aware thread scheduling, to the processor pipeline and the scheduler. Circuit and architecture simulations of GPU benchmarks suites show significant savings in register file area (38%) and energy (68%) over the traditional SRAM implementation, with minimal (1.4%) performance loss.
---
paper_title: ContextPreRF: Enhancing the Performance and Energy of GPUs With Nonuniform Register Access

paper_content:

Register files are a key data storage unit that impacts instruction throughput for graphics processing units (GPUs). Typically, GPU register files are quite large to accommodate many concurrent threads and are implemented using the same SRAM technology as the on-chip cache. We propose contextrf, a new register file architecture that efficiently leverages register files with nonuniform access characteristics, including hybrid SRAM/DRAM (S/D) and spintronic domain-wall memories (DWMs). Contextrf allows greater-capacity register files to be implemented in the same area within the GPU, with reduced power consumption. We also propose contextPreRF, a hardware preswitch scheme to hide switching delays—as soon as a register request is queued, the nonuniform access memories containing the corresponding register are sent a preemptive switch request. Thus, our scheme transparently hides the penalties of switching between register contexts. After replacing the register file SRAM with S/D, we can reduce energy by 37%, with a 1.4% average performance drop. Employing DWM, we reduce register file energy by 74%, with a 0.4% average performance penalty. For the denser DWM, we model converting the saved area into additional registers, cache, and shared memory—this improves performance by 13.5% over the baseline SRAM register file.
---
paper_title: Soft-error reliability and power co-optimization for GPGPUs register file using resistive memory

paper_content:

The increasing adoption of graphics processing units (GPUs) for high-performance computing raises the reliability challenge, which is generally ignored in traditional GPUs. GPUs usually support thousands of parallel threads and require a sizable register file. Such large register file is highly susceptible to soft errors and power-hungry. Although ECC has been adopted to register file in modern GPUs, it causes considerable power overhead, which further increases the power stress. Thus, an energy-efficient soft-error protection mechanism is more desirable. Besides its extremely low leakage power consumption, resistive memory (e.g. spin-transfer torque RAM) is also immune to the radiation induced soft errors due to its magnetic field based storage. In this paper, we propose to LEverage reSistive memory to enhance the Soft-error robustness and reduce the power consumption (LESS) of registers in the General-Purpose computing on GPUs (GPGPUs). Since resistive memory experiences longer write latency compared to SRAM, we explore the unique characteristics of GPGPU applications to obtain the win-win gains: achieving the near-full soft-error protection for the register file, and meanwhile substantially reducing the energy consumption with negligible performance loss. Our experimental results show that LESS is able to mitigate the registers soft-error vulnerability by 86% and achieve 60% energy savings with negligible (e.g. 4%) performance loss.
---
paper_title: A Survey of Methods for Analyzing and Improving GPU Energy Efficiency

paper_content:

Recent years have witnessed phenomenal growth in the computational capabilities and applications of GPUs. However, this trend has also led to a dramatic increase in their power consumption. This article surveys research works on analyzing and improving energy efficiency of GPUs. It also provides a classification of these techniques on the basis of their main research idea. Further, it attempts to synthesize research works that compare the energy efficiency of GPUs with other computing systems (e.g., FPGAs and CPUs). The aim of this survey is to provide researchers with knowledge of the state of the art in GPU power management and motivate them to architect highly energy-efficient GPUs of tomorrow.
---
paper_title: ARGO: aging-aware GPGPU register file allocation

paper_content:

State-of-the-art general-purpose graphic processing units (GPGPUs) implemented in nanoscale CMOS technologies offer very high computational throughput for highly-parallel applications using hundreds of integrated on-chip resources. These resources are stressed during application execution, subjecting them to degradation mechanisms such as negative bias temperature instability (NBTI) that adversely affect their reliability. To support highly parallel execution, GPGPUs contain large register files (RFs) that are among the most highly stressed GPGPU components; however we observe heavy underutilization of RFs (on average only 46%) for typical general-purpose kernels. We present ARGO, an A ging-awa R e G PGPU RF all O cator that opportunistically exploits this RF underutilization by distributing the stress throughout RF. ARGO achieves proper leveling of RF banks through deliberated power-gating of stressful banks. We demonstrate our technique on the AMD Evergreen GPGPU architecture and show that ARGO improves the NBTI-induced threshold voltage degradation by up to 43% (on average 27%), that yields improving RFs static noise margin up to 46% (on average 30%). Furthermore, we estimate a simultaneous reduction in leakage power of 54% by providing sleep states for unused banks.
---
paper_title: Spare register aware prefetching for graph algorithms on GPUs

paper_content:

More and more graph algorithms are being GPU enabled. Graph algorithm implementations on GPUs have irregular control flow and are memory-intensive with many irregular/data-dependent memory accesses. Due to these factors graph algorithms on GPUs have low execution efficiency. In this work we propose a mechanism to improve the execution efficiency of graph algorithms by improving their memory access latency tolerance. We propose a mechanism for prefetching data for load pairs that have one load dependent on the other - such pairs are common in graph algorithms. Our mechanism detects the target loads in hardware and injects instructions into the pipeline to prefetch data into spare registers that are not being used by any active threads. By prefetching data into registers, early eviction of prefetched data can be eliminated. We also propose a mechanism that uses the compiler to identify the target loads. Our mechanism improves performance over no prefetching by 10% on average and upto 51% for nine memory intensive graph algorithm kernels.
---
paper_title: A Survey of Techniques for Modeling and Improving Reliability of Computing Systems

paper_content:

Recent trends of aggressive technology scaling have greatly exacerbated the occurrences and impact of faults in computing systems. This has made ‘reliability’ a first-order design constraint. To address the challenges of reliability, several techniques have been proposed. This paper provides a survey of architectural techniques for improving resilience of computing systems. We especially focus on techniques proposed for microarchitectural components, such as processor registers, functional units, cache and main memory etc. In addition, we discuss techniques proposed for non-volatile memory, GPUs and 3D-stacked processors. To underscore the similarities and differences of the techniques, we classify them based on their key characteristics. We also review the metrics proposed to quantify vulnerability of processor structures. We believe that this survey will help researchers, system-architects and processor designers in gaining insights into the techniques for improving reliability of computing systems.
---
paper_title: Mitigating the Susceptibility of GPGPUs Register File to Process Variations

paper_content:

As technology keeps scaling down at nano-scale, the increasing process variations (PV) induce significant delay variations and limit the maximum clock frequency in GPGPUs (general-purpose computing on graphics processing units). Each computing core (i.e. streaming multiprocessor) in GPGPUs supports thousands of simultaneously active threads, and requires a large register file. Such a sizeable register file is very sensitive to process variations, and becomes one of the major units in determining the core frequency. In this study, we first develop a novel mechanism that classifies registers into fast and slow categories in the highly-banked register architecture to maximize the frequency improvement. We then leverage the unique features in GPGPU applications to effectively tolerate the extra access delay to the slow registers. Our experimental results show that our proposed techniques are able to significantly optimize GPGPUs performance under process variations.
---
paper_title: ARGO: aging-aware GPGPU register file allocation

paper_content:

State-of-the-art general-purpose graphic processing units (GPGPUs) implemented in nanoscale CMOS technologies offer very high computational throughput for highly-parallel applications using hundreds of integrated on-chip resources. These resources are stressed during application execution, subjecting them to degradation mechanisms such as negative bias temperature instability (NBTI) that adversely affect their reliability. To support highly parallel execution, GPGPUs contain large register files (RFs) that are among the most highly stressed GPGPU components; however we observe heavy underutilization of RFs (on average only 46%) for typical general-purpose kernels. We present ARGO, an A ging-awa R e G PGPU RF all O cator that opportunistically exploits this RF underutilization by distributing the stress throughout RF. ARGO achieves proper leveling of RF banks through deliberated power-gating of stressful banks. We demonstrate our technique on the AMD Evergreen GPGPU architecture and show that ARGO improves the NBTI-induced threshold voltage degradation by up to 43% (on average 27%), that yields improving RFs static noise margin up to 46% (on average 30%). Furthermore, we estimate a simultaneous reduction in leakage power of 54% by providing sleep states for unused banks.
---
paper_title: GPUWattch: enabling energy optimizations in GPGPUs

paper_content:

General-purpose GPUs (GPGPUs) are becoming prevalent in mainstream computing, and performance per watt has emerged as a more crucial evaluation metric than peak performance. As such, GPU architects require robust tools that will enable them to quickly explore new ways to optimize GPGPUs for energy efficiency. We propose a new GPGPU power model that is configurable, capable of cycle-level calculations, and carefully validated against real hardware measurements. To achieve configurability, we use a bottom-up methodology and abstract parameters from the microarchitectural components as the model's inputs. We developed a rigorous suite of 80 microbenchmarks that we use to bound any modeling uncertainties and inaccuracies. The power model is comprehensively validated against measurements of two commercially available GPUs, and the measured error is within 9.9% and 13.4% for the two target GPUs (GTX 480 and Quadro FX5600). The model also accurately tracks the power consumption trend over time. We integrated the power model with the cycle-level simulator GPGPU-Sim and demonstrate the energy savings by utilizing dynamic voltage and frequency scaling (DVFS) and clock gating. Traditional DVFS reduces GPU energy consumption by 14.4% by leveraging within-kernel runtime variations. More finer-grained SM cluster-level DVFS improves the energy savings from 6.6% to 13.6% for those benchmarks that show clustered execution behavior. We also show that clock gating inactive lanes during divergence reduces dynamic power by 11.2%.
---
paper_title: Register Caching for Stencil Computations on GPUs

paper_content:

For most applications, taking full advantage of the memory system is key to achieving good performance on GPUs. In this paper, we introduce register caching, a novel idea where registers of multiple threads are combined and used as a shared, last level, manually managed cache for the contributing threads. This method is enabled by the shuffle instruction recently introduced in Nvidia's Kepler GPU architecture, which allows threads in the same warp to exchange data directly, previously only possible by going through shared memory. We evaluate our proposal with a stencil computation benchmark, achieving speedups of up to 2.04, compared to using shared memory on a GTX680 GPU. Stencil computations form the core of many scientific applications, which can therefore benefit from our proposal. Furthermore, our method is not limited to stencil computations, but is applicable to any application with a predictable memory access pattern suitable for manual caching.
---
paper_title: Demystifying GPU microarchitecture through microbenchmarking

paper_content:

Graphics processors (GPU) offer the promise of more than an order of magnitude speedup over conventional processors for certain non-graphics computations. Because the GPU is often presented as a C-like abstraction (e.g., Nvidia's CUDA), little is known about the characteristics of the GPU's architecture beyond what the manufacturer has documented. This work develops a microbechmark suite and measures the CUDA-visible architectural characteristics of the Nvidia GT200 (GTX280) GPU. Various undisclosed characteristics of the processing elements and the memory hierarchies are measured. This analysis exposes undocumented features that impact program performance and correctness. These measurements can be useful for improving performance optimization, analysis, and modeling on this architecture and offer additional insight on the decisions made in developing this GPU.
---
paper_title: An energy-efficient and scalable eDRAM-based register file architecture for GPGPU

paper_content:

The heavily-threaded data processing demands of streaming multiprocessors (SM) in a GPGPU require a large register file (RF). The fast increasing size of the RF makes the area cost and power consumption unaffordable for traditional SRAM designs in the future technologies. In this paper, we propose to use embedded-DRAM (eDRAM) as an alternative in future GPGPUs. Compared with SRAM, eDRAM provides higher density and lower leakage power. However, the limited data retention time in eDRAM poses new challenges. Periodic refresh operations are needed to maintain data integrity. This is exacerbated with the scaling of eDRAM density, process variations and temperature. Unlike conventional CPUs which make use of multi-ported RF, most of the RFs in modern GPGPU are heavily banked but not multi-ported to reduce the hardware cost. This provides a unique opportunity to hide the refresh overhead. We propose two different eDRAM implementations based on 3T1D and 1T1C memory cells. To mitigate the impact of periodic refresh, we propose two novel refresh solutions using bank bubble and bank walk-through. Plus, for the 1T1C RF, we design an interleaved bank organization together with an intelligent warp scheduling strategy to reduce the impact of the destructive reads. The analysis shows that our schemes present better energy efficiency, scalability and variation tolerance than traditional SRAM-based designs.
---
paper_title: A Survey Of Architectural Approaches for Managing Embedded DRAM and Non-Volatile On-Chip Caches

paper_content:

Recent trends of CMOS scaling and increasing number of on-chip cores have led to a large increase in the size of on-chip caches. Since SRAM has low density and consumes large amount of leakage power, its use in designing on-chip caches has become more challenging. To address this issue, researchers are exploring the use of several emerging memory technologies, such as embedded DRAM, spin transfer torque RAM, resistive RAM, phase change RAM and domain wall memory. In this paper, we survey the architectural approaches proposed for designing memory systems and, specifically, caches with these emerging memorytechnologies. To highlight their similarities and differences, we present a classification of these technologies and architectural approaches based on their key characteristics. We also briefly summarize the challenges in using these technologies for architecting caches. We believe that this survey will help the readers gain insights into the emerging memory device technologies, and their potential use in designing future computing systems.
---
paper_title: SRAM-DRAM hybrid memory with applications to efficient register files in fine-grained multi-threading

paper_content:

Large register files are common in highly multi-threaded architectures such as GPUs. This paper presents a hybrid memory design that tightly integrates embedded DRAM into SRAM cells with a main application to reducing area and power consumption of multi-threaded register files. In the hybrid memory, each SRAM cell is augmented with multiple DRAM cells so that multiple bits can be stored in each cell. This configuration results in significant area and energy savings compared to the SRAM array with the same capacity due to compact DRAM cells. On other hand, the hybrid memory requires explicit data movements in order to access DRAM contexts. In order to minimize context switching impact, we introduce write-back buffers, background context switching, and context-aware thread scheduling, to the processor pipeline and the scheduler. Circuit and architecture simulations of GPU benchmarks suites show significant savings in register file area (38%) and energy (68%) over the traditional SRAM implementation, with minimal (1.4%) performance loss.
---
paper_title: Soft-error reliability and power co-optimization for GPGPUs register file using resistive memory

paper_content:

The increasing adoption of graphics processing units (GPUs) for high-performance computing raises the reliability challenge, which is generally ignored in traditional GPUs. GPUs usually support thousands of parallel threads and require a sizable register file. Such large register file is highly susceptible to soft errors and power-hungry. Although ECC has been adopted to register file in modern GPUs, it causes considerable power overhead, which further increases the power stress. Thus, an energy-efficient soft-error protection mechanism is more desirable. Besides its extremely low leakage power consumption, resistive memory (e.g. spin-transfer torque RAM) is also immune to the radiation induced soft errors due to its magnetic field based storage. In this paper, we propose to LEverage reSistive memory to enhance the Soft-error robustness and reduce the power consumption (LESS) of registers in the General-Purpose computing on GPUs (GPGPUs). Since resistive memory experiences longer write latency compared to SRAM, we explore the unique characteristics of GPGPU applications to obtain the win-win gains: achieving the near-full soft-error protection for the register file, and meanwhile substantially reducing the energy consumption with negligible performance loss. Our experimental results show that LESS is able to mitigate the registers soft-error vulnerability by 86% and achieve 60% energy savings with negligible (e.g. 4%) performance loss.
---
paper_title: An energy-efficient and scalable eDRAM-based register file architecture for GPGPU

paper_content:

The heavily-threaded data processing demands of streaming multiprocessors (SM) in a GPGPU require a large register file (RF). The fast increasing size of the RF makes the area cost and power consumption unaffordable for traditional SRAM designs in the future technologies. In this paper, we propose to use embedded-DRAM (eDRAM) as an alternative in future GPGPUs. Compared with SRAM, eDRAM provides higher density and lower leakage power. However, the limited data retention time in eDRAM poses new challenges. Periodic refresh operations are needed to maintain data integrity. This is exacerbated with the scaling of eDRAM density, process variations and temperature. Unlike conventional CPUs which make use of multi-ported RF, most of the RFs in modern GPGPU are heavily banked but not multi-ported to reduce the hardware cost. This provides a unique opportunity to hide the refresh overhead. We propose two different eDRAM implementations based on 3T1D and 1T1C memory cells. To mitigate the impact of periodic refresh, we propose two novel refresh solutions using bank bubble and bank walk-through. Plus, for the 1T1C RF, we design an interleaved bank organization together with an intelligent warp scheduling strategy to reduce the impact of the destructive reads. The analysis shows that our schemes present better energy efficiency, scalability and variation tolerance than traditional SRAM-based designs.
---
paper_title: Reducing shift penalty in domain wall memory through register locality

paper_content:

General-purpose graphics processing units (GPGPUs) have the ability to execute hundreds to thousands of threads simultaneously. Extreme multithreading requires a large register file to hold state of executing threads and facilitate context switching. As feature size reduces, power consumption in the large register file becomes a major concern. In this work, we exploit Domain Wall Memory (DWM) which is a spin-based memory to reduce power consumption in register file. DWM is a promising technology and offers non-volatility, high energy efficiency, and high density by storing several bits into the domains of a ferromagnetic wire. However, despite of favourable properties of DMW over SRAM technology, DWM poses a unique challenge that the bits must be accessed serially through shift operations, leading to variable and potentially higher access latencies. To address this challenge, we propose a new predictive shift policy. In this policy, we exploit register locality across threads and predict source and destination operands of instructions. We record history of registers accessed by instructions and shift magnetic domains of DWM tracks for subsequent instructions, speculatively. Over a wide range of applications from NVIDIA CUDA SDK, ISPASS, and Rodinia, our predictive scheme achieves dramatic energy saving over an SRAM register file while changing performance negligibly.
---
paper_title: SRAM-DRAM hybrid memory with applications to efficient register files in fine-grained multi-threading

paper_content:

Large register files are common in highly multi-threaded architectures such as GPUs. This paper presents a hybrid memory design that tightly integrates embedded DRAM into SRAM cells with a main application to reducing area and power consumption of multi-threaded register files. In the hybrid memory, each SRAM cell is augmented with multiple DRAM cells so that multiple bits can be stored in each cell. This configuration results in significant area and energy savings compared to the SRAM array with the same capacity due to compact DRAM cells. On other hand, the hybrid memory requires explicit data movements in order to access DRAM contexts. In order to minimize context switching impact, we introduce write-back buffers, background context switching, and context-aware thread scheduling, to the processor pipeline and the scheduler. Circuit and architecture simulations of GPU benchmarks suites show significant savings in register file area (38%) and energy (68%) over the traditional SRAM implementation, with minimal (1.4%) performance loss.
---
paper_title: An eDRAM-Based Approximate Register File for GPUs

paper_content:

GPUs require large register files for fast context switching. This paper presents a high-density and energy-efficient approximate register file architecture based on embedded DRAM with lowered refresh rate to the low-order bits of register entries.
---
paper_title: Architecture supported register stash for GPGPU

paper_content:

GPGPU provides abundant hardware resources to support a large number of light-weighted threads. They are organized into blocks and run in warps. All threads of a block must be dispatched to one stream multiprocessor (SM) of GPGPU together. When the remaining resources of an SM cannot support one more block, all threads of the block are held back until former blocks retire from the SM. We found that the register file is prone to be the most limited one among all the resources, especially for SMs with less registers. Meanwhile, we revealed the dynamics of a thread's register requirement: only part of its pre-allocated registers are used for different instructions at run time. This results in considerable register underutilization.We proposed the architecture supported register stash (ASRS). It removes the limitation of registers when dispatching blocks. The hardware registers are allocated at run time according to each instruction's live registers, which can be analyzed statically by a compiler. When the hardware registers cannot meet the requirements of all running warps, some warps are suspended and their registers are reclaimed temporarily. The data in these registers are stashed to memory. On the other hand, if there are spare hardware registers, it will start a new warp or resume a suspended warp after all the warp's stashed register data are loaded from memory. The intra-block synchronization is also taken care of when some of the warps of the same block are not schedulable due to the ASRS.The ASRS alleviates the register underutilization and improves performance without modifying the current programming model or demanding extra effort from the programmers. It also enables an SM with limited registers that cannot even support a single block to execute it. Besides, it helps lower the register file energy consumption and increase the power efficiency. The ASRS achieved speedups of 1.59 and 1.14 when the registers of each SM are limited to 8K and 16K respectively with an insignificant overhead. The speedups compared with the infinite register files are 0.84 and 0.98 with 8K and 16K registers respectively. Compared with the baseline 32K register file, the ASRS decreases the 8K and 16K register file energy consumption to 66.5% and 75.8% respectively. Their power efficiencies (in ratio of performance and power) are increased to 1.29x?and 1.31x?respectively. Register requirement of GPGPU varies among different kernels and during run time.Register file (RF) capacity limits the schedulable warps and performance.Reducing RF capacity lowers energy consumption and area.We proposed a method to support more warps with limited registers.It gains significant speedup and a higher energy efficiency with a smaller RF.
---
paper_title: Mitigating the Susceptibility of GPGPUs Register File to Process Variations

paper_content:

As technology keeps scaling down at nano-scale, the increasing process variations (PV) induce significant delay variations and limit the maximum clock frequency in GPGPUs (general-purpose computing on graphics processing units). Each computing core (i.e. streaming multiprocessor) in GPGPUs supports thousands of simultaneously active threads, and requires a large register file. Such a sizeable register file is very sensitive to process variations, and becomes one of the major units in determining the core frequency. In this study, we first develop a novel mechanism that classifies registers into fast and slow categories in the highly-banked register architecture to maximize the frequency improvement. We then leverage the unique features in GPGPU applications to effectively tolerate the extra access delay to the slow registers. Our experimental results show that our proposed techniques are able to significantly optimize GPGPUs performance under process variations.
---
paper_title: Energy-efficient mechanisms for managing thread context in throughput processors

paper_content:

Modern graphics processing units (GPUs) use a large number of hardware threads to hide both function unit and memory access latency. Extreme multithreading requires a complicated thread scheduler as well as a large register file, which is expensive to access both in terms of energy and latency. We present two complementary techniques for reducing energy on massively-threaded processors such as GPUs. First, we examine register file caching to replace accesses to the large main register file with accesses to a smaller structure containing the immediate register working set of active threads. Second, we investigate a two-level thread scheduler that maintains a small set of active threads to hide ALU and local memory access latency and a larger set of pending threads to hide main memory latency. Combined with register file caching, a two-level thread scheduler provides a further reduction in energy by limiting the allocation of temporary register cache resources to only the currently active subset of threads. We show that on average, across a variety of real world graphics and compute workloads, a 6-entry per-thread register file cache reduces the number of reads and writes to the main register file by 50% and 59% respectively. We further show that the active thread count can be reduced by a factor of 4 with minimal impact on performance, resulting in a 36% reduction of register file energy.
---
paper_title: GPGPU Register File Management by Hardware Co-operated Register Reallocation

paper_content:

To support massive parallel threads context, GPGPUs use a huge register file. Due to their size, register file is one of the most power hungry logic in GPGPU. However, the current trends indicate that GPGPU register file size will continue to get even bigger as the demand for higher single instruction multiple thread (SIMT) parallelism increases, particularly in high performance application domain. In order to reduce power consumption demand, in this work, we exploit a fundamental observation that fairly high portion of the register spaces are unnecessarily allocated and burn power due to the fact that the registers are considered as private resource for each warp. For example, even though a register’s value is no longer used by any instruction, the register space should be occupied by the warp during the program execution. In GPGPU, as single register usage in a code context leads to thousands of register space allocation, the power and space overhead due to any wasted register is significant. Instead, we propose to share the register file across warps. In our proposed allocation, a register is released from its physical register space immediately after its last use. Then, the released register space is reassigned to another warp’s register. The compile time register lifetime analysis information is used for providing hint to the hardware about each register’s release point. To enable this register reassignment, we propose a light weight register renaming in hardware. By releasing registers and reusing them across warps, we can reduce the demand for register file size on average by 30% compared with the optimally compiled applications. The reduced live register space leads to an average of 23% and 26% static power saving over a basic sub-array level and individual register level power gating.
---
paper_title: An efficient STT-RAM-based register file in GPU architectures

paper_content:

Modern GPGPUs employ a large register file (RF) to efficiently process heavily parallel threads in single instruction multiple thread (SIMT) fashion. The up-scaling of RF capacity, however, is greatly constrained by large cell area and high leakage power consumption of SRAM implementation. In this work, we propose a novel GPU RF design based on the emerging multi-level cell (MLC) spin-transfer torque RAM (STT-RAM) technology. Compared to SRAM, MLC STT-RAM (or MLC-STT) has much smaller cell area and almost zero standby power due to its non- volatility. Moreover, by leveraging the asymmetric performance of the soft and the hard bits of a MLC-STT cell, we propose a remapping strategy to perform a flexible tradeoff between the access time and the capacity of the RF based on run-time access patterns. A novel rescheduling scheme is also developed to minimize the waiting time of the issued warps to access register banks. Experimental results over ISPASS2009 and CUDA benchmarks show that on average, our proposed MLC-STT RF can achieve 3.28% performance improvement, 9.48% energy reduction, and 38.9% energy efficiency enhancement compared to conventional SRAM-based design.
---
paper_title: Exploration of GPGPU register file architecture using domain-wall-shift-write based racetrack memory

paper_content:

SRAM based register file (RF) is one of the major factors limiting the scaling of GPGPU. In this work, we propose to use the emerging nonvolatile domain-wall-shift-write based racetrack memory (DWSW-RM) to implement a power-efficient GPGPU RF, of which the power consumption is substantially reduced. A holistic technology set is developed to minimize the high access cost of DWSW-RW caused by the sequential access mechanism. Experiment results show that our proposed techniques can improve the GPGPU performance by 4.6% compared to the baseline with SRAM based RF. The RF energy efficiency is also significantly improved by 2.45×.
---
paper_title: A Write-Aware STTRAM-Based Register File Architecture for GPGPU

paper_content:

The massively parallel processing capacity of GPGPUs requires a large register file (RF), and its size keeps increasing to support more concurrent threads from generation to generation. Using traditional SRAM-based RFs, there are concerns in both area cost and energy consumption, and soon they will become unrealistic. In this work, we analyze the feasibility of using STTRAM-based RF designs, which have benefits in terms of smaller silicon area and zero standby leakage power. However, STTRAM long write latency and high write energy bring new challenges. Therefore, we propose a write-aware STTRAM-based RF architecture (WarRF), which contains two techniques: Split Bank Write modifies the arbitrator design to increase the parallelism of read and write accesses in the same bank; Write Pool reduces the number of repeated write accesses to RFs. Our experiment shows that the performance of STTRAM-based RF is improved by 13p and up to 23p after adopting WarRF. In addition, the energy consumption is reduced by 38p on average compared to SRAM-based RFs.
---
paper_title: Hybrid CMOS-TFET based register files for energy-efficient GPGPUs

paper_content:

State-of-the-art General-Purpose computing on Graphics Processing Unit (GPGPU) is facing severe power challenge due to the increasing number of cores placed on a chip with decreasing feature size. In order to hide the long latency operations, GPGPU employs the fine-grained multi-threading among numerous active threads, leading to the sizeable register files with massive power consumption. Exploring the optimal power savings in register files becomes the critical and first step towards the energy-efficient GPGPUs. The conventional method to reduce dynamic power consumption is the supply voltage scaling, and the inter-bank tunneling FETs (TFETs) are the promising candidates compared to CMOS for low voltage operations regarding to both leakage and performance. However, always executing at the low voltage (so that low frequency) will result in significant performance degradation. In this study, we propose the hybrid CMOS-TFET based register files. To optimize the register power consumption, we allocate TFET-based registers to threads whose execution progress can be delayed to some degree to avoid the memory contentions with other threads, and the CMOS-based registers are still used for threads requiring normal execution speed. Our experimental results show that the proposed technique achieves 30% energy (including both dynamic and leakage) reduction in register files with little performance degradation compared to the baseline case equipped with naive power optimization technique.
---
paper_title: Power-performance co-optimization of throughput core architecture using resistive memory

paper_content:

Massively parallel computing on throughput computers such as GPUs requires myriad memory accesses to register files, on-chip scratchpad, caches, and off-chip DRAM. Unlike CPUs, these processors have a large register file and on-chip scratchpad memory, which consume a significant portion of compute core power (35%-45%). In this paper, we introduce novel throughput architecture by integrating resistive memory (Spin Transfer Torque RAM) inside the compute core, which reduces leakage significantly, but introduces write power overhead and longer write latencies in GPU shared memory and register file accesses. We enhance the compute core by introducing register file organization with differential memory update mechanism to remove update redundancy during write operations. Furthermore, using merged register-write-mechanism and write-back buffer, we coalesce multithreaded GPU register write accesses to save write energy. In addition, we introduce hybrid shared memory design using SRAM and STT-MRAM that provides significant leakage/dynamic power savings without affecting performance. On average, across 23 GPGPU/graphics workloads, our schemes save 46% dynamic power due to register access (83% leakage power saving) with negligible performance degradation. On average, hybrid shared memory provides 10% reduction in dynamic power with maximum 1.6× performance improvement for the current workloads at no additional area overhead.
---
paper_title: Compiler assisted dynamic register file in GPGPU

paper_content:

The large Register File (RF) in General Purpose Graphic Processing Units (GPGPUs) demands tremendous chip area and energy consumption. For a sustainable growth of the size of RF in future GPGPUs, emerging on-chip memory technologies such as embedded-DRAM (eDRAM) have been proposed to replace the conventional SRAM for higher density and lower leakage but with the possible penalty from the periodic refresh operations. This paper explicitly shows that the refresh penalty can be effectively mitigated by leveraging the uniqueness of GPGPU operations. A compiler assisted refresh rescheduling policy can greatly reduce the refresh overhead for maintaining the correctness of the RF operations. The proposed scheme adequately exploits the features in both architecture and compilation, and delivers comparable performance to the SRAM counterpart. At the same time, the energy savings via the removal of large SRAM leakage well compensate for the additional refresh energy. This study promotes the eDRAM-based RF as a promising alternative that enables larger capacity and better power efficiency for future GPGPUs.
---
paper_title: A STT-RAM-based low-power hybrid register file for GPGPUs

paper_content:

Recently, general-purpose graphics processing units (GPGPUs) have been widely used to accelerate computing in various applications. To store the contexts of thousands of concurrent threads on a GPU, a large static random-access memory (SRAM)-based register file is employed. Due to high leakage power of SRAM, the register file consumes 20% to 40% of the total GPU power consumption. Thus, hybrid memory system, which combines SRAM and the emerging non-volatile memory (NVM), has been employed for register file design on GPUs. Although it has shown strong potential to alleviate the power issue of GPUs, existing hybrid memory solutions might not exploit the intrinsic feature of GPU register file. By leveraging the warp schedule on GPU, this paper proposes a hybrid register architecture which consists of a NVM-based register file and mixed SRAM-based write buffers with a warp-aware write back strategy. Simulation results show that our design can eliminate 64% of write accesses to NVM and reduce power of register file by 66% on average, with only 4.2% performance degradation. After we apply the power gating technique, the register power is further reduced to 25% of SRAM counterpart on average.
---
paper_title: ContextPreRF: Enhancing the Performance and Energy of GPUs With Nonuniform Register Access

paper_content:

Register files are a key data storage unit that impacts instruction throughput for graphics processing units (GPUs). Typically, GPU register files are quite large to accommodate many concurrent threads and are implemented using the same SRAM technology as the on-chip cache. We propose contextrf, a new register file architecture that efficiently leverages register files with nonuniform access characteristics, including hybrid SRAM/DRAM (S/D) and spintronic domain-wall memories (DWMs). Contextrf allows greater-capacity register files to be implemented in the same area within the GPU, with reduced power consumption. We also propose contextPreRF, a hardware preswitch scheme to hide switching delays—as soon as a register request is queued, the nonuniform access memories containing the corresponding register are sent a preemptive switch request. Thus, our scheme transparently hides the penalties of switching between register contexts. After replacing the register file SRAM with S/D, we can reduce energy by 37%, with a 1.4% average performance drop. Employing DWM, we reduce register file energy by 74%, with a 0.4% average performance penalty. For the denser DWM, we model converting the saved area into additional registers, cache, and shared memory—this improves performance by 13.5% over the baseline SRAM register file.
---
paper_title: Soft-error reliability and power co-optimization for GPGPUs register file using resistive memory

paper_content:

The increasing adoption of graphics processing units (GPUs) for high-performance computing raises the reliability challenge, which is generally ignored in traditional GPUs. GPUs usually support thousands of parallel threads and require a sizable register file. Such large register file is highly susceptible to soft errors and power-hungry. Although ECC has been adopted to register file in modern GPUs, it causes considerable power overhead, which further increases the power stress. Thus, an energy-efficient soft-error protection mechanism is more desirable. Besides its extremely low leakage power consumption, resistive memory (e.g. spin-transfer torque RAM) is also immune to the radiation induced soft errors due to its magnetic field based storage. In this paper, we propose to LEverage reSistive memory to enhance the Soft-error robustness and reduce the power consumption (LESS) of registers in the General-Purpose computing on GPUs (GPGPUs). Since resistive memory experiences longer write latency compared to SRAM, we explore the unique characteristics of GPGPU applications to obtain the win-win gains: achieving the near-full soft-error protection for the register file, and meanwhile substantially reducing the energy consumption with negligible performance loss. Our experimental results show that LESS is able to mitigate the registers soft-error vulnerability by 86% and achieve 60% energy savings with negligible (e.g. 4%) performance loss.
---
paper_title: A compile-time managed multi-level register file hierarchy

paper_content:

As processors increasingly become power limited, performance improvements will be achieved by rearchitecting systems with energy efficiency as the primary design constraint. While some of these optimizations will be hardware based, combined hardware and software techniques likely will be the most productive. This work redesigns the register file system of a modern throughput processor with a combined hardware and software solution that reduces register file energy without harming system performance. Throughput processors utilize a large number of threads to tolerate latency, requiring a large, energy-intensive register file to store thread context. Our results show that a compiler controlled register file hierarchy can reduce register file energy by up to 54%, compared to a hardware only caching approach that reduces register file energy by 34%. We explore register allocation algorithms that are specifically targeted to improve energy efficiency by sharing temporary register file resources across concurrently running threads and conduct a detailed limit study on the further potential to optimize operand delivery for throughput processors. Our efficiency gains represent a direct performance gain for power limited systems, such as GPUs.
---
paper_title: An energy-efficient and scalable eDRAM-based register file architecture for GPGPU

paper_content:

The heavily-threaded data processing demands of streaming multiprocessors (SM) in a GPGPU require a large register file (RF). The fast increasing size of the RF makes the area cost and power consumption unaffordable for traditional SRAM designs in the future technologies. In this paper, we propose to use embedded-DRAM (eDRAM) as an alternative in future GPGPUs. Compared with SRAM, eDRAM provides higher density and lower leakage power. However, the limited data retention time in eDRAM poses new challenges. Periodic refresh operations are needed to maintain data integrity. This is exacerbated with the scaling of eDRAM density, process variations and temperature. Unlike conventional CPUs which make use of multi-ported RF, most of the RFs in modern GPGPU are heavily banked but not multi-ported to reduce the hardware cost. This provides a unique opportunity to hide the refresh overhead. We propose two different eDRAM implementations based on 3T1D and 1T1C memory cells. To mitigate the impact of periodic refresh, we propose two novel refresh solutions using bank bubble and bank walk-through. Plus, for the 1T1C RF, we design an interleaved bank organization together with an intelligent warp scheduling strategy to reduce the impact of the destructive reads. The analysis shows that our schemes present better energy efficiency, scalability and variation tolerance than traditional SRAM-based designs.
---
paper_title: Precision-aware soft error protection for GPUs

paper_content:

With the advent of general-purpose GPU computing, it is becoming increasingly desirable to protect GPUs from soft errors. For high computation throughout, GPUs must store a significant amount of state and have many execution units. The high power and area costs of full protection from soft errors make selective protection techniques attractive. Such approaches provide maximum error coverage within a fixed area or power limit, but typically treat all errors equally. We observe that for many floating-point-intensive GPGPU applications, small magnitude errors may have little effect on results, while large magnitude errors can be amplified to have a significant negative impact. We therefore propose a novel precision-aware protection approach for the GPU execution logic and register file to mitigate large magnitude errors. We also propose an architecture modification to optimize error coverage for integer computations. Our approach combines selective logic hardening, targeted checker circuits, and intelligent register file encoding for best error protection. We demonstrate that our approach can reduce the mean error magnitude by up to 87% compared to a traditional selective protection approach with the same overhead.
---
paper_title: An efficient STT-RAM-based register file in GPU architectures

paper_content:

Modern GPGPUs employ a large register file (RF) to efficiently process heavily parallel threads in single instruction multiple thread (SIMT) fashion. The up-scaling of RF capacity, however, is greatly constrained by large cell area and high leakage power consumption of SRAM implementation. In this work, we propose a novel GPU RF design based on the emerging multi-level cell (MLC) spin-transfer torque RAM (STT-RAM) technology. Compared to SRAM, MLC STT-RAM (or MLC-STT) has much smaller cell area and almost zero standby power due to its non- volatility. Moreover, by leveraging the asymmetric performance of the soft and the hard bits of a MLC-STT cell, we propose a remapping strategy to perform a flexible tradeoff between the access time and the capacity of the RF based on run-time access patterns. A novel rescheduling scheme is also developed to minimize the waiting time of the issued warps to access register banks. Experimental results over ISPASS2009 and CUDA benchmarks show that on average, our proposed MLC-STT RF can achieve 3.28% performance improvement, 9.48% energy reduction, and 38.9% energy efficiency enhancement compared to conventional SRAM-based design.
---
paper_title: Power-performance co-optimization of throughput core architecture using resistive memory

paper_content:

Massively parallel computing on throughput computers such as GPUs requires myriad memory accesses to register files, on-chip scratchpad, caches, and off-chip DRAM. Unlike CPUs, these processors have a large register file and on-chip scratchpad memory, which consume a significant portion of compute core power (35%-45%). In this paper, we introduce novel throughput architecture by integrating resistive memory (Spin Transfer Torque RAM) inside the compute core, which reduces leakage significantly, but introduces write power overhead and longer write latencies in GPU shared memory and register file accesses. We enhance the compute core by introducing register file organization with differential memory update mechanism to remove update redundancy during write operations. Furthermore, using merged register-write-mechanism and write-back buffer, we coalesce multithreaded GPU register write accesses to save write energy. In addition, we introduce hybrid shared memory design using SRAM and STT-MRAM that provides significant leakage/dynamic power savings without affecting performance. On average, across 23 GPGPU/graphics workloads, our schemes save 46% dynamic power due to register access (83% leakage power saving) with negligible performance degradation. On average, hybrid shared memory provides 10% reduction in dynamic power with maximum 1.6× performance improvement for the current workloads at no additional area overhead.
---
paper_title: A STT-RAM-based low-power hybrid register file for GPGPUs

paper_content:

Recently, general-purpose graphics processing units (GPGPUs) have been widely used to accelerate computing in various applications. To store the contexts of thousands of concurrent threads on a GPU, a large static random-access memory (SRAM)-based register file is employed. Due to high leakage power of SRAM, the register file consumes 20% to 40% of the total GPU power consumption. Thus, hybrid memory system, which combines SRAM and the emerging non-volatile memory (NVM), has been employed for register file design on GPUs. Although it has shown strong potential to alleviate the power issue of GPUs, existing hybrid memory solutions might not exploit the intrinsic feature of GPU register file. By leveraging the warp schedule on GPU, this paper proposes a hybrid register architecture which consists of a NVM-based register file and mixed SRAM-based write buffers with a warp-aware write back strategy. Simulation results show that our design can eliminate 64% of write accesses to NVM and reduce power of register file by 66% on average, with only 4.2% performance degradation. After we apply the power gating technique, the register power is further reduced to 25% of SRAM counterpart on average.
---
paper_title: Soft-error reliability and power co-optimization for GPGPUs register file using resistive memory

paper_content:

The increasing adoption of graphics processing units (GPUs) for high-performance computing raises the reliability challenge, which is generally ignored in traditional GPUs. GPUs usually support thousands of parallel threads and require a sizable register file. Such large register file is highly susceptible to soft errors and power-hungry. Although ECC has been adopted to register file in modern GPUs, it causes considerable power overhead, which further increases the power stress. Thus, an energy-efficient soft-error protection mechanism is more desirable. Besides its extremely low leakage power consumption, resistive memory (e.g. spin-transfer torque RAM) is also immune to the radiation induced soft errors due to its magnetic field based storage. In this paper, we propose to LEverage reSistive memory to enhance the Soft-error robustness and reduce the power consumption (LESS) of registers in the General-Purpose computing on GPUs (GPGPUs). Since resistive memory experiences longer write latency compared to SRAM, we explore the unique characteristics of GPGPU applications to obtain the win-win gains: achieving the near-full soft-error protection for the register file, and meanwhile substantially reducing the energy consumption with negligible performance loss. Our experimental results show that LESS is able to mitigate the registers soft-error vulnerability by 86% and achieve 60% energy savings with negligible (e.g. 4%) performance loss.
---
paper_title: A Survey Of Architectural Approaches for Managing Embedded DRAM and Non-Volatile On-Chip Caches

paper_content:

Recent trends of CMOS scaling and increasing number of on-chip cores have led to a large increase in the size of on-chip caches. Since SRAM has low density and consumes large amount of leakage power, its use in designing on-chip caches has become more challenging. To address this issue, researchers are exploring the use of several emerging memory technologies, such as embedded DRAM, spin transfer torque RAM, resistive RAM, phase change RAM and domain wall memory. In this paper, we survey the architectural approaches proposed for designing memory systems and, specifically, caches with these emerging memorytechnologies. To highlight their similarities and differences, we present a classification of these technologies and architectural approaches based on their key characteristics. We also briefly summarize the challenges in using these technologies for architecting caches. We believe that this survey will help the readers gain insights into the emerging memory device technologies, and their potential use in designing future computing systems.
---
paper_title: Reducing shift penalty in domain wall memory through register locality

paper_content:

General-purpose graphics processing units (GPGPUs) have the ability to execute hundreds to thousands of threads simultaneously. Extreme multithreading requires a large register file to hold state of executing threads and facilitate context switching. As feature size reduces, power consumption in the large register file becomes a major concern. In this work, we exploit Domain Wall Memory (DWM) which is a spin-based memory to reduce power consumption in register file. DWM is a promising technology and offers non-volatility, high energy efficiency, and high density by storing several bits into the domains of a ferromagnetic wire. However, despite of favourable properties of DMW over SRAM technology, DWM poses a unique challenge that the bits must be accessed serially through shift operations, leading to variable and potentially higher access latencies. To address this challenge, we propose a new predictive shift policy. In this policy, we exploit register locality across threads and predict source and destination operands of instructions. We record history of registers accessed by instructions and shift magnetic domains of DWM tracks for subsequent instructions, speculatively. Over a wide range of applications from NVIDIA CUDA SDK, ISPASS, and Rodinia, our predictive scheme achieves dramatic energy saving over an SRAM register file while changing performance negligibly.
---
paper_title: Exploration of GPGPU register file architecture using domain-wall-shift-write based racetrack memory

paper_content:

SRAM based register file (RF) is one of the major factors limiting the scaling of GPGPU. In this work, we propose to use the emerging nonvolatile domain-wall-shift-write based racetrack memory (DWSW-RM) to implement a power-efficient GPGPU RF, of which the power consumption is substantially reduced. A holistic technology set is developed to minimize the high access cost of DWSW-RW caused by the sequential access mechanism. Experiment results show that our proposed techniques can improve the GPGPU performance by 4.6% compared to the baseline with SRAM based RF. The RF energy efficiency is also significantly improved by 2.45×.
---
paper_title: SRAM-DRAM hybrid memory with applications to efficient register files in fine-grained multi-threading

paper_content:

Large register files are common in highly multi-threaded architectures such as GPUs. This paper presents a hybrid memory design that tightly integrates embedded DRAM into SRAM cells with a main application to reducing area and power consumption of multi-threaded register files. In the hybrid memory, each SRAM cell is augmented with multiple DRAM cells so that multiple bits can be stored in each cell. This configuration results in significant area and energy savings compared to the SRAM array with the same capacity due to compact DRAM cells. On other hand, the hybrid memory requires explicit data movements in order to access DRAM contexts. In order to minimize context switching impact, we introduce write-back buffers, background context switching, and context-aware thread scheduling, to the processor pipeline and the scheduler. Circuit and architecture simulations of GPU benchmarks suites show significant savings in register file area (38%) and energy (68%) over the traditional SRAM implementation, with minimal (1.4%) performance loss.
---
paper_title: ContextPreRF: Enhancing the Performance and Energy of GPUs With Nonuniform Register Access

paper_content:

Register files are a key data storage unit that impacts instruction throughput for graphics processing units (GPUs). Typically, GPU register files are quite large to accommodate many concurrent threads and are implemented using the same SRAM technology as the on-chip cache. We propose contextrf, a new register file architecture that efficiently leverages register files with nonuniform access characteristics, including hybrid SRAM/DRAM (S/D) and spintronic domain-wall memories (DWMs). Contextrf allows greater-capacity register files to be implemented in the same area within the GPU, with reduced power consumption. We also propose contextPreRF, a hardware preswitch scheme to hide switching delays—as soon as a register request is queued, the nonuniform access memories containing the corresponding register are sent a preemptive switch request. Thus, our scheme transparently hides the penalties of switching between register contexts. After replacing the register file SRAM with S/D, we can reduce energy by 37%, with a 1.4% average performance drop. Employing DWM, we reduce register file energy by 74%, with a 0.4% average performance penalty. For the denser DWM, we model converting the saved area into additional registers, cache, and shared memory—this improves performance by 13.5% over the baseline SRAM register file.
---
paper_title: Hybrid CMOS-TFET based register files for energy-efficient GPGPUs

paper_content:

State-of-the-art General-Purpose computing on Graphics Processing Unit (GPGPU) is facing severe power challenge due to the increasing number of cores placed on a chip with decreasing feature size. In order to hide the long latency operations, GPGPU employs the fine-grained multi-threading among numerous active threads, leading to the sizeable register files with massive power consumption. Exploring the optimal power savings in register files becomes the critical and first step towards the energy-efficient GPGPUs. The conventional method to reduce dynamic power consumption is the supply voltage scaling, and the inter-bank tunneling FETs (TFETs) are the promising candidates compared to CMOS for low voltage operations regarding to both leakage and performance. However, always executing at the low voltage (so that low frequency) will result in significant performance degradation. In this study, we propose the hybrid CMOS-TFET based register files. To optimize the register power consumption, we allocate TFET-based registers to threads whose execution progress can be delayed to some degree to avoid the memory contentions with other threads, and the CMOS-based registers are still used for threads requiring normal execution speed. Our experimental results show that the proposed technique achieves 30% energy (including both dynamic and leakage) reduction in register files with little performance degradation compared to the baseline case equipped with naive power optimization technique.
---
paper_title: An energy-efficient and scalable eDRAM-based register file architecture for GPGPU

paper_content:

The heavily-threaded data processing demands of streaming multiprocessors (SM) in a GPGPU require a large register file (RF). The fast increasing size of the RF makes the area cost and power consumption unaffordable for traditional SRAM designs in the future technologies. In this paper, we propose to use embedded-DRAM (eDRAM) as an alternative in future GPGPUs. Compared with SRAM, eDRAM provides higher density and lower leakage power. However, the limited data retention time in eDRAM poses new challenges. Periodic refresh operations are needed to maintain data integrity. This is exacerbated with the scaling of eDRAM density, process variations and temperature. Unlike conventional CPUs which make use of multi-ported RF, most of the RFs in modern GPGPU are heavily banked but not multi-ported to reduce the hardware cost. This provides a unique opportunity to hide the refresh overhead. We propose two different eDRAM implementations based on 3T1D and 1T1C memory cells. To mitigate the impact of periodic refresh, we propose two novel refresh solutions using bank bubble and bank walk-through. Plus, for the 1T1C RF, we design an interleaved bank organization together with an intelligent warp scheduling strategy to reduce the impact of the destructive reads. The analysis shows that our schemes present better energy efficiency, scalability and variation tolerance than traditional SRAM-based designs.
---
paper_title: GPUWattch: enabling energy optimizations in GPGPUs

paper_content:

General-purpose GPUs (GPGPUs) are becoming prevalent in mainstream computing, and performance per watt has emerged as a more crucial evaluation metric than peak performance. As such, GPU architects require robust tools that will enable them to quickly explore new ways to optimize GPGPUs for energy efficiency. We propose a new GPGPU power model that is configurable, capable of cycle-level calculations, and carefully validated against real hardware measurements. To achieve configurability, we use a bottom-up methodology and abstract parameters from the microarchitectural components as the model's inputs. We developed a rigorous suite of 80 microbenchmarks that we use to bound any modeling uncertainties and inaccuracies. The power model is comprehensively validated against measurements of two commercially available GPUs, and the measured error is within 9.9% and 13.4% for the two target GPUs (GTX 480 and Quadro FX5600). The model also accurately tracks the power consumption trend over time. We integrated the power model with the cycle-level simulator GPGPU-Sim and demonstrate the energy savings by utilizing dynamic voltage and frequency scaling (DVFS) and clock gating. Traditional DVFS reduces GPU energy consumption by 14.4% by leveraging within-kernel runtime variations. More finer-grained SM cluster-level DVFS improves the energy savings from 6.6% to 13.6% for those benchmarks that show clustered execution behavior. We also show that clock gating inactive lanes during divergence reduces dynamic power by 11.2%.
---
paper_title: Architecture supported register stash for GPGPU

paper_content:

GPGPU provides abundant hardware resources to support a large number of light-weighted threads. They are organized into blocks and run in warps. All threads of a block must be dispatched to one stream multiprocessor (SM) of GPGPU together. When the remaining resources of an SM cannot support one more block, all threads of the block are held back until former blocks retire from the SM. We found that the register file is prone to be the most limited one among all the resources, especially for SMs with less registers. Meanwhile, we revealed the dynamics of a thread's register requirement: only part of its pre-allocated registers are used for different instructions at run time. This results in considerable register underutilization.We proposed the architecture supported register stash (ASRS). It removes the limitation of registers when dispatching blocks. The hardware registers are allocated at run time according to each instruction's live registers, which can be analyzed statically by a compiler. When the hardware registers cannot meet the requirements of all running warps, some warps are suspended and their registers are reclaimed temporarily. The data in these registers are stashed to memory. On the other hand, if there are spare hardware registers, it will start a new warp or resume a suspended warp after all the warp's stashed register data are loaded from memory. The intra-block synchronization is also taken care of when some of the warps of the same block are not schedulable due to the ASRS.The ASRS alleviates the register underutilization and improves performance without modifying the current programming model or demanding extra effort from the programmers. It also enables an SM with limited registers that cannot even support a single block to execute it. Besides, it helps lower the register file energy consumption and increase the power efficiency. The ASRS achieved speedups of 1.59 and 1.14 when the registers of each SM are limited to 8K and 16K respectively with an insignificant overhead. The speedups compared with the infinite register files are 0.84 and 0.98 with 8K and 16K registers respectively. Compared with the baseline 32K register file, the ASRS decreases the 8K and 16K register file energy consumption to 66.5% and 75.8% respectively. Their power efficiencies (in ratio of performance and power) are increased to 1.29x?and 1.31x?respectively. Register requirement of GPGPU varies among different kernels and during run time.Register file (RF) capacity limits the schedulable warps and performance.Reducing RF capacity lowers energy consumption and area.We proposed a method to support more warps with limited registers.It gains significant speedup and a higher energy efficiency with a smaller RF.
---
paper_title: Energy-efficient mechanisms for managing thread context in throughput processors

paper_content:

Modern graphics processing units (GPUs) use a large number of hardware threads to hide both function unit and memory access latency. Extreme multithreading requires a complicated thread scheduler as well as a large register file, which is expensive to access both in terms of energy and latency. We present two complementary techniques for reducing energy on massively-threaded processors such as GPUs. First, we examine register file caching to replace accesses to the large main register file with accesses to a smaller structure containing the immediate register working set of active threads. Second, we investigate a two-level thread scheduler that maintains a small set of active threads to hide ALU and local memory access latency and a larger set of pending threads to hide main memory latency. Combined with register file caching, a two-level thread scheduler provides a further reduction in energy by limiting the allocation of temporary register cache resources to only the currently active subset of threads. We show that on average, across a variety of real world graphics and compute workloads, a 6-entry per-thread register file cache reduces the number of reads and writes to the main register file by 50% and 59% respectively. We further show that the active thread count can be reduced by a factor of 4 with minimal impact on performance, resulting in a 36% reduction of register file energy.
---
paper_title: An Adaptive Thread Scheduling Mechanism With Low-Power Register File for Mobile GPUs

paper_content:

In response to the remarkable increase in 3D applications in consumer electronics devices in recent years, graphics processing units (GPUs) have become widely available on mobile devices. These GPUs typically use hardware multithreaded shaders to improve their throughputs for real-time rendering, but they depend on duplicate register files to maintain the context of each hardware thread, increasing power consumption. However, the register usage of shading programs is often relatively low, which causes many registers to remain unused, thus wasting power. Long latency memory operations can also consume unnecessary power to activate registers. This study proposes a low-power register file with multiple power modes to reduce the power consumption of the register file. This study also presents an adaptive thread scheduling mechanism to achieve a tradeoff between the power consumption of the register file and frames per second (FPS). Results show that the average performance degradation from the proposed low-power register file is only 0.62%. The proposed adaptive thread scheduling has average under prediction ratio of 3.32%. The leakage reduction of the proposed low-power register file is 74.80%. This reduction can be improved to 81.49%, 82.22%, and 84.28% with adaptive thread scheduling at frame rates of 30, 25, and 20, respectively.
---
paper_title: GPGPU Register File Management by Hardware Co-operated Register Reallocation

paper_content:

To support massive parallel threads context, GPGPUs use a huge register file. Due to their size, register file is one of the most power hungry logic in GPGPU. However, the current trends indicate that GPGPU register file size will continue to get even bigger as the demand for higher single instruction multiple thread (SIMT) parallelism increases, particularly in high performance application domain. In order to reduce power consumption demand, in this work, we exploit a fundamental observation that fairly high portion of the register spaces are unnecessarily allocated and burn power due to the fact that the registers are considered as private resource for each warp. For example, even though a register’s value is no longer used by any instruction, the register space should be occupied by the warp during the program execution. In GPGPU, as single register usage in a code context leads to thousands of register space allocation, the power and space overhead due to any wasted register is significant. Instead, we propose to share the register file across warps. In our proposed allocation, a register is released from its physical register space immediately after its last use. Then, the released register space is reassigned to another warp’s register. The compile time register lifetime analysis information is used for providing hint to the hardware about each register’s release point. To enable this register reassignment, we propose a light weight register renaming in hardware. By releasing registers and reusing them across warps, we can reduce the demand for register file size on average by 30% compared with the optimally compiled applications. The reduced live register space leads to an average of 23% and 26% static power saving over a basic sub-array level and individual register level power gating.
---
paper_title: An efficient STT-RAM-based register file in GPU architectures

paper_content:

Modern GPGPUs employ a large register file (RF) to efficiently process heavily parallel threads in single instruction multiple thread (SIMT) fashion. The up-scaling of RF capacity, however, is greatly constrained by large cell area and high leakage power consumption of SRAM implementation. In this work, we propose a novel GPU RF design based on the emerging multi-level cell (MLC) spin-transfer torque RAM (STT-RAM) technology. Compared to SRAM, MLC STT-RAM (or MLC-STT) has much smaller cell area and almost zero standby power due to its non- volatility. Moreover, by leveraging the asymmetric performance of the soft and the hard bits of a MLC-STT cell, we propose a remapping strategy to perform a flexible tradeoff between the access time and the capacity of the RF based on run-time access patterns. A novel rescheduling scheme is also developed to minimize the waiting time of the issued warps to access register banks. Experimental results over ISPASS2009 and CUDA benchmarks show that on average, our proposed MLC-STT RF can achieve 3.28% performance improvement, 9.48% energy reduction, and 38.9% energy efficiency enhancement compared to conventional SRAM-based design.
---
paper_title: A Write-Aware STTRAM-Based Register File Architecture for GPGPU

paper_content:

The massively parallel processing capacity of GPGPUs requires a large register file (RF), and its size keeps increasing to support more concurrent threads from generation to generation. Using traditional SRAM-based RFs, there are concerns in both area cost and energy consumption, and soon they will become unrealistic. In this work, we analyze the feasibility of using STTRAM-based RF designs, which have benefits in terms of smaller silicon area and zero standby leakage power. However, STTRAM long write latency and high write energy bring new challenges. Therefore, we propose a write-aware STTRAM-based RF architecture (WarRF), which contains two techniques: Split Bank Write modifies the arbitrator design to increase the parallelism of read and write accesses in the same bank; Write Pool reduces the number of repeated write accesses to RFs. Our experiment shows that the performance of STTRAM-based RF is improved by 13p and up to 23p after adopting WarRF. In addition, the energy consumption is reduced by 38p on average compared to SRAM-based RFs.
---
paper_title: Register Caching for Stencil Computations on GPUs

paper_content:

For most applications, taking full advantage of the memory system is key to achieving good performance on GPUs. In this paper, we introduce register caching, a novel idea where registers of multiple threads are combined and used as a shared, last level, manually managed cache for the contributing threads. This method is enabled by the shuffle instruction recently introduced in Nvidia's Kepler GPU architecture, which allows threads in the same warp to exchange data directly, previously only possible by going through shared memory. We evaluate our proposal with a stencil computation benchmark, achieving speedups of up to 2.04, compared to using shared memory on a GTX680 GPU. Stencil computations form the core of many scientific applications, which can therefore benefit from our proposal. Furthermore, our method is not limited to stencil computations, but is applicable to any application with a predictable memory access pattern suitable for manual caching.
---
paper_title: Exploiting uniform vector instructions for GPGPU performance, energy efficiency, and opportunistic reliability enhancement

paper_content:

State-of-art graphics processing units (GPUs) employ the single-instruction multiple-data (SIMD) style execution to achieve both high computational throughput and energy efficiency. As previous works have shown, there exists significant computational redundancy in SIMD execution, where different execution lanes operate on the same operand values. Such value locality is referred to as uniform vectors. In this paper, we first show that besides redundancy within a uniform vector, different vectors can also have the identical values. Then, we propose detailed architecture designs to exploit both types of redundancy. For redundancy within a uniform vector, we propose to either extend the vector register file with token bits or add a separate small scalar register file to eliminate redundant computations as well as redundant data storage. For redundancy across different uniform vectors, we adopt instruction reuse, proposed originally for CPU architectures, to detect and eliminate redundancy. The elimination of redundant computations and data storage leads to both significant energy savings and performance improvement. Furthermore, we propose to leverage such redundancy to protect arithmetic-logic units (ALUs) and register files against hardware errors. Our detailed evaluation shows that our proposed design has low hardware overhead and achieves performance gains, up to 23.9% and 12.0% on average, along with energy savings, up to 24.8% and 12.6% on average, as well as a 21.1% and 14.1% protection coverage for ALUs and register files, respectively.
---
paper_title: Compiler assisted dynamic register file in GPGPU

paper_content:

The large Register File (RF) in General Purpose Graphic Processing Units (GPGPUs) demands tremendous chip area and energy consumption. For a sustainable growth of the size of RF in future GPGPUs, emerging on-chip memory technologies such as embedded-DRAM (eDRAM) have been proposed to replace the conventional SRAM for higher density and lower leakage but with the possible penalty from the periodic refresh operations. This paper explicitly shows that the refresh penalty can be effectively mitigated by leveraging the uniqueness of GPGPU operations. A compiler assisted refresh rescheduling policy can greatly reduce the refresh overhead for maintaining the correctness of the RF operations. The proposed scheme adequately exploits the features in both architecture and compilation, and delivers comparable performance to the SRAM counterpart. At the same time, the energy savings via the removal of large SRAM leakage well compensate for the additional refresh energy. This study promotes the eDRAM-based RF as a promising alternative that enables larger capacity and better power efficiency for future GPGPUs.
---
paper_title: Unifying Primary Cache, Scratch, and Register File Memories in a Throughput Processor

paper_content:

Modern throughput processors such as GPUs employ thousands of threads to drive high-bandwidth, long-latency memory systems. These threads require substantial on-chip storage for registers, cache, and scratchpad memory. Existing designs hard-partition this local storage, fixing the capacities of these structures at design time. We evaluate modern GPU workloads and find that they have widely varying capacity needs across these different functions. Therefore, we propose a unified local memory which can dynamically change the partitioning among registers, cache, and scratchpad on a per-application basis. The tuning that this flexibility enables improves both performance and energy consumption, and broadens the scope of applications that can be efficiently executed on GPUs. Compared to a hard-partitioned design, we show that unified local memory provides a performance benefit as high as 71% along with an energy reduction up to 33%.
---
paper_title: Power-efficient computing for compute-intensive GPGPU applications

paper_content:

The peak compute performance of GPUs has been increased by integrating more compute resources and operating them at higher frequency. However, such approaches significantly increase power consumption of GPUs, limiting further performance increase due to the power constraint. Facing such a challenge, we propose three techniques to improve power efficiency and performance of GPUs in this paper. First, we observe that many GPGPU applications are integer-intensive. For such applications, we combine a pair of dependent integer instructions into a composite instruction that can be executed by an enhanced fused multiply-add unit. Second, we observe that computations for many instructions are duplicated across multiple threads. We dynamically detect such instructions and execute them in a separate scalar unit. Finally, we observe that 16 or fewer bits are sufficient for accurate representation of operands and results of many instructions. Thus, we split the 32-bit datapath into two 16-bit datapath slices that can concurrently issue and execute up to two such instructions per cycle. All three proposed techniques can considerably increase utilization of compute resources, improving power efficiency and performance by 20% and 15%, respectively.
---
paper_title: Bank stealing for conflict mitigation in GPGPU Register File

paper_content:

Modern General Purpose Graphic Processing Unit (GPGPU) demands a large Register File (RF), which is typically organized into multiple banks to support the massive parallelism. Although heavy banking benefits RF throughput, its associated area and energy costs with diminishing performance gains greatly limit future RF s-caling. In this paper, we propose an improved RF design with a bank stealing technique, which enables a high RF throughput with compact area. By deeply investigating the GPGPU microarchitecture, we identify the deficiency in the state-of-the-art RF designs as the bank conflict problem, while the majority of conflicts can be eliminated leveraging the fact that the highly-banked RF oftentimes experiences under-utilization. This is especially true in GPGPU where multiple ready warps are available at the scheduling stage with their operands to be wisely coordinated. Our lightweight bank stealing technique can opportunistically fill the idle banks for better operand service, and the average GPGPU performance can be improved under smaller energy budget with significant area saving, which makes it promising for sustainable RF scaling.
---
paper_title: A STT-RAM-based low-power hybrid register file for GPGPUs

paper_content:

Recently, general-purpose graphics processing units (GPGPUs) have been widely used to accelerate computing in various applications. To store the contexts of thousands of concurrent threads on a GPU, a large static random-access memory (SRAM)-based register file is employed. Due to high leakage power of SRAM, the register file consumes 20% to 40% of the total GPU power consumption. Thus, hybrid memory system, which combines SRAM and the emerging non-volatile memory (NVM), has been employed for register file design on GPUs. Although it has shown strong potential to alleviate the power issue of GPUs, existing hybrid memory solutions might not exploit the intrinsic feature of GPU register file. By leveraging the warp schedule on GPU, this paper proposes a hybrid register architecture which consists of a NVM-based register file and mixed SRAM-based write buffers with a warp-aware write back strategy. Simulation results show that our design can eliminate 64% of write accesses to NVM and reduce power of register file by 66% on average, with only 4.2% performance degradation. After we apply the power gating technique, the register power is further reduced to 25% of SRAM counterpart on average.
---
paper_title: ContextPreRF: Enhancing the Performance and Energy of GPUs With Nonuniform Register Access

paper_content:

Register files are a key data storage unit that impacts instruction throughput for graphics processing units (GPUs). Typically, GPU register files are quite large to accommodate many concurrent threads and are implemented using the same SRAM technology as the on-chip cache. We propose contextrf, a new register file architecture that efficiently leverages register files with nonuniform access characteristics, including hybrid SRAM/DRAM (S/D) and spintronic domain-wall memories (DWMs). Contextrf allows greater-capacity register files to be implemented in the same area within the GPU, with reduced power consumption. We also propose contextPreRF, a hardware preswitch scheme to hide switching delays—as soon as a register request is queued, the nonuniform access memories containing the corresponding register are sent a preemptive switch request. Thus, our scheme transparently hides the penalties of switching between register contexts. After replacing the register file SRAM with S/D, we can reduce energy by 37%, with a 1.4% average performance drop. Employing DWM, we reduce register file energy by 74%, with a 0.4% average performance penalty. For the denser DWM, we model converting the saved area into additional registers, cache, and shared memory—this improves performance by 13.5% over the baseline SRAM register file.
---
paper_title: Spare register aware prefetching for graph algorithms on GPUs

paper_content:

More and more graph algorithms are being GPU enabled. Graph algorithm implementations on GPUs have irregular control flow and are memory-intensive with many irregular/data-dependent memory accesses. Due to these factors graph algorithms on GPUs have low execution efficiency. In this work we propose a mechanism to improve the execution efficiency of graph algorithms by improving their memory access latency tolerance. We propose a mechanism for prefetching data for load pairs that have one load dependent on the other - such pairs are common in graph algorithms. Our mechanism detects the target loads in hardware and injects instructions into the pipeline to prefetch data into spare registers that are not being used by any active threads. By prefetching data into registers, early eviction of prefetched data can be eliminated. We also propose a mechanism that uses the compiler to identify the target loads. Our mechanism improves performance over no prefetching by 10% on average and upto 51% for nine memory intensive graph algorithm kernels.
---
paper_title: Soft-error reliability and power co-optimization for GPGPUs register file using resistive memory

paper_content:

The increasing adoption of graphics processing units (GPUs) for high-performance computing raises the reliability challenge, which is generally ignored in traditional GPUs. GPUs usually support thousands of parallel threads and require a sizable register file. Such large register file is highly susceptible to soft errors and power-hungry. Although ECC has been adopted to register file in modern GPUs, it causes considerable power overhead, which further increases the power stress. Thus, an energy-efficient soft-error protection mechanism is more desirable. Besides its extremely low leakage power consumption, resistive memory (e.g. spin-transfer torque RAM) is also immune to the radiation induced soft errors due to its magnetic field based storage. In this paper, we propose to LEverage reSistive memory to enhance the Soft-error robustness and reduce the power consumption (LESS) of registers in the General-Purpose computing on GPUs (GPGPUs). Since resistive memory experiences longer write latency compared to SRAM, we explore the unique characteristics of GPGPU applications to obtain the win-win gains: achieving the near-full soft-error protection for the register file, and meanwhile substantially reducing the energy consumption with negligible performance loss. Our experimental results show that LESS is able to mitigate the registers soft-error vulnerability by 86% and achieve 60% energy savings with negligible (e.g. 4%) performance loss.
---
paper_title: Warped-compression: enabling power efficient GPUs through register compression

paper_content:

This paper presents Warped-Compression, a warp-level register compression scheme for reducing GPU power consumption. This work is motivated by the observation that the register values of threads within the same warp are similar, namely the arithmetic differences between two successive thread registers is small. Removing data redundancy of register values through register compression reduces the effective register width, thereby enabling power reduction opportunities. GPU register files are huge as they are necessary to keep concurrent execution contexts and to enable fast context switching. As a result register file consumes a large fraction of the total GPU chip power. GPU design trends show that the register file size will continue to increase to enable even more thread level parallelism. To reduce register file data redundancy warped-compression uses low-cost and implementation-efficient base-delta-immediate (BDI) compression scheme, that takes advantage of banked register file organization used in GPUs. Since threads within a warp write values with strong similarity, BDI can quickly compress and decompress by selecting either a single register, or one of the register banks, as the primary base and then computing delta values of all the other registers, or banks. Warped-compression can be used to reduce both dynamic and leakage power. By compressing register values, each warp-level register access activates fewer register banks, which leads to reduction in dynamic power. When fewer banks are used to store the register content, leakage power can be reduced by power gating the unused banks. Evaluation results show that register compression saves 25% of the total register file power consumption.
---
paper_title: A compile-time managed multi-level register file hierarchy

paper_content:

As processors increasingly become power limited, performance improvements will be achieved by rearchitecting systems with energy efficiency as the primary design constraint. While some of these optimizations will be hardware based, combined hardware and software techniques likely will be the most productive. This work redesigns the register file system of a modern throughput processor with a combined hardware and software solution that reduces register file energy without harming system performance. Throughput processors utilize a large number of threads to tolerate latency, requiring a large, energy-intensive register file to store thread context. Our results show that a compiler controlled register file hierarchy can reduce register file energy by up to 54%, compared to a hardware only caching approach that reduces register file energy by 34%. We explore register allocation algorithms that are specifically targeted to improve energy efficiency by sharing temporary register file resources across concurrently running threads and conduct a detailed limit study on the further potential to optimize operand delivery for throughput processors. Our efficiency gains represent a direct performance gain for power limited systems, such as GPUs.
---
paper_title: An energy-efficient and scalable eDRAM-based register file architecture for GPGPU

paper_content:

The heavily-threaded data processing demands of streaming multiprocessors (SM) in a GPGPU require a large register file (RF). The fast increasing size of the RF makes the area cost and power consumption unaffordable for traditional SRAM designs in the future technologies. In this paper, we propose to use embedded-DRAM (eDRAM) as an alternative in future GPGPUs. Compared with SRAM, eDRAM provides higher density and lower leakage power. However, the limited data retention time in eDRAM poses new challenges. Periodic refresh operations are needed to maintain data integrity. This is exacerbated with the scaling of eDRAM density, process variations and temperature. Unlike conventional CPUs which make use of multi-ported RF, most of the RFs in modern GPGPU are heavily banked but not multi-ported to reduce the hardware cost. This provides a unique opportunity to hide the refresh overhead. We propose two different eDRAM implementations based on 3T1D and 1T1C memory cells. To mitigate the impact of periodic refresh, we propose two novel refresh solutions using bank bubble and bank walk-through. Plus, for the 1T1C RF, we design an interleaved bank organization together with an intelligent warp scheduling strategy to reduce the impact of the destructive reads. The analysis shows that our schemes present better energy efficiency, scalability and variation tolerance than traditional SRAM-based designs.
---
paper_title: Reducing shift penalty in domain wall memory through register locality

paper_content:

General-purpose graphics processing units (GPGPUs) have the ability to execute hundreds to thousands of threads simultaneously. Extreme multithreading requires a large register file to hold state of executing threads and facilitate context switching. As feature size reduces, power consumption in the large register file becomes a major concern. In this work, we exploit Domain Wall Memory (DWM) which is a spin-based memory to reduce power consumption in register file. DWM is a promising technology and offers non-volatility, high energy efficiency, and high density by storing several bits into the domains of a ferromagnetic wire. However, despite of favourable properties of DMW over SRAM technology, DWM poses a unique challenge that the bits must be accessed serially through shift operations, leading to variable and potentially higher access latencies. To address this challenge, we propose a new predictive shift policy. In this policy, we exploit register locality across threads and predict source and destination operands of instructions. We record history of registers accessed by instructions and shift magnetic domains of DWM tracks for subsequent instructions, speculatively. Over a wide range of applications from NVIDIA CUDA SDK, ISPASS, and Rodinia, our predictive scheme achieves dramatic energy saving over an SRAM register file while changing performance negligibly.
---
paper_title: SRAM-DRAM hybrid memory with applications to efficient register files in fine-grained multi-threading

paper_content:

Large register files are common in highly multi-threaded architectures such as GPUs. This paper presents a hybrid memory design that tightly integrates embedded DRAM into SRAM cells with a main application to reducing area and power consumption of multi-threaded register files. In the hybrid memory, each SRAM cell is augmented with multiple DRAM cells so that multiple bits can be stored in each cell. This configuration results in significant area and energy savings compared to the SRAM array with the same capacity due to compact DRAM cells. On other hand, the hybrid memory requires explicit data movements in order to access DRAM contexts. In order to minimize context switching impact, we introduce write-back buffers, background context switching, and context-aware thread scheduling, to the processor pipeline and the scheduler. Circuit and architecture simulations of GPU benchmarks suites show significant savings in register file area (38%) and energy (68%) over the traditional SRAM implementation, with minimal (1.4%) performance loss.
---
paper_title: An eDRAM-Based Approximate Register File for GPUs

paper_content:

GPUs require large register files for fast context switching. This paper presents a high-density and energy-efficient approximate register file architecture based on embedded DRAM with lowered refresh rate to the low-order bits of register entries.
---
paper_title: GPUWattch: enabling energy optimizations in GPGPUs

paper_content:

General-purpose GPUs (GPGPUs) are becoming prevalent in mainstream computing, and performance per watt has emerged as a more crucial evaluation metric than peak performance. As such, GPU architects require robust tools that will enable them to quickly explore new ways to optimize GPGPUs for energy efficiency. We propose a new GPGPU power model that is configurable, capable of cycle-level calculations, and carefully validated against real hardware measurements. To achieve configurability, we use a bottom-up methodology and abstract parameters from the microarchitectural components as the model's inputs. We developed a rigorous suite of 80 microbenchmarks that we use to bound any modeling uncertainties and inaccuracies. The power model is comprehensively validated against measurements of two commercially available GPUs, and the measured error is within 9.9% and 13.4% for the two target GPUs (GTX 480 and Quadro FX5600). The model also accurately tracks the power consumption trend over time. We integrated the power model with the cycle-level simulator GPGPU-Sim and demonstrate the energy savings by utilizing dynamic voltage and frequency scaling (DVFS) and clock gating. Traditional DVFS reduces GPU energy consumption by 14.4% by leveraging within-kernel runtime variations. More finer-grained SM cluster-level DVFS improves the energy savings from 6.6% to 13.6% for those benchmarks that show clustered execution behavior. We also show that clock gating inactive lanes during divergence reduces dynamic power by 11.2%.
---
paper_title: Architecture supported register stash for GPGPU

paper_content:

GPGPU provides abundant hardware resources to support a large number of light-weighted threads. They are organized into blocks and run in warps. All threads of a block must be dispatched to one stream multiprocessor (SM) of GPGPU together. When the remaining resources of an SM cannot support one more block, all threads of the block are held back until former blocks retire from the SM. We found that the register file is prone to be the most limited one among all the resources, especially for SMs with less registers. Meanwhile, we revealed the dynamics of a thread's register requirement: only part of its pre-allocated registers are used for different instructions at run time. This results in considerable register underutilization.We proposed the architecture supported register stash (ASRS). It removes the limitation of registers when dispatching blocks. The hardware registers are allocated at run time according to each instruction's live registers, which can be analyzed statically by a compiler. When the hardware registers cannot meet the requirements of all running warps, some warps are suspended and their registers are reclaimed temporarily. The data in these registers are stashed to memory. On the other hand, if there are spare hardware registers, it will start a new warp or resume a suspended warp after all the warp's stashed register data are loaded from memory. The intra-block synchronization is also taken care of when some of the warps of the same block are not schedulable due to the ASRS.The ASRS alleviates the register underutilization and improves performance without modifying the current programming model or demanding extra effort from the programmers. It also enables an SM with limited registers that cannot even support a single block to execute it. Besides, it helps lower the register file energy consumption and increase the power efficiency. The ASRS achieved speedups of 1.59 and 1.14 when the registers of each SM are limited to 8K and 16K respectively with an insignificant overhead. The speedups compared with the infinite register files are 0.84 and 0.98 with 8K and 16K registers respectively. Compared with the baseline 32K register file, the ASRS decreases the 8K and 16K register file energy consumption to 66.5% and 75.8% respectively. Their power efficiencies (in ratio of performance and power) are increased to 1.29x?and 1.31x?respectively. Register requirement of GPGPU varies among different kernels and during run time.Register file (RF) capacity limits the schedulable warps and performance.Reducing RF capacity lowers energy consumption and area.We proposed a method to support more warps with limited registers.It gains significant speedup and a higher energy efficiency with a smaller RF.
---
paper_title: Mitigating the Susceptibility of GPGPUs Register File to Process Variations

paper_content:

As technology keeps scaling down at nano-scale, the increasing process variations (PV) induce significant delay variations and limit the maximum clock frequency in GPGPUs (general-purpose computing on graphics processing units). Each computing core (i.e. streaming multiprocessor) in GPGPUs supports thousands of simultaneously active threads, and requires a large register file. Such a sizeable register file is very sensitive to process variations, and becomes one of the major units in determining the core frequency. In this study, we first develop a novel mechanism that classifies registers into fast and slow categories in the highly-banked register architecture to maximize the frequency improvement. We then leverage the unique features in GPGPU applications to effectively tolerate the extra access delay to the slow registers. Our experimental results show that our proposed techniques are able to significantly optimize GPGPUs performance under process variations.
---
paper_title: Energy-efficient mechanisms for managing thread context in throughput processors

paper_content:

Modern graphics processing units (GPUs) use a large number of hardware threads to hide both function unit and memory access latency. Extreme multithreading requires a complicated thread scheduler as well as a large register file, which is expensive to access both in terms of energy and latency. We present two complementary techniques for reducing energy on massively-threaded processors such as GPUs. First, we examine register file caching to replace accesses to the large main register file with accesses to a smaller structure containing the immediate register working set of active threads. Second, we investigate a two-level thread scheduler that maintains a small set of active threads to hide ALU and local memory access latency and a larger set of pending threads to hide main memory latency. Combined with register file caching, a two-level thread scheduler provides a further reduction in energy by limiting the allocation of temporary register cache resources to only the currently active subset of threads. We show that on average, across a variety of real world graphics and compute workloads, a 6-entry per-thread register file cache reduces the number of reads and writes to the main register file by 50% and 59% respectively. We further show that the active thread count can be reduced by a factor of 4 with minimal impact on performance, resulting in a 36% reduction of register file energy.
---
paper_title: An Adaptive Thread Scheduling Mechanism With Low-Power Register File for Mobile GPUs

paper_content:

In response to the remarkable increase in 3D applications in consumer electronics devices in recent years, graphics processing units (GPUs) have become widely available on mobile devices. These GPUs typically use hardware multithreaded shaders to improve their throughputs for real-time rendering, but they depend on duplicate register files to maintain the context of each hardware thread, increasing power consumption. However, the register usage of shading programs is often relatively low, which causes many registers to remain unused, thus wasting power. Long latency memory operations can also consume unnecessary power to activate registers. This study proposes a low-power register file with multiple power modes to reduce the power consumption of the register file. This study also presents an adaptive thread scheduling mechanism to achieve a tradeoff between the power consumption of the register file and frames per second (FPS). Results show that the average performance degradation from the proposed low-power register file is only 0.62%. The proposed adaptive thread scheduling has average under prediction ratio of 3.32%. The leakage reduction of the proposed low-power register file is 74.80%. This reduction can be improved to 81.49%, 82.22%, and 84.28% with adaptive thread scheduling at frame rates of 30, 25, and 20, respectively.
---
paper_title: GPGPU Register File Management by Hardware Co-operated Register Reallocation

paper_content:

To support massive parallel threads context, GPGPUs use a huge register file. Due to their size, register file is one of the most power hungry logic in GPGPU. However, the current trends indicate that GPGPU register file size will continue to get even bigger as the demand for higher single instruction multiple thread (SIMT) parallelism increases, particularly in high performance application domain. In order to reduce power consumption demand, in this work, we exploit a fundamental observation that fairly high portion of the register spaces are unnecessarily allocated and burn power due to the fact that the registers are considered as private resource for each warp. For example, even though a register’s value is no longer used by any instruction, the register space should be occupied by the warp during the program execution. In GPGPU, as single register usage in a code context leads to thousands of register space allocation, the power and space overhead due to any wasted register is significant. Instead, we propose to share the register file across warps. In our proposed allocation, a register is released from its physical register space immediately after its last use. Then, the released register space is reassigned to another warp’s register. The compile time register lifetime analysis information is used for providing hint to the hardware about each register’s release point. To enable this register reassignment, we propose a light weight register renaming in hardware. By releasing registers and reusing them across warps, we can reduce the demand for register file size on average by 30% compared with the optimally compiled applications. The reduced live register space leads to an average of 23% and 26% static power saving over a basic sub-array level and individual register level power gating.
---
paper_title: An efficient STT-RAM-based register file in GPU architectures

paper_content:

Modern GPGPUs employ a large register file (RF) to efficiently process heavily parallel threads in single instruction multiple thread (SIMT) fashion. The up-scaling of RF capacity, however, is greatly constrained by large cell area and high leakage power consumption of SRAM implementation. In this work, we propose a novel GPU RF design based on the emerging multi-level cell (MLC) spin-transfer torque RAM (STT-RAM) technology. Compared to SRAM, MLC STT-RAM (or MLC-STT) has much smaller cell area and almost zero standby power due to its non- volatility. Moreover, by leveraging the asymmetric performance of the soft and the hard bits of a MLC-STT cell, we propose a remapping strategy to perform a flexible tradeoff between the access time and the capacity of the RF based on run-time access patterns. A novel rescheduling scheme is also developed to minimize the waiting time of the issued warps to access register banks. Experimental results over ISPASS2009 and CUDA benchmarks show that on average, our proposed MLC-STT RF can achieve 3.28% performance improvement, 9.48% energy reduction, and 38.9% energy efficiency enhancement compared to conventional SRAM-based design.
---
paper_title: Exploration of GPGPU register file architecture using domain-wall-shift-write based racetrack memory

paper_content:

SRAM based register file (RF) is one of the major factors limiting the scaling of GPGPU. In this work, we propose to use the emerging nonvolatile domain-wall-shift-write based racetrack memory (DWSW-RM) to implement a power-efficient GPGPU RF, of which the power consumption is substantially reduced. A holistic technology set is developed to minimize the high access cost of DWSW-RW caused by the sequential access mechanism. Experiment results show that our proposed techniques can improve the GPGPU performance by 4.6% compared to the baseline with SRAM based RF. The RF energy efficiency is also significantly improved by 2.45×.
---
paper_title: Register Caching for Stencil Computations on GPUs

paper_content:

For most applications, taking full advantage of the memory system is key to achieving good performance on GPUs. In this paper, we introduce register caching, a novel idea where registers of multiple threads are combined and used as a shared, last level, manually managed cache for the contributing threads. This method is enabled by the shuffle instruction recently introduced in Nvidia's Kepler GPU architecture, which allows threads in the same warp to exchange data directly, previously only possible by going through shared memory. We evaluate our proposal with a stencil computation benchmark, achieving speedups of up to 2.04, compared to using shared memory on a GTX680 GPU. Stencil computations form the core of many scientific applications, which can therefore benefit from our proposal. Furthermore, our method is not limited to stencil computations, but is applicable to any application with a predictable memory access pattern suitable for manual caching.
---
paper_title: Hybrid CMOS-TFET based register files for energy-efficient GPGPUs

paper_content:

State-of-the-art General-Purpose computing on Graphics Processing Unit (GPGPU) is facing severe power challenge due to the increasing number of cores placed on a chip with decreasing feature size. In order to hide the long latency operations, GPGPU employs the fine-grained multi-threading among numerous active threads, leading to the sizeable register files with massive power consumption. Exploring the optimal power savings in register files becomes the critical and first step towards the energy-efficient GPGPUs. The conventional method to reduce dynamic power consumption is the supply voltage scaling, and the inter-bank tunneling FETs (TFETs) are the promising candidates compared to CMOS for low voltage operations regarding to both leakage and performance. However, always executing at the low voltage (so that low frequency) will result in significant performance degradation. In this study, we propose the hybrid CMOS-TFET based register files. To optimize the register power consumption, we allocate TFET-based registers to threads whose execution progress can be delayed to some degree to avoid the memory contentions with other threads, and the CMOS-based registers are still used for threads requiring normal execution speed. Our experimental results show that the proposed technique achieves 30% energy (including both dynamic and leakage) reduction in register files with little performance degradation compared to the baseline case equipped with naive power optimization technique.
---
paper_title: Exploiting uniform vector instructions for GPGPU performance, energy efficiency, and opportunistic reliability enhancement

paper_content:

State-of-art graphics processing units (GPUs) employ the single-instruction multiple-data (SIMD) style execution to achieve both high computational throughput and energy efficiency. As previous works have shown, there exists significant computational redundancy in SIMD execution, where different execution lanes operate on the same operand values. Such value locality is referred to as uniform vectors. In this paper, we first show that besides redundancy within a uniform vector, different vectors can also have the identical values. Then, we propose detailed architecture designs to exploit both types of redundancy. For redundancy within a uniform vector, we propose to either extend the vector register file with token bits or add a separate small scalar register file to eliminate redundant computations as well as redundant data storage. For redundancy across different uniform vectors, we adopt instruction reuse, proposed originally for CPU architectures, to detect and eliminate redundancy. The elimination of redundant computations and data storage leads to both significant energy savings and performance improvement. Furthermore, we propose to leverage such redundancy to protect arithmetic-logic units (ALUs) and register files against hardware errors. Our detailed evaluation shows that our proposed design has low hardware overhead and achieves performance gains, up to 23.9% and 12.0% on average, along with energy savings, up to 24.8% and 12.6% on average, as well as a 21.1% and 14.1% protection coverage for ALUs and register files, respectively.
---
paper_title: Warped register file: A power efficient register file for GPGPUs

paper_content:

General purpose graphics processing units (GPGPUs) have the ability to execute hundreds of concurrent threads. To support massive parallelism GPGPUs provide a very large register file, even larger than a cache, to hold the state of each thread. As technology scales, the leakage power consumption of the SRAM cells is getting worse making the register file static power consumption a major concern. As the supply voltage scaling slows, dynamic power consumption of a register file is not reducing. These concerns are particularly acute in GPGPUs due to their large register file size. This paper presents two techniques to reduce the GPGPU register file power consumption. By exploiting the unique software execution model of GPGPUs, we propose a tri-modal register access control unit to reduce the leakage power. This unit first turns off any unallocated register, and places all allocated registers into drowsy state immediately after each access. The average inter-access distance to a register is 789 cycles in GPGPUs. Hence, aggressively moving a register into drowsy state immediately after each access results in 90% reduction in leakage power with negligible performance impact. To reduce dynamic power this paper proposes an active mask aware activity gating unit that avoids charging bit lines and wordlines of registers associated with all inactive threads within a warp. Due to insufficient parallelism and branch divergence warps have many inactive threads. Hence, registers associated with inactive threads can be identified precisely using the active mask. By combining the two techniques we show that the power consumption of the register file can be reduced by 69% on average.
---
paper_title: Compiler assisted dynamic register file in GPGPU

paper_content:

The large Register File (RF) in General Purpose Graphic Processing Units (GPGPUs) demands tremendous chip area and energy consumption. For a sustainable growth of the size of RF in future GPGPUs, emerging on-chip memory technologies such as embedded-DRAM (eDRAM) have been proposed to replace the conventional SRAM for higher density and lower leakage but with the possible penalty from the periodic refresh operations. This paper explicitly shows that the refresh penalty can be effectively mitigated by leveraging the uniqueness of GPGPU operations. A compiler assisted refresh rescheduling policy can greatly reduce the refresh overhead for maintaining the correctness of the RF operations. The proposed scheme adequately exploits the features in both architecture and compilation, and delivers comparable performance to the SRAM counterpart. At the same time, the energy savings via the removal of large SRAM leakage well compensate for the additional refresh energy. This study promotes the eDRAM-based RF as a promising alternative that enables larger capacity and better power efficiency for future GPGPUs.
---
paper_title: Unifying Primary Cache, Scratch, and Register File Memories in a Throughput Processor

paper_content:

Modern throughput processors such as GPUs employ thousands of threads to drive high-bandwidth, long-latency memory systems. These threads require substantial on-chip storage for registers, cache, and scratchpad memory. Existing designs hard-partition this local storage, fixing the capacities of these structures at design time. We evaluate modern GPU workloads and find that they have widely varying capacity needs across these different functions. Therefore, we propose a unified local memory which can dynamically change the partitioning among registers, cache, and scratchpad on a per-application basis. The tuning that this flexibility enables improves both performance and energy consumption, and broadens the scope of applications that can be efficiently executed on GPUs. Compared to a hard-partitioned design, we show that unified local memory provides a performance benefit as high as 71% along with an energy reduction up to 33%.
---
paper_title: ARGO: aging-aware GPGPU register file allocation

paper_content:

State-of-the-art general-purpose graphic processing units (GPGPUs) implemented in nanoscale CMOS technologies offer very high computational throughput for highly-parallel applications using hundreds of integrated on-chip resources. These resources are stressed during application execution, subjecting them to degradation mechanisms such as negative bias temperature instability (NBTI) that adversely affect their reliability. To support highly parallel execution, GPGPUs contain large register files (RFs) that are among the most highly stressed GPGPU components; however we observe heavy underutilization of RFs (on average only 46%) for typical general-purpose kernels. We present ARGO, an A ging-awa R e G PGPU RF all O cator that opportunistically exploits this RF underutilization by distributing the stress throughout RF. ARGO achieves proper leveling of RF banks through deliberated power-gating of stressful banks. We demonstrate our technique on the AMD Evergreen GPGPU architecture and show that ARGO improves the NBTI-induced threshold voltage degradation by up to 43% (on average 27%), that yields improving RFs static noise margin up to 46% (on average 30%). Furthermore, we estimate a simultaneous reduction in leakage power of 54% by providing sleep states for unused banks.
---
paper_title: Power-efficient computing for compute-intensive GPGPU applications

paper_content:

The peak compute performance of GPUs has been increased by integrating more compute resources and operating them at higher frequency. However, such approaches significantly increase power consumption of GPUs, limiting further performance increase due to the power constraint. Facing such a challenge, we propose three techniques to improve power efficiency and performance of GPUs in this paper. First, we observe that many GPGPU applications are integer-intensive. For such applications, we combine a pair of dependent integer instructions into a composite instruction that can be executed by an enhanced fused multiply-add unit. Second, we observe that computations for many instructions are duplicated across multiple threads. We dynamically detect such instructions and execute them in a separate scalar unit. Finally, we observe that 16 or fewer bits are sufficient for accurate representation of operands and results of many instructions. Thus, we split the 32-bit datapath into two 16-bit datapath slices that can concurrently issue and execute up to two such instructions per cycle. All three proposed techniques can considerably increase utilization of compute resources, improving power efficiency and performance by 20% and 15%, respectively.
---
paper_title: An Energy-Efficient Unified Register File for Mobile GPUs

paper_content:

The programmability of mobile GPUs have raised in recently years, where the shaders inside are instructed by shading programs for realistic 3D effects. The register files for a conventional high throughput multithreaded shader consumes 10% to 20% energy of it. However the register usages of shading program are quite low. In order to reduce the dynamic energy for register file in a multithreaded mobile shader, this paper proposed an unified register file design to reduce both dynamic and leakage energy of it. The result shows that proposed design reduces 85% of dynamic energy in a multithreaded register file. Furthermore, the proposed design reduces 59% of leakage energy and 25% of area with negligible performance degradation. Also, the energy savings in proposed designs are at least 75% more than related work.
---
paper_title: Bank stealing for conflict mitigation in GPGPU Register File

paper_content:

Modern General Purpose Graphic Processing Unit (GPGPU) demands a large Register File (RF), which is typically organized into multiple banks to support the massive parallelism. Although heavy banking benefits RF throughput, its associated area and energy costs with diminishing performance gains greatly limit future RF s-caling. In this paper, we propose an improved RF design with a bank stealing technique, which enables a high RF throughput with compact area. By deeply investigating the GPGPU microarchitecture, we identify the deficiency in the state-of-the-art RF designs as the bank conflict problem, while the majority of conflicts can be eliminated leveraging the fact that the highly-banked RF oftentimes experiences under-utilization. This is especially true in GPGPU where multiple ready warps are available at the scheduling stage with their operands to be wisely coordinated. Our lightweight bank stealing technique can opportunistically fill the idle banks for better operand service, and the average GPGPU performance can be improved under smaller energy budget with significant area saving, which makes it promising for sustainable RF scaling.
---
paper_title: A STT-RAM-based low-power hybrid register file for GPGPUs

paper_content:

Recently, general-purpose graphics processing units (GPGPUs) have been widely used to accelerate computing in various applications. To store the contexts of thousands of concurrent threads on a GPU, a large static random-access memory (SRAM)-based register file is employed. Due to high leakage power of SRAM, the register file consumes 20% to 40% of the total GPU power consumption. Thus, hybrid memory system, which combines SRAM and the emerging non-volatile memory (NVM), has been employed for register file design on GPUs. Although it has shown strong potential to alleviate the power issue of GPUs, existing hybrid memory solutions might not exploit the intrinsic feature of GPU register file. By leveraging the warp schedule on GPU, this paper proposes a hybrid register architecture which consists of a NVM-based register file and mixed SRAM-based write buffers with a warp-aware write back strategy. Simulation results show that our design can eliminate 64% of write accesses to NVM and reduce power of register file by 66% on average, with only 4.2% performance degradation. After we apply the power gating technique, the register power is further reduced to 25% of SRAM counterpart on average.
---
paper_title: ContextPreRF: Enhancing the Performance and Energy of GPUs With Nonuniform Register Access

paper_content:

Register files are a key data storage unit that impacts instruction throughput for graphics processing units (GPUs). Typically, GPU register files are quite large to accommodate many concurrent threads and are implemented using the same SRAM technology as the on-chip cache. We propose contextrf, a new register file architecture that efficiently leverages register files with nonuniform access characteristics, including hybrid SRAM/DRAM (S/D) and spintronic domain-wall memories (DWMs). Contextrf allows greater-capacity register files to be implemented in the same area within the GPU, with reduced power consumption. We also propose contextPreRF, a hardware preswitch scheme to hide switching delays—as soon as a register request is queued, the nonuniform access memories containing the corresponding register are sent a preemptive switch request. Thus, our scheme transparently hides the penalties of switching between register contexts. After replacing the register file SRAM with S/D, we can reduce energy by 37%, with a 1.4% average performance drop. Employing DWM, we reduce register file energy by 74%, with a 0.4% average performance penalty. For the denser DWM, we model converting the saved area into additional registers, cache, and shared memory—this improves performance by 13.5% over the baseline SRAM register file.
---
paper_title: Spare register aware prefetching for graph algorithms on GPUs

paper_content:

More and more graph algorithms are being GPU enabled. Graph algorithm implementations on GPUs have irregular control flow and are memory-intensive with many irregular/data-dependent memory accesses. Due to these factors graph algorithms on GPUs have low execution efficiency. In this work we propose a mechanism to improve the execution efficiency of graph algorithms by improving their memory access latency tolerance. We propose a mechanism for prefetching data for load pairs that have one load dependent on the other - such pairs are common in graph algorithms. Our mechanism detects the target loads in hardware and injects instructions into the pipeline to prefetch data into spare registers that are not being used by any active threads. By prefetching data into registers, early eviction of prefetched data can be eliminated. We also propose a mechanism that uses the compiler to identify the target loads. Our mechanism improves performance over no prefetching by 10% on average and upto 51% for nine memory intensive graph algorithm kernels.
---
paper_title: Soft-error reliability and power co-optimization for GPGPUs register file using resistive memory

paper_content:

The increasing adoption of graphics processing units (GPUs) for high-performance computing raises the reliability challenge, which is generally ignored in traditional GPUs. GPUs usually support thousands of parallel threads and require a sizable register file. Such large register file is highly susceptible to soft errors and power-hungry. Although ECC has been adopted to register file in modern GPUs, it causes considerable power overhead, which further increases the power stress. Thus, an energy-efficient soft-error protection mechanism is more desirable. Besides its extremely low leakage power consumption, resistive memory (e.g. spin-transfer torque RAM) is also immune to the radiation induced soft errors due to its magnetic field based storage. In this paper, we propose to LEverage reSistive memory to enhance the Soft-error robustness and reduce the power consumption (LESS) of registers in the General-Purpose computing on GPUs (GPGPUs). Since resistive memory experiences longer write latency compared to SRAM, we explore the unique characteristics of GPGPU applications to obtain the win-win gains: achieving the near-full soft-error protection for the register file, and meanwhile substantially reducing the energy consumption with negligible performance loss. Our experimental results show that LESS is able to mitigate the registers soft-error vulnerability by 86% and achieve 60% energy savings with negligible (e.g. 4%) performance loss.
---
paper_title: Warped-compression: enabling power efficient GPUs through register compression

paper_content:

This paper presents Warped-Compression, a warp-level register compression scheme for reducing GPU power consumption. This work is motivated by the observation that the register values of threads within the same warp are similar, namely the arithmetic differences between two successive thread registers is small. Removing data redundancy of register values through register compression reduces the effective register width, thereby enabling power reduction opportunities. GPU register files are huge as they are necessary to keep concurrent execution contexts and to enable fast context switching. As a result register file consumes a large fraction of the total GPU chip power. GPU design trends show that the register file size will continue to increase to enable even more thread level parallelism. To reduce register file data redundancy warped-compression uses low-cost and implementation-efficient base-delta-immediate (BDI) compression scheme, that takes advantage of banked register file organization used in GPUs. Since threads within a warp write values with strong similarity, BDI can quickly compress and decompress by selecting either a single register, or one of the register banks, as the primary base and then computing delta values of all the other registers, or banks. Warped-compression can be used to reduce both dynamic and leakage power. By compressing register values, each warp-level register access activates fewer register banks, which leads to reduction in dynamic power. When fewer banks are used to store the register content, leakage power can be reduced by power gating the unused banks. Evaluation results show that register compression saves 25% of the total register file power consumption.
---
paper_title: A compile-time managed multi-level register file hierarchy

paper_content:

As processors increasingly become power limited, performance improvements will be achieved by rearchitecting systems with energy efficiency as the primary design constraint. While some of these optimizations will be hardware based, combined hardware and software techniques likely will be the most productive. This work redesigns the register file system of a modern throughput processor with a combined hardware and software solution that reduces register file energy without harming system performance. Throughput processors utilize a large number of threads to tolerate latency, requiring a large, energy-intensive register file to store thread context. Our results show that a compiler controlled register file hierarchy can reduce register file energy by up to 54%, compared to a hardware only caching approach that reduces register file energy by 34%. We explore register allocation algorithms that are specifically targeted to improve energy efficiency by sharing temporary register file resources across concurrently running threads and conduct a detailed limit study on the further potential to optimize operand delivery for throughput processors. Our efficiency gains represent a direct performance gain for power limited systems, such as GPUs.
---
paper_title: A Write-Aware STTRAM-Based Register File Architecture for GPGPU

paper_content:

The massively parallel processing capacity of GPGPUs requires a large register file (RF), and its size keeps increasing to support more concurrent threads from generation to generation. Using traditional SRAM-based RFs, there are concerns in both area cost and energy consumption, and soon they will become unrealistic. In this work, we analyze the feasibility of using STTRAM-based RF designs, which have benefits in terms of smaller silicon area and zero standby leakage power. However, STTRAM long write latency and high write energy bring new challenges. Therefore, we propose a write-aware STTRAM-based RF architecture (WarRF), which contains two techniques: Split Bank Write modifies the arbitrator design to increase the parallelism of read and write accesses in the same bank; Write Pool reduces the number of repeated write accesses to RFs. Our experiment shows that the performance of STTRAM-based RF is improved by 13p and up to 23p after adopting WarRF. In addition, the energy consumption is reduced by 38p on average compared to SRAM-based RFs.
---
paper_title: Bank stealing for conflict mitigation in GPGPU Register File

paper_content:

Modern General Purpose Graphic Processing Unit (GPGPU) demands a large Register File (RF), which is typically organized into multiple banks to support the massive parallelism. Although heavy banking benefits RF throughput, its associated area and energy costs with diminishing performance gains greatly limit future RF s-caling. In this paper, we propose an improved RF design with a bank stealing technique, which enables a high RF throughput with compact area. By deeply investigating the GPGPU microarchitecture, we identify the deficiency in the state-of-the-art RF designs as the bank conflict problem, while the majority of conflicts can be eliminated leveraging the fact that the highly-banked RF oftentimes experiences under-utilization. This is especially true in GPGPU where multiple ready warps are available at the scheduling stage with their operands to be wisely coordinated. Our lightweight bank stealing technique can opportunistically fill the idle banks for better operand service, and the average GPGPU performance can be improved under smaller energy budget with significant area saving, which makes it promising for sustainable RF scaling.
---
paper_title: GPUWattch: enabling energy optimizations in GPGPUs

paper_content:

General-purpose GPUs (GPGPUs) are becoming prevalent in mainstream computing, and performance per watt has emerged as a more crucial evaluation metric than peak performance. As such, GPU architects require robust tools that will enable them to quickly explore new ways to optimize GPGPUs for energy efficiency. We propose a new GPGPU power model that is configurable, capable of cycle-level calculations, and carefully validated against real hardware measurements. To achieve configurability, we use a bottom-up methodology and abstract parameters from the microarchitectural components as the model's inputs. We developed a rigorous suite of 80 microbenchmarks that we use to bound any modeling uncertainties and inaccuracies. The power model is comprehensively validated against measurements of two commercially available GPUs, and the measured error is within 9.9% and 13.4% for the two target GPUs (GTX 480 and Quadro FX5600). The model also accurately tracks the power consumption trend over time. We integrated the power model with the cycle-level simulator GPGPU-Sim and demonstrate the energy savings by utilizing dynamic voltage and frequency scaling (DVFS) and clock gating. Traditional DVFS reduces GPU energy consumption by 14.4% by leveraging within-kernel runtime variations. More finer-grained SM cluster-level DVFS improves the energy savings from 6.6% to 13.6% for those benchmarks that show clustered execution behavior. We also show that clock gating inactive lanes during divergence reduces dynamic power by 11.2%.
---
paper_title: An Adaptive Thread Scheduling Mechanism With Low-Power Register File for Mobile GPUs

paper_content:

In response to the remarkable increase in 3D applications in consumer electronics devices in recent years, graphics processing units (GPUs) have become widely available on mobile devices. These GPUs typically use hardware multithreaded shaders to improve their throughputs for real-time rendering, but they depend on duplicate register files to maintain the context of each hardware thread, increasing power consumption. However, the register usage of shading programs is often relatively low, which causes many registers to remain unused, thus wasting power. Long latency memory operations can also consume unnecessary power to activate registers. This study proposes a low-power register file with multiple power modes to reduce the power consumption of the register file. This study also presents an adaptive thread scheduling mechanism to achieve a tradeoff between the power consumption of the register file and frames per second (FPS). Results show that the average performance degradation from the proposed low-power register file is only 0.62%. The proposed adaptive thread scheduling has average under prediction ratio of 3.32%. The leakage reduction of the proposed low-power register file is 74.80%. This reduction can be improved to 81.49%, 82.22%, and 84.28% with adaptive thread scheduling at frame rates of 30, 25, and 20, respectively.
---
paper_title: Warped register file: A power efficient register file for GPGPUs

paper_content:

General purpose graphics processing units (GPGPUs) have the ability to execute hundreds of concurrent threads. To support massive parallelism GPGPUs provide a very large register file, even larger than a cache, to hold the state of each thread. As technology scales, the leakage power consumption of the SRAM cells is getting worse making the register file static power consumption a major concern. As the supply voltage scaling slows, dynamic power consumption of a register file is not reducing. These concerns are particularly acute in GPGPUs due to their large register file size. This paper presents two techniques to reduce the GPGPU register file power consumption. By exploiting the unique software execution model of GPGPUs, we propose a tri-modal register access control unit to reduce the leakage power. This unit first turns off any unallocated register, and places all allocated registers into drowsy state immediately after each access. The average inter-access distance to a register is 789 cycles in GPGPUs. Hence, aggressively moving a register into drowsy state immediately after each access results in 90% reduction in leakage power with negligible performance impact. To reduce dynamic power this paper proposes an active mask aware activity gating unit that avoids charging bit lines and wordlines of registers associated with all inactive threads within a warp. Due to insufficient parallelism and branch divergence warps have many inactive threads. Hence, registers associated with inactive threads can be identified precisely using the active mask. By combining the two techniques we show that the power consumption of the register file can be reduced by 69% on average.
---
paper_title: Base-delta-immediate compression: practical data compression for on-chip caches

paper_content:

Cache compression is a promising technique to increase on-chip cache capacity and to decrease on-chip and off-chip bandwidth usage. Unfortunately, directly applying well-known compression algorithms (usually implemented in software) leads to high hardware complexity and unacceptable decompression/compression latencies, which in turn can negatively affect performance. Hence, there is a need for a simple yet efficient compression technique that can effectively compress common in-cache data patterns, and has minimal effect on cache access latency.
---
paper_title: Warped-compression: enabling power efficient GPUs through register compression

paper_content:

This paper presents Warped-Compression, a warp-level register compression scheme for reducing GPU power consumption. This work is motivated by the observation that the register values of threads within the same warp are similar, namely the arithmetic differences between two successive thread registers is small. Removing data redundancy of register values through register compression reduces the effective register width, thereby enabling power reduction opportunities. GPU register files are huge as they are necessary to keep concurrent execution contexts and to enable fast context switching. As a result register file consumes a large fraction of the total GPU chip power. GPU design trends show that the register file size will continue to increase to enable even more thread level parallelism. To reduce register file data redundancy warped-compression uses low-cost and implementation-efficient base-delta-immediate (BDI) compression scheme, that takes advantage of banked register file organization used in GPUs. Since threads within a warp write values with strong similarity, BDI can quickly compress and decompress by selecting either a single register, or one of the register banks, as the primary base and then computing delta values of all the other registers, or banks. Warped-compression can be used to reduce both dynamic and leakage power. By compressing register values, each warp-level register access activates fewer register banks, which leads to reduction in dynamic power. When fewer banks are used to store the register content, leakage power can be reduced by power gating the unused banks. Evaluation results show that register compression saves 25% of the total register file power consumption.
---
paper_title: Power Modeling for GPU Architectures Using McPAT

paper_content:

Graphics Processing Units (GPUs) are very popular for both graphics and general-purpose applications. Since GPUs operate many processing units and manage multiple levels of memory hierarchy, they consume a significant amount of power. Although several power models for CPUs are available, the power consumption of GPUs has not been studied much yet. In this article we develop a new power model for GPUs by utilizing McPAT, a CPU power tool. We generate initial power model data from McPAT with a detailed GPU configuration, and then adjust the models by comparing them with empirical data. We use the NVIDIA's Fermi architecture for building the power model, and our model estimates the GPU power consumption with an average error of 7.7p and 12.8p for the microbenchmarks and Merge benchmarks, respectively.
---
paper_title: Architecture supported register stash for GPGPU

paper_content:

GPGPU provides abundant hardware resources to support a large number of light-weighted threads. They are organized into blocks and run in warps. All threads of a block must be dispatched to one stream multiprocessor (SM) of GPGPU together. When the remaining resources of an SM cannot support one more block, all threads of the block are held back until former blocks retire from the SM. We found that the register file is prone to be the most limited one among all the resources, especially for SMs with less registers. Meanwhile, we revealed the dynamics of a thread's register requirement: only part of its pre-allocated registers are used for different instructions at run time. This results in considerable register underutilization.We proposed the architecture supported register stash (ASRS). It removes the limitation of registers when dispatching blocks. The hardware registers are allocated at run time according to each instruction's live registers, which can be analyzed statically by a compiler. When the hardware registers cannot meet the requirements of all running warps, some warps are suspended and their registers are reclaimed temporarily. The data in these registers are stashed to memory. On the other hand, if there are spare hardware registers, it will start a new warp or resume a suspended warp after all the warp's stashed register data are loaded from memory. The intra-block synchronization is also taken care of when some of the warps of the same block are not schedulable due to the ASRS.The ASRS alleviates the register underutilization and improves performance without modifying the current programming model or demanding extra effort from the programmers. It also enables an SM with limited registers that cannot even support a single block to execute it. Besides, it helps lower the register file energy consumption and increase the power efficiency. The ASRS achieved speedups of 1.59 and 1.14 when the registers of each SM are limited to 8K and 16K respectively with an insignificant overhead. The speedups compared with the infinite register files are 0.84 and 0.98 with 8K and 16K registers respectively. Compared with the baseline 32K register file, the ASRS decreases the 8K and 16K register file energy consumption to 66.5% and 75.8% respectively. Their power efficiencies (in ratio of performance and power) are increased to 1.29x?and 1.31x?respectively. Register requirement of GPGPU varies among different kernels and during run time.Register file (RF) capacity limits the schedulable warps and performance.Reducing RF capacity lowers energy consumption and area.We proposed a method to support more warps with limited registers.It gains significant speedup and a higher energy efficiency with a smaller RF.
---
paper_title: GPGPU Register File Management by Hardware Co-operated Register Reallocation

paper_content:

To support massive parallel threads context, GPGPUs use a huge register file. Due to their size, register file is one of the most power hungry logic in GPGPU. However, the current trends indicate that GPGPU register file size will continue to get even bigger as the demand for higher single instruction multiple thread (SIMT) parallelism increases, particularly in high performance application domain. In order to reduce power consumption demand, in this work, we exploit a fundamental observation that fairly high portion of the register spaces are unnecessarily allocated and burn power due to the fact that the registers are considered as private resource for each warp. For example, even though a register’s value is no longer used by any instruction, the register space should be occupied by the warp during the program execution. In GPGPU, as single register usage in a code context leads to thousands of register space allocation, the power and space overhead due to any wasted register is significant. Instead, we propose to share the register file across warps. In our proposed allocation, a register is released from its physical register space immediately after its last use. Then, the released register space is reassigned to another warp’s register. The compile time register lifetime analysis information is used for providing hint to the hardware about each register’s release point. To enable this register reassignment, we propose a light weight register renaming in hardware. By releasing registers and reusing them across warps, we can reduce the demand for register file size on average by 30% compared with the optimally compiled applications. The reduced live register space leads to an average of 23% and 26% static power saving over a basic sub-array level and individual register level power gating.
---
paper_title: Exploiting uniform vector instructions for GPGPU performance, energy efficiency, and opportunistic reliability enhancement

paper_content:

State-of-art graphics processing units (GPUs) employ the single-instruction multiple-data (SIMD) style execution to achieve both high computational throughput and energy efficiency. As previous works have shown, there exists significant computational redundancy in SIMD execution, where different execution lanes operate on the same operand values. Such value locality is referred to as uniform vectors. In this paper, we first show that besides redundancy within a uniform vector, different vectors can also have the identical values. Then, we propose detailed architecture designs to exploit both types of redundancy. For redundancy within a uniform vector, we propose to either extend the vector register file with token bits or add a separate small scalar register file to eliminate redundant computations as well as redundant data storage. For redundancy across different uniform vectors, we adopt instruction reuse, proposed originally for CPU architectures, to detect and eliminate redundancy. The elimination of redundant computations and data storage leads to both significant energy savings and performance improvement. Furthermore, we propose to leverage such redundancy to protect arithmetic-logic units (ALUs) and register files against hardware errors. Our detailed evaluation shows that our proposed design has low hardware overhead and achieves performance gains, up to 23.9% and 12.0% on average, along with energy savings, up to 24.8% and 12.6% on average, as well as a 21.1% and 14.1% protection coverage for ALUs and register files, respectively.
---
paper_title: Power-efficient computing for compute-intensive GPGPU applications

paper_content:

The peak compute performance of GPUs has been increased by integrating more compute resources and operating them at higher frequency. However, such approaches significantly increase power consumption of GPUs, limiting further performance increase due to the power constraint. Facing such a challenge, we propose three techniques to improve power efficiency and performance of GPUs in this paper. First, we observe that many GPGPU applications are integer-intensive. For such applications, we combine a pair of dependent integer instructions into a composite instruction that can be executed by an enhanced fused multiply-add unit. Second, we observe that computations for many instructions are duplicated across multiple threads. We dynamically detect such instructions and execute them in a separate scalar unit. Finally, we observe that 16 or fewer bits are sufficient for accurate representation of operands and results of many instructions. Thus, we split the 32-bit datapath into two 16-bit datapath slices that can concurrently issue and execute up to two such instructions per cycle. All three proposed techniques can considerably increase utilization of compute resources, improving power efficiency and performance by 20% and 15%, respectively.
---
paper_title: An eDRAM-Based Approximate Register File for GPUs

paper_content:

GPUs require large register files for fast context switching. This paper presents a high-density and energy-efficient approximate register file architecture based on embedded DRAM with lowered refresh rate to the low-order bits of register entries.
---
paper_title: Register Caching for Stencil Computations on GPUs

paper_content:

For most applications, taking full advantage of the memory system is key to achieving good performance on GPUs. In this paper, we introduce register caching, a novel idea where registers of multiple threads are combined and used as a shared, last level, manually managed cache for the contributing threads. This method is enabled by the shuffle instruction recently introduced in Nvidia's Kepler GPU architecture, which allows threads in the same warp to exchange data directly, previously only possible by going through shared memory. We evaluate our proposal with a stencil computation benchmark, achieving speedups of up to 2.04, compared to using shared memory on a GTX680 GPU. Stencil computations form the core of many scientific applications, which can therefore benefit from our proposal. Furthermore, our method is not limited to stencil computations, but is applicable to any application with a predictable memory access pattern suitable for manual caching.
---
paper_title: Spare register aware prefetching for graph algorithms on GPUs

paper_content:

More and more graph algorithms are being GPU enabled. Graph algorithm implementations on GPUs have irregular control flow and are memory-intensive with many irregular/data-dependent memory accesses. Due to these factors graph algorithms on GPUs have low execution efficiency. In this work we propose a mechanism to improve the execution efficiency of graph algorithms by improving their memory access latency tolerance. We propose a mechanism for prefetching data for load pairs that have one load dependent on the other - such pairs are common in graph algorithms. Our mechanism detects the target loads in hardware and injects instructions into the pipeline to prefetch data into spare registers that are not being used by any active threads. By prefetching data into registers, early eviction of prefetched data can be eliminated. We also propose a mechanism that uses the compiler to identify the target loads. Our mechanism improves performance over no prefetching by 10% on average and upto 51% for nine memory intensive graph algorithm kernels.
---
paper_title: Precision-aware soft error protection for GPUs

paper_content:

With the advent of general-purpose GPU computing, it is becoming increasingly desirable to protect GPUs from soft errors. For high computation throughout, GPUs must store a significant amount of state and have many execution units. The high power and area costs of full protection from soft errors make selective protection techniques attractive. Such approaches provide maximum error coverage within a fixed area or power limit, but typically treat all errors equally. We observe that for many floating-point-intensive GPGPU applications, small magnitude errors may have little effect on results, while large magnitude errors can be amplified to have a significant negative impact. We therefore propose a novel precision-aware protection approach for the GPU execution logic and register file to mitigate large magnitude errors. We also propose an architecture modification to optimize error coverage for integer computations. Our approach combines selective logic hardening, targeted checker circuits, and intelligent register file encoding for best error protection. We demonstrate that our approach can reduce the mean error magnitude by up to 87% compared to a traditional selective protection approach with the same overhead.
---
paper_title: A Survey of Techniques for Modeling and Improving Reliability of Computing Systems

paper_content:

Recent trends of aggressive technology scaling have greatly exacerbated the occurrences and impact of faults in computing systems. This has made ‘reliability’ a first-order design constraint. To address the challenges of reliability, several techniques have been proposed. This paper provides a survey of architectural techniques for improving resilience of computing systems. We especially focus on techniques proposed for microarchitectural components, such as processor registers, functional units, cache and main memory etc. In addition, we discuss techniques proposed for non-volatile memory, GPUs and 3D-stacked processors. To underscore the similarities and differences of the techniques, we classify them based on their key characteristics. We also review the metrics proposed to quantify vulnerability of processor structures. We believe that this survey will help researchers, system-architects and processor designers in gaining insights into the techniques for improving reliability of computing systems.
---
paper_title: Exploiting uniform vector instructions for GPGPU performance, energy efficiency, and opportunistic reliability enhancement

paper_content:

State-of-art graphics processing units (GPUs) employ the single-instruction multiple-data (SIMD) style execution to achieve both high computational throughput and energy efficiency. As previous works have shown, there exists significant computational redundancy in SIMD execution, where different execution lanes operate on the same operand values. Such value locality is referred to as uniform vectors. In this paper, we first show that besides redundancy within a uniform vector, different vectors can also have the identical values. Then, we propose detailed architecture designs to exploit both types of redundancy. For redundancy within a uniform vector, we propose to either extend the vector register file with token bits or add a separate small scalar register file to eliminate redundant computations as well as redundant data storage. For redundancy across different uniform vectors, we adopt instruction reuse, proposed originally for CPU architectures, to detect and eliminate redundancy. The elimination of redundant computations and data storage leads to both significant energy savings and performance improvement. Furthermore, we propose to leverage such redundancy to protect arithmetic-logic units (ALUs) and register files against hardware errors. Our detailed evaluation shows that our proposed design has low hardware overhead and achieves performance gains, up to 23.9% and 12.0% on average, along with energy savings, up to 24.8% and 12.6% on average, as well as a 21.1% and 14.1% protection coverage for ALUs and register files, respectively.
---
paper_title: An energy-efficient and scalable eDRAM-based register file architecture for GPGPU

paper_content:

The heavily-threaded data processing demands of streaming multiprocessors (SM) in a GPGPU require a large register file (RF). The fast increasing size of the RF makes the area cost and power consumption unaffordable for traditional SRAM designs in the future technologies. In this paper, we propose to use embedded-DRAM (eDRAM) as an alternative in future GPGPUs. Compared with SRAM, eDRAM provides higher density and lower leakage power. However, the limited data retention time in eDRAM poses new challenges. Periodic refresh operations are needed to maintain data integrity. This is exacerbated with the scaling of eDRAM density, process variations and temperature. Unlike conventional CPUs which make use of multi-ported RF, most of the RFs in modern GPGPU are heavily banked but not multi-ported to reduce the hardware cost. This provides a unique opportunity to hide the refresh overhead. We propose two different eDRAM implementations based on 3T1D and 1T1C memory cells. To mitigate the impact of periodic refresh, we propose two novel refresh solutions using bank bubble and bank walk-through. Plus, for the 1T1C RF, we design an interleaved bank organization together with an intelligent warp scheduling strategy to reduce the impact of the destructive reads. The analysis shows that our schemes present better energy efficiency, scalability and variation tolerance than traditional SRAM-based designs.
---
paper_title: Modeling and characterizing GPGPU reliability in the presence of soft errors

paper_content:

Abstract The general-purpose computing on graphic processing units (GPGPUs) becomes increasingly popular due to its high computational throughput for data parallel applications. Modern GPU architectures have limited capability for error detection and fault tolerance since they are originally designed for graphics processing. However, the rigorous execution correctness is required for general-purpose applications, which makes reliability a growing concern in the GPGPU architecture design. With CMOS processing technologies continuously scaling down to the nano-scale, on-chip soft error rate (SER) has been predicted to increase exponentially. GPGPUs with hundreds of cores integrated into a single chip are prone to manifest high SER. This paper explores a first step to model and characterize GPGPU reliability in light of soft errors. We develop GPGPU-SODA (GPGPU SOftware Dependability Analysis), a framework to estimate the soft-error vulnerability of GPGPU microarchitecture. By using GPGPU-SODA, we observe that several microarchitecture structures in GPGPUs exhibit high soft-error susceptibility, and the structure vulnerability is sensitive to the workload characteristics (e.g. branch divergences, memory access pattern). We further investigate the impact of several architectural optimizations on GPU soft-error robustness. For example, we find that increasing the number of threads supported by GPU significantly affects the GPGPU soft-error robustness. However, changing the warp scheduling policy has little impact on the structure vulnerability. The observations made in this study provide designers the useful guidance to build resilient GPGPUs: a comprehensive resiliency solution for GPGPUs should consider the entire GPGPU design instead of solely focusing on a particular structure.
---
paper_title: Precision-aware soft error protection for GPUs

paper_content:

With the advent of general-purpose GPU computing, it is becoming increasingly desirable to protect GPUs from soft errors. For high computation throughout, GPUs must store a significant amount of state and have many execution units. The high power and area costs of full protection from soft errors make selective protection techniques attractive. Such approaches provide maximum error coverage within a fixed area or power limit, but typically treat all errors equally. We observe that for many floating-point-intensive GPGPU applications, small magnitude errors may have little effect on results, while large magnitude errors can be amplified to have a significant negative impact. We therefore propose a novel precision-aware protection approach for the GPU execution logic and register file to mitigate large magnitude errors. We also propose an architecture modification to optimize error coverage for integer computations. Our approach combines selective logic hardening, targeted checker circuits, and intelligent register file encoding for best error protection. We demonstrate that our approach can reduce the mean error magnitude by up to 87% compared to a traditional selective protection approach with the same overhead.
---
paper_title: Mitigating the Susceptibility of GPGPUs Register File to Process Variations

paper_content:

As technology keeps scaling down at nano-scale, the increasing process variations (PV) induce significant delay variations and limit the maximum clock frequency in GPGPUs (general-purpose computing on graphics processing units). Each computing core (i.e. streaming multiprocessor) in GPGPUs supports thousands of simultaneously active threads, and requires a large register file. Such a sizeable register file is very sensitive to process variations, and becomes one of the major units in determining the core frequency. In this study, we first develop a novel mechanism that classifies registers into fast and slow categories in the highly-banked register architecture to maximize the frequency improvement. We then leverage the unique features in GPGPU applications to effectively tolerate the extra access delay to the slow registers. Our experimental results show that our proposed techniques are able to significantly optimize GPGPUs performance under process variations.
---
paper_title: ARGO: aging-aware GPGPU register file allocation

paper_content:

State-of-the-art general-purpose graphic processing units (GPGPUs) implemented in nanoscale CMOS technologies offer very high computational throughput for highly-parallel applications using hundreds of integrated on-chip resources. These resources are stressed during application execution, subjecting them to degradation mechanisms such as negative bias temperature instability (NBTI) that adversely affect their reliability. To support highly parallel execution, GPGPUs contain large register files (RFs) that are among the most highly stressed GPGPU components; however we observe heavy underutilization of RFs (on average only 46%) for typical general-purpose kernels. We present ARGO, an A ging-awa R e G PGPU RF all O cator that opportunistically exploits this RF underutilization by distributing the stress throughout RF. ARGO achieves proper leveling of RF banks through deliberated power-gating of stressful banks. We demonstrate our technique on the AMD Evergreen GPGPU architecture and show that ARGO improves the NBTI-induced threshold voltage degradation by up to 43% (on average 27%), that yields improving RFs static noise margin up to 46% (on average 30%). Furthermore, we estimate a simultaneous reduction in leakage power of 54% by providing sleep states for unused banks.
---
paper_title: An energy-efficient and scalable eDRAM-based register file architecture for GPGPU

paper_content:

The heavily-threaded data processing demands of streaming multiprocessors (SM) in a GPGPU require a large register file (RF). The fast increasing size of the RF makes the area cost and power consumption unaffordable for traditional SRAM designs in the future technologies. In this paper, we propose to use embedded-DRAM (eDRAM) as an alternative in future GPGPUs. Compared with SRAM, eDRAM provides higher density and lower leakage power. However, the limited data retention time in eDRAM poses new challenges. Periodic refresh operations are needed to maintain data integrity. This is exacerbated with the scaling of eDRAM density, process variations and temperature. Unlike conventional CPUs which make use of multi-ported RF, most of the RFs in modern GPGPU are heavily banked but not multi-ported to reduce the hardware cost. This provides a unique opportunity to hide the refresh overhead. We propose two different eDRAM implementations based on 3T1D and 1T1C memory cells. To mitigate the impact of periodic refresh, we propose two novel refresh solutions using bank bubble and bank walk-through. Plus, for the 1T1C RF, we design an interleaved bank organization together with an intelligent warp scheduling strategy to reduce the impact of the destructive reads. The analysis shows that our schemes present better energy efficiency, scalability and variation tolerance than traditional SRAM-based designs.
---
paper_title: Mitigating the Susceptibility of GPGPUs Register File to Process Variations

paper_content:

As technology keeps scaling down at nano-scale, the increasing process variations (PV) induce significant delay variations and limit the maximum clock frequency in GPGPUs (general-purpose computing on graphics processing units). Each computing core (i.e. streaming multiprocessor) in GPGPUs supports thousands of simultaneously active threads, and requires a large register file. Such a sizeable register file is very sensitive to process variations, and becomes one of the major units in determining the core frequency. In this study, we first develop a novel mechanism that classifies registers into fast and slow categories in the highly-banked register architecture to maximize the frequency improvement. We then leverage the unique features in GPGPU applications to effectively tolerate the extra access delay to the slow registers. Our experimental results show that our proposed techniques are able to significantly optimize GPGPUs performance under process variations.
---
paper_title: A Survey of CPU-GPU Heterogeneous Computing Techniques

paper_content:

As both CPUs and GPUs become employed in a wide range of applications, it has been acknowledged that both of these Processing Units (PUs) have their unique features and strengths and hence, CPU-GPU collaboration is inevitable to achieve high-performance computing. This has motivated a significant amount of research on heterogeneous computing techniques, along with the design of CPU-GPU fused chips and petascale heterogeneous supercomputers. In this article, we survey Heterogeneous Computing Techniques (HCTs) such as workload partitioning that enable utilizing both CPUs and GPUs to improve performance and/or energy efficiency. We review heterogeneous computing approaches at runtime, algorithm, programming, compiler, and application levels. Further, we review both discrete and fused CPU-GPU systems and discuss benchmark suites designed for evaluating Heterogeneous Computing Systems (HCSs). We believe that this article will provide insights into the workings and scope of applications of HCTs to researchers and motivate them to further harness the computational powers of CPUs and GPUs to achieve the goal of exascale performance.
---

Title: A Survey of Techniques for Architecting and Managing GPU Register File

Section 1: INTRODUCTION
Description 1: Provide an introduction to the evolution of GPU architecture, the importance of register file (RF) management, and the objectives of the survey.

Section 2: Achieving High Energy Efficiency
Description 2: Discuss techniques aimed at designing GPU RF with high-speed transistors and achieving energy efficiency.

Section 3: Achieving High Bandwidth and Throughput
Description 3: Explain methods to provide high throughput and concurrent access capabilities in GPU RF.

Section 4: Achieving High Capacity
Description 4: Review approaches for architecting large-capacity RFs within limited area budgets.

Section 5: Achieving High Resource-Utilization
Description 5: Cover strategies to avoid under-utilization and inefficiencies in RF resources.

Section 6: Achieving a Balance with Other Objectives
Description 6: Delve into balancing RF size with other optimization considerations and constraints.

Section 7: Achieving High Reliability
Description 7: Discuss various reliability issues of GPU RF and techniques for mitigating their impact.

Section 8: GPU Evolution Trends
Description 8: Analyze the trends in GPU evolution and their implications on RF management techniques.

Section 9: TECHNIQUES FOR ARCHITECTING RF USING NOVEL MEMORY TECHNOLOGIES
Description 9: Explore different memory technologies used for designing GPU RF and techniques enabling their use.

Section 10: PERFORMANCE AND ENERGY EFFICIENCY IMPROVEMENT TECHNIQUES FOR RF
Description 10: Summarize key ideas and classify techniques focused on improving performance and energy efficiency of RF.

Section 11: Techniques for Reorganizing RF Architecture
Description 11: Explain methods for reducing RF accesses and energy consumption through architectural reorganizations.

Section 12: Techniques for RF Access Scheduling
Description 12: Detail scheduling techniques aimed at addressing latency and energy consumption limitations in RF.

Section 13: Techniques for Managing RF Power Consumption
Description 13: Discuss strategies for reducing power consumption in GPU RFs.

Section 14: Techniques for Register Sharing
Description 14: Examine methods to share registers across warps to improve performance and resource utilization.

Section 15: Techniques for Leveraging Redundancy in GPU Execution
Description 15: Highlight techniques that exploit redundancy within and across warps to improve performance and energy efficiency.

Section 16: Approximation Techniques for RF
Description 16: Review approximation techniques to save power while maintaining acceptable output quality.

Section 17: Techniques for Using RF as Cache or Shared Memory
Description 17: Discuss using RF as cache or shared memory for specific applications to improve performance.

Section 18: RELIABILITY IMPROVEMENT TECHNIQUES FOR RF
Description 18: Summarize different techniques that enhance RF reliability.

Section 19: Mitigating Soft-Error Impact on RF
Description 19: Provide an overview of methods to reduce the impact of soft-errors on GPU RF.

Section 20: Mitigating NBTI Impact on RF
Description 20: Discuss techniques for mitigating the impact of negative bias temperature instability (NBTI) on RF.

Section 21: Mitigating Process Variation Impact on RF
Description 21: Review strategies for addressing the impact of process variations on GPU RF.

Section 22: CONCLUSION AND FUTURE OUTLOOK
Description 22: Conclude the survey with a summary and discuss future research directions in GPU RF management.