A Survey Of Architectural Approaches for Managing Embedded DRAM and Non-Volatile On-Chip Caches
---
paper_title: POWER7: IBM's next generation server processor

paper_content:

The Power7 is IBM's first eight-core processor, with each core capable of four-way simultaneous-multithreading operation. Its key architectural features include an advanced memory hierarchy with three levels of on-chip cache; embedded-DRAM devices used in the highest level of the cache; and a new memory interface. This balanced multicore design scales from 1 to 32 sockets in commercial and scientific environments.
---
paper_title: Scaling the bandwidth wall: challenges in and avenues for CMP scaling

paper_content:

As transistor density continues to grow at an exponential rate in accordance to Moore's law, the goal for many Chip Multi-Processor (CMP) systems is to scale the number of on-chip cores proportionally. Unfortunately, off-chip memory bandwidth capacity is projected to grow slowly compared to the desired growth in the number of cores. This creates a situation in which each core will have a decreasing amount of off-chip bandwidth that it can use to load its data from off-chip memory. The situation in which off-chip bandwidth is becoming a performance and throughput bottleneck is referred to as the bandwidth wall problem. In this study, we seek to answer two questions: (1) to what extent does the bandwidth wall problem restrict future multicore scaling, and (2) to what extent are various bandwidth conservation techniques able to mitigate this problem. To address them, we develop a simple but powerful analytical model to predict the number of on-chip cores that a CMP can support given a limited growth in memory traffic capacity. We find that the bandwidth wall can severely limit core scaling. When starting with a balanced 8-core CMP, in four technology generations the number of cores can only scale to 24, as opposed to 128 cores under proportional scaling, without increasing the memory traffic requirement. We find that various individual bandwidth conservation techniques we evaluate have a wide ranging impact on core scaling, and when combined together, these techniques have the potential to enable super-proportional core scaling for up to 4 technology generations.
---
paper_title: An energy efficient cache design using spin torque transfer (STT) RAM

paper_content:

The on-chip memory is a dominant source of power and energy consumption in modern and future processors. This paper explores the use of a new emerging non-volatile memory technology as a replacement for SRAM based lower level caches - Spin Torque Transfer(STT) RAM. While STTRAM achieves a reduction in leakage energy of 90% compared to SRAM, the dynamic energy for a write operation is 2X that of SRAM. Consequently, we propose additional microarchitectural optimizations to reduce overall dynamic energy which achieve an average reduction in dynamic energy over the base case of 30% with a range of 16% to 60% across 10 benchmarks.
---
paper_title: Embedded DRAM: technology platform for the Blue Gene/L chip

paper_content:

The Blue Gene®/L chip is a technological tour de force that embodies the system-on-a-chip concept in its entirety. This paper outlines the salient features of this 130-nm complementary metal oxide semiconductor (CMOS) technology, including the IBM unique embedded dynamic random access memory (DRAM) technology. Crucial to the execution of Blue Gene/L is the simultaneous instantiation of multiple PowerPC® cores, high-performance static random access memory (SRAM), DRAM, and several other logic design blocks on a single-platform technology. The IBM embedded DRAM platform allows this seamless integration without compromising performance, reliability, or yield. We discuss the process architecture, the key parameters of the logic components used in the processor cores and other logic design blocks, the SRAM features used in the L2 cache, and the embedded DRAM that forms the L3 cache. We also discuss the evolution of embedded DRAM technology into a higher-performance space in the 90-nm and 65-nm nodes and the potential for dynamic memory to improve overall memory subsystem performance.
---
paper_title: 5.9 Haswell: A family of IA 22nm processors

paper_content:

The 4th Generation Intel® Core™ processor, codenamed Haswell, is a family of products implemented on Intel 22nm Tri-gate process technology [1]. The primary goals for the Haswell program are platform integration and low power to enable smaller form factors. Haswell incorporates several building blocks, including: platform controller hubs (PCHs), memory, CPU, graphics and media processing engines, thus creating a portfolio of product segments from fan-less Ultrabooks™ to high-performance desktop, as shown in Fig. 5.9.1. It also integrates a number of new technologies: a fully integrated voltage regulator (VR) consolidating 5 platform VRs down to 1, on-die eDRAM cache for improved graphics performance, lower-power states, optimized IO interfaces, an Intel AVX2 instruction set that supports floating-point multiply-add (FMA), and 256b SIMD integer achieving 2× the number of floating-point and integer operations over its predecessor. The 22nm process is optimized for Haswell and includes 11 metal layers (2 additional metal layers vs. Ivy Bridge [2]), high-density metal-insulator-metal (MIM) capacitors, and is tuned for different leakage/speed targets based on the market segment. For example, in some low-power products, the process is optimized to reduce leakage by 75% at Vmin, while paying only 12% intrinsic device degradation at the high-voltage corner.
---
paper_title: Technology comparison for large last-level caches (L3Cs): Low-leakage SRAM, low write-energy STT-RAM, and refresh-optimized eDRAM

paper_content:

Large last-level caches (L3Cs) are frequently used to bridge the performance and power gap between processor and memory. Although traditional processors implement caches as SRAMs, technologies such as STT-RAM (MRAM), and eDRAM have been used and/or considered for the implementation of L3Cs. Each of these technologies has inherent weaknesses: SRAM is relatively low density and has high leakage current; STT-RAM has high write latency and write energy consumption; and eDRAM requires refresh operations. As future processors are expected to have larger last-level caches, the goal of this paper is to study the trade-offs associated with using each of these technologies to implement L3Cs. In order to make useful comparisons between SRAM, STTRAM, and eDRAM L3Cs, we model them in detail and apply low power techniques to each of these technologies to address their respective weaknesses. We optimize SRAM for low leakage and optimize STT-RAM for low write energy. Moreover, we classify eDRAM refresh-reduction schemes into two categories and demonstrate the effectiveness of using dead-line prediction to eliminate unnecessary refreshes. A comparison of these technologies through full-system simulation shows that the proposed refresh-reduction method makes eDRAM a viable, energy-efficient technology for implementing L3Cs.
---
paper_title: Reducing cache power with low-cost, multi-bit error-correcting codes

paper_content:

Technology advancements have enabled the integration of large on-die embedded DRAM (eDRAM) caches. eDRAM is significantly denser than traditional SRAMs, but must be periodically refreshed to retain data. Like SRAM, eDRAM is susceptible to device variations, which play a role in determining refresh time for eDRAM cells. Refresh power potentially represents a large fraction of overall system power, particularly during low-power states when the CPU is idle. Future designs need to reduce cache power without incurring the high cost of flushing cache data when entering low-power states. In this paper, we show the significant impact of variations on refresh time and cache power consumption for large eDRAM caches. We propose Hi-ECC, a technique that incorporates multi-bit error-correcting codes to significantly reduce refresh rate. Multi-bit error-correcting codes usually have a complex decoder design and high storage cost. Hi-ECC avoids the decoder complexity by using strong ECC codes to identify and disable sections of the cache with multi-bit failures, while providing efficient single-bit error correction for the common case. Hi-ECC includes additional optimizations that allow us to amortize the storage cost of the code over large data words, providing the benefit of multi-bit correction at same storage cost as a single-bit error-correcting (SECDED) code (2% overhead). Our proposal achieves a 93% reduction in refresh power vs. a baseline eDRAM cache without error correcting capability, and a 66% reduction in refresh power vs. a system using SECDED codes.
---
paper_title: Versatile refresh: low complexity refresh scheduling for high-throughput multi-banked eDRAM

paper_content:

Multi-banked embedded DRAM (eDRAM) has become increasingly popular in high-performance systems. However, the data retention problem of eDRAM is exacerbated by the larger number of banks and the high-performance environment in which it is deployed: The data retention time of each memory cell decreases while the number of cells to be refreshed increases. For this, multi-bank designs offer a concurrent refresh mode, where idle banks can be refreshed concurrently during read and write operations. However, conventional techniques such as periodically scheduling refreshes---with priority given to refreshes in case of conflicts with reads or writes---have variable performance, increase read latency, and can perform poorly in worst case memory access patterns. We propose a novel refresh scheduling algorithm that is low-complexity, produces near-optimal throughput with universal guarantees, and is tolerant to bursty memory access patterns. The central idea is to decouple the scheduler into two simple-to-implement modules: one determines which cell to refresh next and the other determines when to force an idle cycle in all banks. We derive necessary and sufficient conditions to guarantee data integrity for all access patterns, with any given number of banks, rows per bank, read/write ports and data retention time. Our analysis shows that there is a tradeoff between refresh overhead and burst tolerance and characterizes this tradeoff precisely. The algorithm is shown to be near-optimal and achieves, for instance, 76.6% reduction in worst-case refresh overhead from the periodic refresh algorithm for a 250MHz eDRAM with 10us retention time and 16 banks each with 128 rows. Simulations with Apex-Map synthetic benchmarks and switch lookup table traffic show that VR can almost completely hide the refresh overhead for memory accesses with moderate-to-high multiplexing across memory banks.
---
paper_title: Refrint: Intelligent refresh to minimize power in on-chip multiprocessor cache hierarchies

paper_content:

As manycores use dynamic energy ever more efficiently, static power consumption becomes a major concern. In particular, in a large manycore running at a low voltage, leakage in on-chip memory modules contributes substantially to the chip's power draw. This is unfortunate, given that, intuitively, the large multi-level cache hierarchy of a manycore is likely to contain a lot of useless data. An effective way to reduce this problem is to use a low-leakage technology such as embedded DRAM (eDRAM). However, eDRAM requires refresh. In this paper, we examine the opportunity of minimizing on-chip memory power further by intelligently refreshing on-chip eDRAM. We present Refrint, a simple approach to perform fine-grained, intelligent refresh of on-chip eDRAM multiprocessor cache hierarchies. We introduce the Refrint algorithms and microarchitecture. We evaluate Refrint in a simulated manycore running 16-threaded parallel applications. We show that an eDRAM-based memory hierarchy with Refrint consumes only 30% of the energy of a conventional SRAM-based memory hierarchy, and induces a slowdown of only 6%. In contrast, an eDRAM-based memory hierarchy without Refrint consumes 56% of the energy of the conventional memory hierarchy, inducing a slowdown of 25%.
---
paper_title: A 500 MHz Random Cycle, 1.5 ns Latency, SOI Embedded DRAM Macro Featuring a Three-Transistor Micro Sense Amplifier

paper_content:

-As microprocessors enter the highly multi-core/multi-threaded era, higher density, lower latency embedded memory will be required to meet cache design needs. This paper describes a 500MHz random cycle Silicon on Insulator (SOI) embedded DRAM macro which features a three-transistor micro sense amplifier, realizing significant performance gains over traditional array design methods. To address the realities of process integration, we describe the features and issues associated with integrating this DRAM into SOI technology, including deep trench processing and floating body effects. After a brief description of the macro architecture, details are provided on the three-transistor micro sense amplifier scheme, which is key to achieving a high transfer ratio with minimal area overhead. The paper concludes with hardware results and a summary.
---
paper_title: Bandwidth-aware reconfigurable cache design with hybrid memory technologies

paper_content:

In chip-multiprocessor (CMP) designs, limited memory bandwidth is a potential bottleneck of the system performance. New memory technologies, such as spin-torque-transfer memory (STT-RAM), resistive memory (RRAM), and embedded DRAM (eDRAM), are promising on-chip memory solutions for CMPs. In this paper, we propose a bandwidth-aware re-configurable cache hierarchy (BARCH) with hybrid memory technologies. BARCH consists of a hybrid cache hierarchy, a reconfiguration mechanism, and a statistical prediction engine. Our hybrid cache hierarchy chooses different memory technologies to configure each level so that the bandwidth provided by the overall hierarchy is optimized. Furthermore, we present a reconfiguration mechanism to dynamically adapt the cache space of each level based on the predicted bandwidth demands of different applications, which is guaranteed by our prediction engine. We evaluate the system performance gain obtained by our method with a set of multithreaded and multiprogrammed applications. Compared to traditional SRAM-based cache designs, our proposed design improves the system throughput by 58% and 14% for multithreaded and multiprogrammed applications, respectively.1
---
paper_title: Relaxing non-volatility for fast and energy-efficient STT-RAM caches

paper_content:

Spin-Transfer Torque RAM (STT-RAM) is an emerging non-volatile memory technology that is a potential universal memory that could replace SRAM in processor caches. This paper presents a novel approach for redesigning STT-RAM memory cells to reduce the high dynamic energy and slow write latencies. We lower the retention time by reducing the planar area of the cell, thereby reducing the write current, which we then use with CACTI to design caches and memories. We simulate quad-core processor designs using a combination of SRAM- and STT-RAM-based caches. Since ultra-low retention STT-RAM may lose data, we also provide a preliminary evaluation for a simple, DRAMstyle refresh policy. We found that a pure STT-RAM cache hierarchy provides the best energy efficiency, though a hybrid design of SRAM-based L1 caches with reduced-retention STT-RAM L2 and L3 caches eliminates performance loss while still reducing the energy-delay product by more than 70%.
---
paper_title: TapeCache: a high density, energy efficient cache based on domain wall memory

paper_content:

Domain Wall Memory (DWM) is a recently developed spin-based memory technology in which several bits of data are densely packed into the domains of a ferromagnetic wire. DWM has shown great promise in enabling non-volatile memory with unprecedented density and high energy efficiency. In this work, we propose TapeCache, a first attempt to employ DWMs as last-level caches in general purpose computing platforms. DWMs enable much higher density compared to SRAM, DRAM, and other spin-based memory technologies such as STT-MRAM. However, they also pose unique challenges such as serial access to the bits stored in a DWM cell, leading to variable access latencies. We propose a novel circuit-architecture co-design for TapeCache, consisting of (i) a multi-port DWM macro-cell optimized for read operations considering the asymmetry in applications' read/write characteristics, and (ii) a new cache organization and suitable management policies that mitigate the performance penalty arising from serial access to bits in a macro-cell. Over a wide range of SPEC 2006 benchmarks, TapeCache achieves 7.8X improvement in area, an average energy improvement of 7.3X, and an average performance improvement of 1.2% compared to an iso-capacity SRAM cache. Compared to an iso-capacity STT-MRAM cache, TapeCache obtains 2.3X improvement in area and 1.4X average energy savings with virtually identical performance.
---
paper_title: Circuit and microarchitecture evaluation of 3D stacking magnetic RAM (MRAM) as a universal memory replacement

paper_content:

Magnetic random access memory (MRAM) has been considered as a promising memory technology due to many attractive properties. Integrating MRAM with CMOS logic may incur extra manufacture cost, due to its hybrid magnetic-CMOS fabrication process. Stacking MRAM on top of CMOS logics using 3D integration is a way to minimize this cost overhead. In this paper, we discuss the circuit design issues for MRAM, and present the MRAM cache model. Based on the model, we compare MRAM against SRAM and DRAM in terms of area, performance, and energy. Finally we conduct architectural evaluation for 3D microprocessor stacking with MRAM. The experimental results show that MRAM stacking offers competitive IPC performance with a large reduction in power consumption compared to SRAM and DRAM counterparts.
---
paper_title: Hybrid cache architecture with disparate memory technologies

paper_content:

Caching techniques have been an efficient mechanism for mitigating the effects of the processor-memory speed gap. Traditional multi-level SRAM-based cache hierarchies, especially in the context of chip multiprocessors (CMPs), present many challenges in area requirements, core-to-cache balance, power consumption, and design complexity. New advancements in technology enable caches to be built from other technologies, such as Embedded DRAM (EDRAM), Magnetic RAM (MRAM), and Phase-change RAM (PRAM), in both 2D chips or 3D stacked chips. Caches fabricated in these technologies offer dramatically different power and performance characteristics when compared with SRAM-based caches, particularly in the areas of access latency, cell density, and overall power consumption. In this paper, we propose to take advantage of the best characteristics that each technology offers, through the use of Hybrid Cache Architecture (HCA) designs. We discuss and evaluate two types of hybrid cache architectures: inter cache Level HCA (LHCA), in which the levels in a cache hierarchy can be made of disparate memory technologies; and intra cache level or cache Region based HCA (RHCA), where a single level of cache can be partitioned into multiple regions, each of a different memory technology. We have studied a number of different HCA architectures and explored the potential of hardware support for intra-cache data movement and power consumption management within HCA caches. Utilizing a full-system simulator that has been validated against real hardware, we demonstrate that an LHCA design can provide a geometric mean 7% IPC improvement over a baseline 3-level SRAM cache design under the same area constraint across a collection of 25 workloads. A more aggressive RHCA-based design provides 12% IPC improvement over the baseline. Finally, a 2-layer 3D cache stack (3DHCA) of high density memory technology within the same chip footprint gives 18% IPC improvement over the baseline. Furthermore, up to 70% reduction in power consumption over a baseline SRAM-only design is achieved.
---
paper_title: Energy Saving Techniques for Phase Change Memory (PCM)

paper_content:

In recent years, the energy consumption of computing systems has increased and a large fraction of this energy is consumed in main memory. Towards this, researchers have proposed use of non-volatile memory, such as phase change memory (PCM), which has low read latency and power; and nearly zero leakage power. However, the write latency and power of PCM are very high and this, along with limited write endurance of PCM present significant challenges in enabling wide-spread adoption of PCM. To address this, several architecture-level techniques have been proposed. In this report, we review several techniques to manage power consumption of PCM. We also classify these techniques based on their characteristics to provide insights into them. The aim of this work is encourage researchers to propose even better techniques for improving energy efficiency of PCM based main memory.
---
paper_title: Technology comparison for large last-level caches (L3Cs): Low-leakage SRAM, low write-energy STT-RAM, and refresh-optimized eDRAM

paper_content:

Large last-level caches (L3Cs) are frequently used to bridge the performance and power gap between processor and memory. Although traditional processors implement caches as SRAMs, technologies such as STT-RAM (MRAM), and eDRAM have been used and/or considered for the implementation of L3Cs. Each of these technologies has inherent weaknesses: SRAM is relatively low density and has high leakage current; STT-RAM has high write latency and write energy consumption; and eDRAM requires refresh operations. As future processors are expected to have larger last-level caches, the goal of this paper is to study the trade-offs associated with using each of these technologies to implement L3Cs. In order to make useful comparisons between SRAM, STTRAM, and eDRAM L3Cs, we model them in detail and apply low power techniques to each of these technologies to address their respective weaknesses. We optimize SRAM for low leakage and optimize STT-RAM for low write energy. Moreover, we classify eDRAM refresh-reduction schemes into two categories and demonstrate the effectiveness of using dead-line prediction to eliminate unnecessary refreshes. A comparison of these technologies through full-system simulation shows that the proposed refresh-reduction method makes eDRAM a viable, energy-efficient technology for implementing L3Cs.
---
paper_title: Cache revive: architecting volatile STT-RAM caches for enhanced performance in CMPs

paper_content:

High density, low leakage and non-volatility are the attractive features of Spin-Transfer-Torque-RAM (STT-RAM), which has made it a strong competitor against SRAM as a universal memory replacement in multi-core systems. However, STT-RAM suffers from high write latency and energy which has impeded its widespread adoption. To this end, we look at trading-off STT-RAM's non-volatility property (data-retention-time) to overcome these problems. We formulate the relationship between retention-time and write-latency, and find optimal retention-time for architecting an efficient cache hierarchy using STT-RAM. Our results show that, compared to SRAM-based design, our proposal can improve performance and energy consumption by 18% and 60%, respectively.
---
paper_title: Phase Change Memory: From Devices to Systems

paper_content:

As conventional memory technologies such as DRAM and Flash run into scaling challenges, architects and system designers are forced to look at alternative technologies for building future computer systems. This synthesis lecture begins by listing the requirements for a next generation memory technology and briefly surveying the landscape of novel non-volatile memories. Among these, Phase Change Memory (PCM) is emerging as a leading contender, and the authors discuss the material, device, and circuit advances underlying this exciting technology. The lecture then describes architectural solutions to enable PCM for main memories. Finally, the authors explore the impact of such byte-addressable non-volatile memories on future storage and system designs. Table of Contents: Next Generation Memory Technologies / Architecting PCM for Main Memories / Tolerating Slow Writes in PCM / Wear Leveling for Durability / Wear Leveling Under Adversarial Settings / Error Resilience in Phase Change Memories / Storage and System Design With Emerging Non-Volatile Memories
---
paper_title: Dynamically reconfigurable hybrid cache: An energy-efficient last-level cache design

paper_content:

The recent development of non-volatile memory (NVM), such as spin-torque transfer magnetoresistive RAM (STT-RAM) and phase-change RAM (PRAM), with the advantage of low leakage and high density, provides an energy-efficient alternative to traditional SRAM in cache systems. We propose a novel reconfigurable hybrid cache architecture (RHC), in which NVM is incorporated in the last-level cache together with SRAM. RHC can be reconfigured by powering on/off SRAM/NVM arrays in a way-based manner. In this work, we discuss both the architecture and circuit design issues for RHC. Furthermore, we provide hardware-based mechanisms to dynamically reconfigure RHC on-the-fly based on the cache demand. Experimental results on a wide range of benchmarks show that the proposed RHC achieves an average 63%, 48% and 25% energy saving over non-reconfigurable SRAM-based cache, non-reconfigurable hybrid cache, and reconfigurable SRAM-based cache, while maintaining the system performance (at most 4% performance overhead).
---
paper_title: A software approach for combating asymmetries of non-volatile memories

paper_content:

The recent advances in non-volatile memory technologies promise the delivery of future high performance and low power computing systems. While these technologies provide attractive features, they exhibit different degrees of asymmetric read/write behavior, resulting in under utilization of the technology benefits in many scenarios. Future systems should be aware of this asymmetry to fully utilize these memory technologies. In this paper, we propose software dispatch, a cross-layer approach to distribute data to appropriate memory resources based on an application's data access characteristics. We demonstrate the application of the proposed technique through a case study system with hybrid memory caches, on which we achieve over 5% performance improvement and 9.8% power savings compared to the state-of-the-art technique.
---
paper_title: After Hard Drives—What Comes Next?

paper_content:

There are numerous emerging nonvolatile memory technologies, which have been proposed as being capable of replacing hard disk drives (HDDs). In this paper, the prospects for these alternative technologies to displace HDDs in 2020 are analyzed. In order to compare technologies, projections were made of storage density and performance in year 2020 for both hard disks and the alternative technologies, assuming the alternative technologies could solve their remaining problems and assuming that hard drives would continue to advance areal density at a pace of about 40% per year, which would result in a two-disk 2.5-in disk drive that stores approximately 40 Terabytes and costs about $40. A major conclusion of the study is that to compete with hard drives on a cost per terabyte basis will be challenging for any solid state technology, because the ITRS lithography roadmap limits the density that most alternative technologies can achieve. Those technologies with the best opportunity have a small cell size and the capability of storing multiple bits per cell. Phase change random access memory (PCRAM) and spin transfer torque random access memory (STTRAM) appear to meet these criteria. PCRAMs are being marketed by at least one supplier and therefore appear to be closer to practical realization. On the other hand, STTRAMs would appear to have a performance edge assuming they, too, can be brought to market with multiple bits per cell. Although there are technologies that are not limited by the lithography roadmap and thus have greater areal density potential, they tend to be further from practical realization.
---
paper_title: Spin-Transfer Torque MRAM (STT-MRAM): Challenges and Prospects

paper_content:

Spin-transfer torque (STT) switching demonstrated in submicron sized magnetic tunnel junctions (MTJs) has stimulated considerable interest for developments of STT switched magnetic random access memory (STT-MRAM). Remarkable progress in STT switching with MgO MTJs and increasing interest in STTMRAM in semiconductor industry have been witnessed in recent years. This paper will present a review on the progress in the intrinsic switching current density reduction and STT-MRAM prototype chip demonstration. Challenges to overcome in order for STT-MRAM to be a mainstream memory technology in future technology nodes will be discussed. Finally, potential applications of STT-MRAM in embedded and standalone memory markets will be outlined.
---
paper_title: High-endurance hybrid cache design in CMP architecture with cache partitioning and access-aware policy

paper_content:

In recent years, NVM (non-volatile memory) technologies, such as STT-RAM (spin transfer torque RAM) and PRAM (phase change RAM), have drawn a lot of attention due to their low leakage and high density. However, both NVMs suffer from high write latency and limited endurance problems. To overcome these problems, the SRAM/NVM hybrid cache architecture has been proposed, and the write pressure on NVM can be mitigated with appropriate write management policy. Moreover, many wear leveling techniques have been proposed to extend the lifetime of NVM in the hybrid cache. In this paper, we proposed a hybrid cache design that includes SRAM cache, STT-RAM cache, and STT-RAM/SRAM hybrid cache banks for CMP (chip multi-processors) architecture. We also propose a partition-level wear leveling scheme and access-aware policies to mitigate unbalanced wear-out of STT-RAM lines within a partition and among different cache partitions. Experimental results show that, our proposed scheme and policies can achieve an average of 89 times improvement in cache lifetime and are able to save 58% power consumption compared to SRAM cache.
---
paper_title: CASHIER: A Cache Energy Saving Technique for QoS Systems

paper_content:

With each CMOS technology generation, leakage energy has been increasing at an exponential rate and hence, managing the energy consumption of large, last-level caches is becoming a critical research issue in modern chip design. Saving cache energy in QoS systems is especially challenging, since, to avoid missing deadlines, a suitable balance needs to be made between energy saving and performance loss. We present CASHIER, a Cache Energy Saving Technique for Quality of Service Systems. Cashier uses dynamic profiling to estimate the memory subsystem energy and execution time of the program under multiple last level cache (LLC) configurations. It then reconfigures LLC to an energy efficient configuration with a view to meet the deadline. In QoS systems, allowed slack may be specified either as percentage of baseline execution time or as absolute slack and Cashier can work for both these cases. The experiments show the effectiveness of Cashier in saving cache energy. For example, for an L2 cache size of 2MB and 5% allowed-slack over baseline, the average saving in memory subsystem energy by using Cashier is 23.6%.
---
paper_title: NVSim: A Circuit-Level Performance, Energy, and Area Model for Emerging Nonvolatile Memory

paper_content:

Various new nonvolatile memory (NVM) technologies have emerged recently. Among all the investigated new NVM candidate technologies, spin-torque-transfer memory (STT-RAM, or MRAM), phase-change random-access memory (PCRAM), and resistive random-access memory (ReRAM) are regarded as the most promising candidates. As the ultimate goal of this NVM research is to deploy them into multiple levels in the memory hierarchy, it is necessary to explore the wide NVM design space and find the proper implementation at different memory hierarchy levels from highly latency-optimized caches to highly density- optimized secondary storage. While abundant tools are available as SRAM/DRAM design assistants, similar tools for NVM designs are currently missing. Thus, in this paper, we develop NVSim, a circuit-level model for NVM performance, energy, and area estimation, which supports various NVM technologies, including STT-RAM, PCRAM, ReRAM, and legacy NAND Flash. NVSim is successfully validated against industrial NVM prototypes, and it is expected to help boost architecture-level NVM-related studies.
---
paper_title: Bi-layered RRAM with unlimited endurance and extremely uniform switching

paper_content:

We demonstrate resistive random access memory (RRAM) architecture with bi-layered switching element for reliable resistive switching memory. Based on the modulated Schottky barrier modeling, several key functions to achieve a realiable bipolar switching property are extracted. Our device shows an excellent memory performance such as enduracne of 1011 cycles at 30ns, data retention of >104s at 200°C, and calculated bit error rate below 10−11.
---
paper_title: NVSim: A Circuit-Level Performance, Energy, and Area Model for Emerging Nonvolatile Memory

paper_content:

Various new nonvolatile memory (NVM) technologies have emerged recently. Among all the investigated new NVM candidate technologies, spin-torque-transfer memory (STT-RAM, or MRAM), phase-change random-access memory (PCRAM), and resistive random-access memory (ReRAM) are regarded as the most promising candidates. As the ultimate goal of this NVM research is to deploy them into multiple levels in the memory hierarchy, it is necessary to explore the wide NVM design space and find the proper implementation at different memory hierarchy levels from highly latency-optimized caches to highly density- optimized secondary storage. While abundant tools are available as SRAM/DRAM design assistants, similar tools for NVM designs are currently missing. Thus, in this paper, we develop NVSim, a circuit-level model for NVM performance, energy, and area estimation, which supports various NVM technologies, including STT-RAM, PCRAM, ReRAM, and legacy NAND Flash. NVSim is successfully validated against industrial NVM prototypes, and it is expected to help boost architecture-level NVM-related studies.
---
paper_title: Hybrid cache architecture with disparate memory technologies

paper_content:

Caching techniques have been an efficient mechanism for mitigating the effects of the processor-memory speed gap. Traditional multi-level SRAM-based cache hierarchies, especially in the context of chip multiprocessors (CMPs), present many challenges in area requirements, core-to-cache balance, power consumption, and design complexity. New advancements in technology enable caches to be built from other technologies, such as Embedded DRAM (EDRAM), Magnetic RAM (MRAM), and Phase-change RAM (PRAM), in both 2D chips or 3D stacked chips. Caches fabricated in these technologies offer dramatically different power and performance characteristics when compared with SRAM-based caches, particularly in the areas of access latency, cell density, and overall power consumption. In this paper, we propose to take advantage of the best characteristics that each technology offers, through the use of Hybrid Cache Architecture (HCA) designs. We discuss and evaluate two types of hybrid cache architectures: inter cache Level HCA (LHCA), in which the levels in a cache hierarchy can be made of disparate memory technologies; and intra cache level or cache Region based HCA (RHCA), where a single level of cache can be partitioned into multiple regions, each of a different memory technology. We have studied a number of different HCA architectures and explored the potential of hardware support for intra-cache data movement and power consumption management within HCA caches. Utilizing a full-system simulator that has been validated against real hardware, we demonstrate that an LHCA design can provide a geometric mean 7% IPC improvement over a baseline 3-level SRAM cache design under the same area constraint across a collection of 25 workloads. A more aggressive RHCA-based design provides 12% IPC improvement over the baseline. Finally, a 2-layer 3D cache stack (3DHCA) of high density memory technology within the same chip footprint gives 18% IPC improvement over the baseline. Furthermore, up to 70% reduction in power consumption over a baseline SRAM-only design is achieved.
---
paper_title: Energy- and endurance-aware design of phase change memory caches

paper_content:

Phase change memory (PCM) is one of the most promising technology among emerging non-volatile random access memory technologies. Implementing a cache memory using PCM provides many benefits such as high density, non-volatility, low leakage power, and high immunity to soft error. However, its disadvantages such as high write latency, high write energy, and limited write endurance prevent it from being used as a drop-in replacement of an SRAM cache. In this paper, we study a set of techniques to design an energy- and endurance-aware PCM cache. We also modeled the timing, energy, endurance, and area of PCM caches and integrated them into a PCM cache simulator to evaluate the techniques. Experiments show that our PCM cache design can achieve 8% of energy saving and 3.8 years of lifetime compared with a baseline PCM cache having less than a hour of lifetime.
---
paper_title: i2WAP: Improving non-volatile cache lifetime by reducing inter- and intra-set write variations

paper_content:

Modern computers require large on-chip caches, but the scalability of traditional SRAM and eDRAM caches is constrained by leakage and cell density. Emerging non-volatile memory (NVM) is a promising alternative to build large on-chip caches. However, limited write endurance is a common problem for non-volatile memory technologies. In addition, today's cache management might result in unbalanced write traffic to cache blocks causing heavily-written cache blocks to fail much earlier than others. Unfortunately, existing wear-leveling techniques for NVM-based main memories cannot be simply applied to NVM-based on-chip caches because cache writes have intra-set variations as well as inter-set variations. To solve this problem, we propose i2WAP, a new cache management policy that can reduce both inter- and intra-set write variations. i2WAP has two features: (1) Swap-Shift, an enhancement based on previous main memory wear-leveling to reduce cache inter-set write variations; (2) Probabilistic Set Line Flush, a novel technique to reduce cache intra-set write variations. Implementing i2WAP only needs two global counters and two global registers. By adopting i2WAP, we can improve the lifetime of on-chip non-volatile caches by 75% on average and up to 224%.
---
paper_title: TapeCache: a high density, energy efficient cache based on domain wall memory

paper_content:

Domain Wall Memory (DWM) is a recently developed spin-based memory technology in which several bits of data are densely packed into the domains of a ferromagnetic wire. DWM has shown great promise in enabling non-volatile memory with unprecedented density and high energy efficiency. In this work, we propose TapeCache, a first attempt to employ DWMs as last-level caches in general purpose computing platforms. DWMs enable much higher density compared to SRAM, DRAM, and other spin-based memory technologies such as STT-MRAM. However, they also pose unique challenges such as serial access to the bits stored in a DWM cell, leading to variable access latencies. We propose a novel circuit-architecture co-design for TapeCache, consisting of (i) a multi-port DWM macro-cell optimized for read operations considering the asymmetry in applications' read/write characteristics, and (ii) a new cache organization and suitable management policies that mitigate the performance penalty arising from serial access to bits in a macro-cell. Over a wide range of SPEC 2006 benchmarks, TapeCache achieves 7.8X improvement in area, an average energy improvement of 7.3X, and an average performance improvement of 1.2% compared to an iso-capacity SRAM cache. Compared to an iso-capacity STT-MRAM cache, TapeCache obtains 2.3X improvement in area and 1.4X average energy savings with virtually identical performance.
---
paper_title: Asynchronous Asymmetrical Write Termination (AAWT) for a low power STT-MRAM

paper_content:

Spin Transfer Torque (STT) memory is an emerging and promising non-volatile storage technology. However, the high write current is still a major challenge which leads to a huge power consumption of the memory. Due to an inherent torque asymmetry of the Magnetic Tunnel Junction (MTJ) device employed in STT memories, the switching time between parallel to anti-parallel and anti-parallel to parallel magnetization is significantly different. Hence, the write latencies for writing `0' and `1' are also considerably different. In this paper, we propose a technique called Asynchronous Asymmetrical Write Termination (AAWT) which utilizes this asymmetrical behavior to terminate the write operations asynchronously and as a result significantly reduces the write power consumption. Furthermore, we present two different AAWT implementations to determine the actual write termination times. The first one makes use of a clock signal and the second one employs a self-timing approach based on an internal delay element. As shown by our experimental results, AAWT can reduce the total write energy by 30% in average with a negligible area overhead.
---
paper_title: Avoiding unnecessary write operations in STT-MRAM for low power implementation

paper_content:

Spin Transfer Torque (STT) is a promising emerging memory technology because of its various advantages such as non-volatility, high density, virtually infinite endurance, scalability and CMOS compatibility. Despite all these features, high write current is still a challenge for its widespread use. When writing a value that is already stored, a significant current flows through the Magnetic Tunnel Junction (MTJ) cell which is almost the same as that required to flip the stored data. This increases the total power consumption of the memory. To address this issue, we propose a technique which can avoid unnecessary write operations with bit-level granularity. Our technique can save 68.9% of the total write power consumption with a minor area overhead (0.68 %) and only a small timing penalty (1.33 %).
---
paper_title: Bandwidth-aware reconfigurable cache design with hybrid memory technologies

paper_content:

In chip-multiprocessor (CMP) designs, limited memory bandwidth is a potential bottleneck of the system performance. New memory technologies, such as spin-torque-transfer memory (STT-RAM), resistive memory (RRAM), and embedded DRAM (eDRAM), are promising on-chip memory solutions for CMPs. In this paper, we propose a bandwidth-aware re-configurable cache hierarchy (BARCH) with hybrid memory technologies. BARCH consists of a hybrid cache hierarchy, a reconfiguration mechanism, and a statistical prediction engine. Our hybrid cache hierarchy chooses different memory technologies to configure each level so that the bandwidth provided by the overall hierarchy is optimized. Furthermore, we present a reconfiguration mechanism to dynamically adapt the cache space of each level based on the predicted bandwidth demands of different applications, which is guaranteed by our prediction engine. We evaluate the system performance gain obtained by our method with a set of multithreaded and multiprogrammed applications. Compared to traditional SRAM-based cache designs, our proposed design improves the system throughput by 58% and 14% for multithreaded and multiprogrammed applications, respectively.1
---
paper_title: Prediction table based management policy for STT-RAM and SRAM hybrid cache

paper_content:

STT-RAM has emerged as one of the most promising candidates for future on-chip cache due to STT-RAM's high density and low leakage. However, the STT-RAM has many disadvantages, such as: high write energy and long writing latency. To alleviate the problems, many STT-RAM and SRAM hybrid cache architectures have been proposed. In this paper, we propose a prediction table based hybrid L2 cache management policy named PTHCM. By adding a prediction table to the hybrid cache, the access information of each cache line can be recorded and we can predict the cache lines that have become dead in SRAM, lines that will be less-written and frequently-written in advance. Based on these information provided by prediction table, we propose a new cache replacement and management policy for hybrid cache architecture which replaces the dead cache lines in SRAM to improve the utilization ratio of SRAM, places frequently-written lines into SRAM and swaps out less-written lines from SRAM as early as possible. Compared with a baseline hybrid cache design, our design reduces 23.3% of the write operations to STT-RAM, and achieves 8.2% power reduction.
---
paper_title: Technology comparison for large last-level caches (L3Cs): Low-leakage SRAM, low write-energy STT-RAM, and refresh-optimized eDRAM

paper_content:

Large last-level caches (L3Cs) are frequently used to bridge the performance and power gap between processor and memory. Although traditional processors implement caches as SRAMs, technologies such as STT-RAM (MRAM), and eDRAM have been used and/or considered for the implementation of L3Cs. Each of these technologies has inherent weaknesses: SRAM is relatively low density and has high leakage current; STT-RAM has high write latency and write energy consumption; and eDRAM requires refresh operations. As future processors are expected to have larger last-level caches, the goal of this paper is to study the trade-offs associated with using each of these technologies to implement L3Cs. In order to make useful comparisons between SRAM, STTRAM, and eDRAM L3Cs, we model them in detail and apply low power techniques to each of these technologies to address their respective weaknesses. We optimize SRAM for low leakage and optimize STT-RAM for low write energy. Moreover, we classify eDRAM refresh-reduction schemes into two categories and demonstrate the effectiveness of using dead-line prediction to eliminate unnecessary refreshes. A comparison of these technologies through full-system simulation shows that the proposed refresh-reduction method makes eDRAM a viable, energy-efficient technology for implementing L3Cs.
---
paper_title: Dynamically reconfigurable hybrid cache: An energy-efficient last-level cache design

paper_content:

The recent development of non-volatile memory (NVM), such as spin-torque transfer magnetoresistive RAM (STT-RAM) and phase-change RAM (PRAM), with the advantage of low leakage and high density, provides an energy-efficient alternative to traditional SRAM in cache systems. We propose a novel reconfigurable hybrid cache architecture (RHC), in which NVM is incorporated in the last-level cache together with SRAM. RHC can be reconfigured by powering on/off SRAM/NVM arrays in a way-based manner. In this work, we discuss both the architecture and circuit design issues for RHC. Furthermore, we provide hardware-based mechanisms to dynamically reconfigure RHC on-the-fly based on the cache demand. Experimental results on a wide range of benchmarks show that the proposed RHC achieves an average 63%, 48% and 25% energy saving over non-reconfigurable SRAM-based cache, non-reconfigurable hybrid cache, and reconfigurable SRAM-based cache, while maintaining the system performance (at most 4% performance overhead).
---
paper_title: CASHIER: A Cache Energy Saving Technique for QoS Systems

paper_content:

With each CMOS technology generation, leakage energy has been increasing at an exponential rate and hence, managing the energy consumption of large, last-level caches is becoming a critical research issue in modern chip design. Saving cache energy in QoS systems is especially challenging, since, to avoid missing deadlines, a suitable balance needs to be made between energy saving and performance loss. We present CASHIER, a Cache Energy Saving Technique for Quality of Service Systems. Cashier uses dynamic profiling to estimate the memory subsystem energy and execution time of the program under multiple last level cache (LLC) configurations. It then reconfigures LLC to an energy efficient configuration with a view to meet the deadline. In QoS systems, allowed slack may be specified either as percentage of baseline execution time or as absolute slack and Cashier can work for both these cases. The experiments show the effectiveness of Cashier in saving cache energy. For example, for an L2 cache size of 2MB and 5% allowed-slack over baseline, the average saving in memory subsystem energy by using Cashier is 23.6%.
---
paper_title: Refrint: Intelligent refresh to minimize power in on-chip multiprocessor cache hierarchies

paper_content:

As manycores use dynamic energy ever more efficiently, static power consumption becomes a major concern. In particular, in a large manycore running at a low voltage, leakage in on-chip memory modules contributes substantially to the chip's power draw. This is unfortunate, given that, intuitively, the large multi-level cache hierarchy of a manycore is likely to contain a lot of useless data. An effective way to reduce this problem is to use a low-leakage technology such as embedded DRAM (eDRAM). However, eDRAM requires refresh. In this paper, we examine the opportunity of minimizing on-chip memory power further by intelligently refreshing on-chip eDRAM. We present Refrint, a simple approach to perform fine-grained, intelligent refresh of on-chip eDRAM multiprocessor cache hierarchies. We introduce the Refrint algorithms and microarchitecture. We evaluate Refrint in a simulated manycore running 16-threaded parallel applications. We show that an eDRAM-based memory hierarchy with Refrint consumes only 30% of the energy of a conventional SRAM-based memory hierarchy, and induces a slowdown of only 6%. In contrast, an eDRAM-based memory hierarchy without Refrint consumes 56% of the energy of the conventional memory hierarchy, inducing a slowdown of 25%.
---
paper_title: Cache decay: exploiting generational behavior to reduce cache leakage power

paper_content:

Power dissipation is increasingly important in CPUs ranging from those intended for mobile use, all the way up to high performance processors for high-end servers. While the bulk of the power dissipated is dynamic switching power leakage power is also beginning to be a concern. Chipmakers expect that in future chip generations, leakage's proportion of total chip power will increase significantly. This paper examines methods for reducing leakage power within the cache memories of the CPU. Because caches comprise much of a CPU chip's area and transistor counts, they are reasonable targets for attacking leakage. We discuss policies and implementations for reducing cache leakage by invalidating and "turning off" cache lines when they hold data not likely to be reused. In particular our approach is targeted at the generational nature of cache line usage. That is, cache lines typically have a flurry of frequent use when first brought into the cache, and then have a period of "dead time" before they are evicted. By devising effective, low-power ways of deducing dead time, our results show that in many cases we can reduce Ll cache leakage energy by 4/spl times/ in SPEC2000 applications without impacting performance. Because our decay-based techniques have notions of competitive on-line algorithms at their roots, their energy usage can be theoretically bounded at within a factor of two of the optimal oracle-based policy. We also examine adaptive decay-based policies that make energy-minimizing policy choices on a per-application basis by choosing appropriate decay intervals individually, for each cache line. Our proposed adaptive policies effectively reduce Ll cache leakage energy by 5/spl times/ for the SPEC2000 with only negligible degradations in performance.
---
paper_title: Probabilistic design methodology to improve run-time stability and performance of STT-RAM caches

paper_content:

Using the spin-transfer torque random access memory (STT-RAM) technology as lower level on-chip caches has been proposed to minimize leakage power consumption and enhance cache capacity at the scaled technologies. However, programming STT-RAM is a stochastic process due to the random thermal fluctuations. Conventional worst-case (corner) design with a fixed write pulse period cannot completely eliminate the write failures but maintain it at a low level by paying high cost in hardware complexity and system performance. In this work, we systematically study the impacts of the stochastic switching of STT-RAM on circuit and cache performance. Two probabilistic design techniques, write-verify-rewrite with adaptive period (WRAP) and verify-one-while-writing (VOW), then are proposed for performance improvement and write failure reduction. Our simulation results show that compared to the result of the conventional design using Hamming Code to correct the write failures, WRAP is write error free while reducing the cache write latency and energy consumption by 40% and 26%, respectively. When an extremely low write failure rate (i.e., 10−22) is allowed, VOW can further boost the reductions on write latency and energy to 52% and 29%, respectively. Furthermore, a hybrid STT-RAM based cache hierarchy taking advantages of probabilistic design techniques is proposed. The novel hierarchy can reduce the write failure rate of STT-RAM cache to 10−30, while improving the speed by 6.8% and saving 15% of energy consumption compared to a conventional design with Hamming Code.
---
paper_title: Mosaic: Exploiting the spatial locality of process variation to reduce refresh energy in on-chip eDRAM modules

paper_content:

EDRAM cells require periodic refresh, which ends up consuming substantial energy for large last-level caches. In practice, it is well known that different eDRAM cells can exhibit very different charge-retention properties. Unfortunately, current systems pessimistically assume worst-case retention times, and end up refreshing all the cells at a conservatively-high rate. In this paper, we propose an alternative approach. We use known facts about the factors that determine the retention properties of cells to build a new model of eDRAM retention times. The model is called Mosaic. The model shows that the retention times of cells in large eDRAM modules exhibit spatial correlation. Therefore, we logically divide the eDRAM module into regions or tiles, profile the retention properties of each tile, and program their refresh requirements in small counters in the cache controller. With this architecture, also called Mosaic, we refresh each tile at a different rate. The result is a 20x reduction in the number of refreshes in large eDRAM modules — practically eliminating refresh as a source of energy consumption.
---
paper_title: Design techniques to improve the device write margin for MRAM-based cache memory

paper_content:

As one promising non-volatile memory technology, magnetoresistive RAM (MRAM) based on magnetic tunneling junctions (MTJs) has recently attracted much attention. However, latest device research has discovered that, in order to maintain sufficient MTJ write margin to prevent device breakdown, MTJs will be subject to unconventionally high random write error rates (e.g., 10-3 and above) as memory cell size is being scaled down. This new discovery seriously threatens the scalability of MRAM, and the material/device research community is actively searching for solutions to largely reduce MTJ write error rates and meanwhile maintain sufficient device write margin. In this paper, we attempt to address this challenge from the architecture level when using MRAM to implement cache memory. In particular, we show that two simple cache architecture design techniques can be used to effectively tolerate high MTJ write error rates at small performance and implementation cost, which makes it much easier to maintain sufficient MTJ write margin and hence push the MRAM scalability envelope. Using the full system simulator PTLsim and a variety of benchmarks, we show that the proposed design techniques can readily accommodate MTJ write error rate up to 0.75% at the penalty of less than 4% processor performance degradation, less than 10% silicon area overhead, and 6% energy consumption overhead.
---
paper_title: Bandwidth-aware reconfigurable cache design with hybrid memory technologies

paper_content:

In chip-multiprocessor (CMP) designs, limited memory bandwidth is a potential bottleneck of the system performance. New memory technologies, such as spin-torque-transfer memory (STT-RAM), resistive memory (RRAM), and embedded DRAM (eDRAM), are promising on-chip memory solutions for CMPs. In this paper, we propose a bandwidth-aware re-configurable cache hierarchy (BARCH) with hybrid memory technologies. BARCH consists of a hybrid cache hierarchy, a reconfiguration mechanism, and a statistical prediction engine. Our hybrid cache hierarchy chooses different memory technologies to configure each level so that the bandwidth provided by the overall hierarchy is optimized. Furthermore, we present a reconfiguration mechanism to dynamically adapt the cache space of each level based on the predicted bandwidth demands of different applications, which is guaranteed by our prediction engine. We evaluate the system performance gain obtained by our method with a set of multithreaded and multiprogrammed applications. Compared to traditional SRAM-based cache designs, our proposed design improves the system throughput by 58% and 14% for multithreaded and multiprogrammed applications, respectively.1
---
paper_title: A low-power phase change memory based hybrid cache architecture

paper_content:

Sub-threshold leakage in SRAM based cache memories is becoming a predominant source of power consumption in deep-sub micron CMOS designs. Phase Change Random Access Memory (PRAM), a high density, fast access, non-volatile memory is being considered as a candidate for future universal memory technologies. In this paper, we investigate the architectural challenges in integrating a PRAM based memory into the conventional cache hierarchy. First, we develop PRAM cache delay and energy models. We then propose a hybrid PRAM architecture for L1 instruction caches on embedded processors. We also propose a PRAM based unified cache architecture for L2 caches on high-end microprocessors. Finally, we evaluate the proposed architectures, in terms of area, performance, and energy. The experimental results show that the PRAM based cache architectures achieve close to 80% reduction in the leakage energy consumption of a L1-L2 cache hierarchy.
---
paper_title: Static and dynamic co-optimizations for blocks mapping in hybrid caches

paper_content:

In this paper, a combined static and dynamic scheme is proposed to optimize the block placement for endurance and energy-efficiency in a hybrid SRAM and STT-RAM cache. With the proposed scheme, STT-RAM endurance is maximized while performance is maintained. We use the compiler to provide static hints to guide initial data placement, and use the hardware to correct the hints based on the run-time cache behavior. Experimental results show that the combined scheme improves the endurance by 23.9x and 5.9x compared to pure static and pure dynamic optimizations respectively. Furthermore, the system energy can be reduced by 17% compared to pure dynamic optimization through minimizing STT-RAM writes.
---
paper_title: Write intensity prediction for energy-efficient non-volatile caches

paper_content:

This paper presents a novel concept called write intensity prediction for energy-efficient non-volatile caches as well as the architecture that implements the concept. The key idea is to correlate write intensity of cache blocks with addresses of memory access instructions that incur cache misses of those blocks. The predictor keeps track of instructions that tend to load write-intensive blocks and utilizes that information to predict write intensity of blocks. Based on this concept, we propose a block placement strategy driven by write intensity prediction for SRAM/STT-RAM hybrid caches. Experimental results show that the proposed approach reduces write energy consumption by 55% on average compared to the existing hybrid cache architecture.
---
paper_title: Relaxing non-volatility for fast and energy-efficient STT-RAM caches

paper_content:

Spin-Transfer Torque RAM (STT-RAM) is an emerging non-volatile memory technology that is a potential universal memory that could replace SRAM in processor caches. This paper presents a novel approach for redesigning STT-RAM memory cells to reduce the high dynamic energy and slow write latencies. We lower the retention time by reducing the planar area of the cell, thereby reducing the write current, which we then use with CACTI to design caches and memories. We simulate quad-core processor designs using a combination of SRAM- and STT-RAM-based caches. Since ultra-low retention STT-RAM may lose data, we also provide a preliminary evaluation for a simple, DRAMstyle refresh policy. We found that a pure STT-RAM cache hierarchy provides the best energy efficiency, though a hybrid design of SRAM-based L1 caches with reduced-retention STT-RAM L2 and L3 caches eliminates performance loss while still reducing the energy-delay product by more than 70%.
---
paper_title: Selectively protecting error-correcting code for area-efficient and reliable STT-RAM caches

paper_content:

Recent researches on STT-RAM revealed that device scaling makes its write operations unreliable. To mitigate the impact of this problem, this paper proposes a low-cost, ECC-based solution for STT-RAM caches. In particular, it proposes to share storage for ECC among different blocks within a set and to use them only for unsuccessful write operations. Experimental results show that our scheme reduces 74% to 98% of area overhead incurred by the conventional per-block ECC while maintaining system performance and reliability.
---
paper_title: TapeCache: a high density, energy efficient cache based on domain wall memory

paper_content:

Domain Wall Memory (DWM) is a recently developed spin-based memory technology in which several bits of data are densely packed into the domains of a ferromagnetic wire. DWM has shown great promise in enabling non-volatile memory with unprecedented density and high energy efficiency. In this work, we propose TapeCache, a first attempt to employ DWMs as last-level caches in general purpose computing platforms. DWMs enable much higher density compared to SRAM, DRAM, and other spin-based memory technologies such as STT-MRAM. However, they also pose unique challenges such as serial access to the bits stored in a DWM cell, leading to variable access latencies. We propose a novel circuit-architecture co-design for TapeCache, consisting of (i) a multi-port DWM macro-cell optimized for read operations considering the asymmetry in applications' read/write characteristics, and (ii) a new cache organization and suitable management policies that mitigate the performance penalty arising from serial access to bits in a macro-cell. Over a wide range of SPEC 2006 benchmarks, TapeCache achieves 7.8X improvement in area, an average energy improvement of 7.3X, and an average performance improvement of 1.2% compared to an iso-capacity SRAM cache. Compared to an iso-capacity STT-MRAM cache, TapeCache obtains 2.3X improvement in area and 1.4X average energy savings with virtually identical performance.
---
paper_title: Asymmetric-access aware optimization for STT-RAM caches with process variations

paper_content:

STT-RAM (Spin Transfer Torque Random Access Memory) has been extensively researched as a potential replacement of SRAM (Static RAM) as on-chip caches. Prior work has shown that STT-RAM caches can improve performance and reduce power consumption because of its advantages of high density, fast read speed, low standby power, etc. However, under the impact of process variations, using worst-case design can induce significant performance and power overhead in STT-RAM caches. In order to overcome the problem of process variations, we propose to apply the variable-latency access method to STT-RAM caches by introducing a variation-aware LRU (Least Recently Used) policy. Moreover, we show that simply applying traditional variable-latency access method is inefficient due to the read/write asymmetry. First, we demonstrate that a write-oriented data migration is preferred. Second, a block remapping is necessary to prevent some cache sets from being significantly affected by process variations. After using our techniques, the experimental results show that the performance can be improved by 13.8% and power consumption can be reduced by 14.1% compared to a prior approach [3].
---
paper_title: Energy-efficient Spin-Transfer Torque RAM cache exploiting additional all-zero-data flags

paper_content:

Large on-chip caches account for a considerable fraction of the total energy consumption in modern microprocessors. In this context, emerging Spin-Transfer Torque RAM (STT-RAM) has been regarded as a promising candidate to replace large on-chip SRAM caches in virtue of its nature of the zero leakage. However, large energy requirement of STT-RAM on write operations, resulting in a huge amount of dynamic energy consumption, precludes it from application to on-chip cache designs. In order to reduce the write energy of the STT-RAM cache thereby the total energy consumption, this paper provides an architectural technique which exploits the fact that many applications process a large number of zero data. The proposed design appends additional flags in cache tag arrays and set these additional bits if the corresponding data in the cache line is the zero-valued data in which all data bits are zero. Our experimental results show that the proposed cache design can reduce 73.78% and 69.30% of the dynamic energy on write operations at the byte and word granularities, respectively; total energy consumption reduced by 36.18% and 42.51%, respectively. In addition to the energy reduction, performance evaluation results indicate that the proposed cache improves the processor performance by 5.44% on average.
---
paper_title: Coding Last Level STT-RAM Cache for High Endurance and Low Power

paper_content:

STT-RAM technology has recently emerged as one of the most promising memory technologies. However, its major problems, limited write endurance and high write energy, are still preventing it from being used as a drop-in replacement of SRAM cache. In this paper, we propose a novel coding scheme for STT-RAM last level cache based on the concept of value locality. We reduce switching probability in cache by swapping common patterns with limited weight codes (LWC) to make writes less often as well as more uniform. We also define some policies for swapping these patterns. Our evaluation shows that bit write variance in memory cells can be reduced by about 20% on average resulting in a more uniform wear-out directly enhancing lifetime and improving cell reliability. In addition, writes in cache lines can be reduced by about 12% compared to one of the most effective circuit level techniques known as early write termination (EWT) [12]. Our method increases memory hierarchy access time by about 0.08% on average, which is negligible. We have shown that our method doesn't adversely affect last level cache energy-delay2. The non-uniformity caused by the coding scheme can be used for another coding scheme at main memory or L1 cache depending on their technologies.
---
paper_title: Circuit and microarchitecture evaluation of 3D stacking magnetic RAM (MRAM) as a universal memory replacement

paper_content:

Magnetic random access memory (MRAM) has been considered as a promising memory technology due to many attractive properties. Integrating MRAM with CMOS logic may incur extra manufacture cost, due to its hybrid magnetic-CMOS fabrication process. Stacking MRAM on top of CMOS logics using 3D integration is a way to minimize this cost overhead. In this paper, we discuss the circuit design issues for MRAM, and present the MRAM cache model. Based on the model, we compare MRAM against SRAM and DRAM in terms of area, performance, and energy. Finally we conduct architectural evaluation for 3D microprocessor stacking with MRAM. The experimental results show that MRAM stacking offers competitive IPC performance with a large reduction in power consumption compared to SRAM and DRAM counterparts.
---
paper_title: Prediction table based management policy for STT-RAM and SRAM hybrid cache

paper_content:

STT-RAM has emerged as one of the most promising candidates for future on-chip cache due to STT-RAM's high density and low leakage. However, the STT-RAM has many disadvantages, such as: high write energy and long writing latency. To alleviate the problems, many STT-RAM and SRAM hybrid cache architectures have been proposed. In this paper, we propose a prediction table based hybrid L2 cache management policy named PTHCM. By adding a prediction table to the hybrid cache, the access information of each cache line can be recorded and we can predict the cache lines that have become dead in SRAM, lines that will be less-written and frequently-written in advance. Based on these information provided by prediction table, we propose a new cache replacement and management policy for hybrid cache architecture which replaces the dead cache lines in SRAM to improve the utilization ratio of SRAM, places frequently-written lines into SRAM and swaps out less-written lines from SRAM as early as possible. Compared with a baseline hybrid cache design, our design reduces 23.3% of the write operations to STT-RAM, and achieves 8.2% power reduction.
---
paper_title: Hybrid cache architecture with disparate memory technologies

paper_content:

Caching techniques have been an efficient mechanism for mitigating the effects of the processor-memory speed gap. Traditional multi-level SRAM-based cache hierarchies, especially in the context of chip multiprocessors (CMPs), present many challenges in area requirements, core-to-cache balance, power consumption, and design complexity. New advancements in technology enable caches to be built from other technologies, such as Embedded DRAM (EDRAM), Magnetic RAM (MRAM), and Phase-change RAM (PRAM), in both 2D chips or 3D stacked chips. Caches fabricated in these technologies offer dramatically different power and performance characteristics when compared with SRAM-based caches, particularly in the areas of access latency, cell density, and overall power consumption. In this paper, we propose to take advantage of the best characteristics that each technology offers, through the use of Hybrid Cache Architecture (HCA) designs. We discuss and evaluate two types of hybrid cache architectures: inter cache Level HCA (LHCA), in which the levels in a cache hierarchy can be made of disparate memory technologies; and intra cache level or cache Region based HCA (RHCA), where a single level of cache can be partitioned into multiple regions, each of a different memory technology. We have studied a number of different HCA architectures and explored the potential of hardware support for intra-cache data movement and power consumption management within HCA caches. Utilizing a full-system simulator that has been validated against real hardware, we demonstrate that an LHCA design can provide a geometric mean 7% IPC improvement over a baseline 3-level SRAM cache design under the same area constraint across a collection of 25 workloads. A more aggressive RHCA-based design provides 12% IPC improvement over the baseline. Finally, a 2-layer 3D cache stack (3DHCA) of high density memory technology within the same chip footprint gives 18% IPC improvement over the baseline. Furthermore, up to 70% reduction in power consumption over a baseline SRAM-only design is achieved.
---
paper_title: On-chip caches built on multilevel spin-transfer torque RAM cells and its optimizations

paper_content:

It has been predicted that a processor's caches could occupy as much as 90p of chip area a few technology nodes from the current ones. In this article, we investigate the use of multilevel spin-transfer torque RAM (STT-RAM) cells in the design of processor caches. We start with examining the access (read and write) scheme for multilevel cell (MLC) STT-RAM from a circuit design perspective, detailing the read and write circuits. Compared to traditional SRAM caches, a multilevel cell (MLC) STT-RAM cache design is denser, fast, and requires less energy. However, a number of critical architecture-level issues remain to be solved before MLC STT-RAM technology can be deployed in processor caches. We shall offer solutions to the issue of bit encoding as well as tackle the write endurance problem. In particular, the latter has been neglected in previous works on STT-RAM caches. We propose a set remapping scheme that can potentially prolong the lifetime of a MLC STT-RAM cache by 80× on average. Furthermore, a method for recovering the performance that may be lost in some applications due to set remapping is proposed. The impacts of process variations of the MLC STT-RAM cell on the robustness of the memory hierarchy is also discussed, together with various enhancement techniques, namely, ECC and design redundancy.
---
paper_title: Technology comparison for large last-level caches (L3Cs): Low-leakage SRAM, low write-energy STT-RAM, and refresh-optimized eDRAM

paper_content:

Large last-level caches (L3Cs) are frequently used to bridge the performance and power gap between processor and memory. Although traditional processors implement caches as SRAMs, technologies such as STT-RAM (MRAM), and eDRAM have been used and/or considered for the implementation of L3Cs. Each of these technologies has inherent weaknesses: SRAM is relatively low density and has high leakage current; STT-RAM has high write latency and write energy consumption; and eDRAM requires refresh operations. As future processors are expected to have larger last-level caches, the goal of this paper is to study the trade-offs associated with using each of these technologies to implement L3Cs. In order to make useful comparisons between SRAM, STTRAM, and eDRAM L3Cs, we model them in detail and apply low power techniques to each of these technologies to address their respective weaknesses. We optimize SRAM for low leakage and optimize STT-RAM for low write energy. Moreover, we classify eDRAM refresh-reduction schemes into two categories and demonstrate the effectiveness of using dead-line prediction to eliminate unnecessary refreshes. A comparison of these technologies through full-system simulation shows that the proposed refresh-reduction method makes eDRAM a viable, energy-efficient technology for implementing L3Cs.
---
paper_title: Hybrid cache architecture replacing SRAM cache with future memory technology

paper_content:

Recently, hybrid cache architecture has become illuminated. As heterogeneous memory dies are stacked, it improves the performance of microprocessor enhanced in terms of power consumption and processing speed. This paper analyzed the hybrid cache architecture using different programs and memory types. SRAM is fixed for L1 cache memory, whereas DRAM, MRAM, and PRAM are the candidates for L2 cache memory. Each memory structure has the area satisfying the least Average Memory Access Time (AMAT) under a given area condition. Architecture composed of SRAM and MRAM shows 16.9% reduction in average memory access time and 15.2% of power reduction compared with that composed of homogeneous SRAM. Structure of SRAM and DRAM represents 33.0% reduction in power consumption, and that of SRAM and PRAM shows a potential to reduce area and power consumption due to their high density.
---
paper_title: Energy- and endurance-aware design of phase change memory caches

paper_content:

Phase change memory (PCM) is one of the most promising technology among emerging non-volatile random access memory technologies. Implementing a cache memory using PCM provides many benefits such as high density, non-volatility, low leakage power, and high immunity to soft error. However, its disadvantages such as high write latency, high write energy, and limited write endurance prevent it from being used as a drop-in replacement of an SRAM cache. In this paper, we study a set of techniques to design an energy- and endurance-aware PCM cache. We also modeled the timing, energy, endurance, and area of PCM caches and integrated them into a PCM cache simulator to evaluate the techniques. Experiments show that our PCM cache design can achieve 8% of energy saving and 3.8 years of lifetime compared with a baseline PCM cache having less than a hour of lifetime.
---
paper_title: High-endurance and performance-efficient design of hybrid cache architectures through adaptive line replacement

paper_content:

In this paper, we propose a run-time strategy for managing writes onto last level cache in chip multiprocessors where STT-RAM memory is used as baseline technology. To this end, we assume that each cache set is decomposed into limited SRAM lines and large number of STT-RAM lines. SRAM lines are target of frequently-written data and rarely-written or read-only ones are pushed into STT-RAM. As a novel contribution, a low-overhead, fully-hardware technique is utilized to detect write-intensive data blocks of working set and place them into SRAM lines while the remaining data blocks are candidates to be remapped onto STT-RAM blocks during system operation. Therefore, the achieved cache architecture has large capacity and consumes near zero leakage energy using STT-RAM array; while dynamic write energy, acceptable write latency, and long lifetime is guaranteed via SRAM array. Results of full-system simulation for a quad-core CMP running PARSEC-2 benchmark suit confirm an average of 49 times improvement in cache lifetime and more than 50% reduction in cache power consumption when compared to baseline configurations.
---
paper_title: Future cache design using STT MRAMs for improved energy efficiency: devices, circuits and architecture

paper_content:

Spin-transfer torque magnetic RAM (STT MRAM) has emerged as a promising candidate for on-chip memory in future computing platforms. We present a cross-layer (device-circuit-architecture) approach to energy-efficient cache design using STT MRAM. At the device and circuit levels, we consider different genres of MTJs and bitcells, and evaluate their impact on the area, energy and performance of caches. In addition, we propose micro-architectural techniques viz. sequential cache read and partial cache line update, which exploit the non-volatility of STT MRAM to further improve energy efficiency of STT MRAM caches. A detailed comparison of STT MRAM caches with SRAM-based caches is also presented. Our results indicate that the proposed optimizations significantly enhance the efficiency of STT MRAM for designing lower level caches.
---
paper_title: STT-RAM based energy-efficiency hybrid cache for CMPs

paper_content:

Modern high performance Chip Multiprocessor (CMP) systems rely on large on-chip cache hierarchy. As technology scales down, the leakage power of present SRAM based cache gradually dominates the on-chip power consumption, which can severely jeopardize system performance. The emerging nonvolatile Spin Transfer Torque RAM (STT-RAM) is a promising candidate for large on-chip cache because of the ultra low leakage power. However, the write operations on STT-RAM suffer from considerably higher energy as well as longer latency compared with SRAM which will make STT-RAM in trouble for write-intensive workloads. In this paper, we propose to integrate SRAM with STT-RAM to construct a novel hybrid cache architecture for CMPs. We also propose dedicated microarchitectural mechanisms to make the hybrid cache robust to workloads with different write patterns. Extensive simulation results demonstrate that the proposed hybrid scheme is adaptive to variations of workloads. Overall power consumption is reduced by 37.1% and performance is improved by 23.6% on average compared with SRAM based static NUCA under the same area configuration.
---
paper_title: STT-RAM for Shared Memory in GPUs

paper_content:

In the context of massively parallel processors such as Graphics Processing Units (GPUs), an emerging non-volatile memory – STT-RAM – provides substantial power, area savings, and increased capacity compared to the conventionally used SRAM. The use of highly dense, low static power STT-RAM in processors that run just few threads of execution does not seem attractive because of several times slower write latency, which can relatively impair the performance of the system. However, hundreds to thousands of threads executing in parallel in high-throughput GPUs hide the long write latency of STT-RAM through fine-grained multithreading. In this thesis, evaluation of possibility of STT-RAM for the shared memory in GPUs was done. Performance, energy and area were evaluated across various configurations of shared memory capacity, banks and ports across a set of benchmarks displaying different characteristics. Results show performance degrades only up to 2% on average for an STT-RAM write latency, which is four times that of SRAM write latency. Performance is even increased by 20% on average when the denser STT-RAM is used to increase shared memory capacity, banks and ports. The energy savings are up to 17% and area savings up to 50%. The evaluation helps understand the trade-offs involved in the use of STT-RAM in GPUs. In the process, a few configurations were identified which encourage their use. To better understand the low impact of high shared memory latency on GPU’s performance, a theoretical analysis was done.
---
paper_title: Cross-Layer Techniques for Optimizing Systems Utilizing Memories with Asymmetric Access Characteristics

paper_content:

Non-volatile memory technologies promise a variety of advantages for memory architectures of next generation computing systems. However, these capabilities come at the cost of some inefficiencies governing the operation of these memories. The most well understood is the asymmetry of access. In order to most effectively take advantage of the benefits of these memory technologies in terms of density and reduced static power in systems while mitigating access complexity an one-size fits all method is not sufficient for all types of applications. Instead, cross-layer techniques that include the compiler, operating system, and hardware layer can extract characteristics from the application that can be used to deliver the highest possible performance while minimizing power consumption for systems using these memories.
---
paper_title: Exploiting reuse information to reduce refresh energy in on-chip eDRAM caches

paper_content:

This work introduces a novel refresh mechanism that leverages reuse information to decide which blocks should be refreshed in an energy-aware eDRAM last-level cache. Experimental results show that, compared to a conventional eDRAM cache, the energy-aware approach achieves refresh energy savings up to 71%, while the reduction on the overall dynamic energy is by 65% with negligible performance losses.
---
paper_title: Cache revive: architecting volatile STT-RAM caches for enhanced performance in CMPs

paper_content:

High density, low leakage and non-volatility are the attractive features of Spin-Transfer-Torque-RAM (STT-RAM), which has made it a strong competitor against SRAM as a universal memory replacement in multi-core systems. However, STT-RAM suffers from high write latency and energy which has impeded its widespread adoption. To this end, we look at trading-off STT-RAM's non-volatility property (data-retention-time) to overcome these problems. We formulate the relationship between retention-time and write-latency, and find optimal retention-time for architecting an efficient cache hierarchy using STT-RAM. Our results show that, compared to SRAM-based design, our proposal can improve performance and energy consumption by 18% and 60%, respectively.
---
paper_title: A novel architecture of the 3D stacked MRAM L2 cache for CMPs

paper_content:

Magnetic random access memory (MRAM) is a promising memory technology, which has fast read access, high density, and non-volatility. Using 3D heterogeneous integrations, it becomes feasible and cost-efficient to stack MRAM atop conventional chip multiprocessors (CMPs). However, one disadvantage of MRAM is its long write latency and its high write energy. In this paper, we first stackMRAM-based L2 caches directly atop CMPs and compare it against SRAM counterparts in terms of performance and energy. We observe that the direct MRAM stacking might harm the chip performance due to the aforementioned long write latency and high write energy. To solve this problem, we then propose two architectural techniques: read-preemptive write buffer and SRAM-MRAM hybrid L2 cache. The simulation result shows that our optimized MRAM L2 cache improves performance by 4.91% and reduces power by 73.5%compared to the conventional SRAM L2 cache with the similar area.
---
paper_title: A circuit-architecture co-optimization framework for evaluating emerging memory hierarchies

paper_content:

Energy-efficient and cost-effective memory hierarchies are needed in the next era of computing. Currently, many emerging non-volatile memory technologies such as PCRAM, STTRAM, and ReRAM, can potentially meet this requirement. It is necessary to have a framework that can quickly find the optimal memory technology choice and the corresponding circuit design style in terms of performance, energy, or cost. In this work, we first build a circuit-architecture joint design space exploration framework by combining RC circuit analysis and ANN-based performance modeling. Then, we use this framework to design and optimize different memory hierarchy levels by adopting new memory technologies such as ReRAM.
---
paper_title: WriteSmoothing: improving lifetime of non-volatile caches using intra-set wear-leveling

paper_content:

Driven by the trends of increasing core-count and bandwidth-wall problem, the size of last level caches (LLCs) has greatly increased. Since SRAM consumes high leakage power, researchers have explored use of non-volatile memories (NVMs) for designing caches as they provide high density and consume low leakage power. However, since NVMs have low write-endurance and the existing cache management policies are write variation-unaware, effective wear-leveling techniques are required for achieving reasonable cache lifetimes using NVMs. We present WriteSmoothing, a technique for mitigating intra-set write variation in NVM caches. WriteSmoothing logically divides the cache-sets into multiple modules. For each module, WriteSmoothing collectively records number of writes in each way for any of the sets. It then periodically makes most frequently written ways in a module unavailable to shift the write-pressure to other ways in the sets of the module. Extensive simulation results have shown that on average, for single and dual-core system configurations, WriteSmoothing improves cache lifetime by 2.17X and 2.75X, respectively. Also, its implementation overhead is small and it works well for a wide range of algorithm and system parameters.
---
paper_title: Embedded memory hierarchy exploration based on magnetic RAM

paper_content:

SRAM, DRAM and FLASH are the three main employed technologies in design of on-chip processor memories. However, manufacturing constraints for this technologies in the most advanced nodes compromises further evolution. MRAM (Magnetic memory) presents itself as an attractive alternative for these technologies, as it has reasonable timing and power characteristics. Last results in the state of the art demonstrate that MRAM access time is can be less than 5ns and read/write energy per bit in same order of magnitude as SRAM, also it can evolve with the manufacturing process. One important feature of MRAM is the non-volatility, allowing to define new instant on/off policies and mainly optimizing leakage current. In this paper we demonstrate how MRAM can be used into memory hierarchy of embedded systems. The main objective is to demonstrate the interest to use MRAM for Level-1 & 2 cache and to better understand the architectural choice in order to minimize the impact of the higher write latency of MRAMs.
---
paper_title: Combining RAM technologies for hard-error recovery in L1 data caches working at very-low power modes

paper_content:

Low-power modes in modern microprocessors rely on low frequencies and low voltages to reduce the energy budget. Nevertheless, manufacturing induced parameter variations can make SRAM cells unreliable producing hard errors at supply voltages below Vccmin. Recent proposals provide a rather low fault-coverage due to the fault coverage/overhead trade-off. We propose a new fault-tolerant L1 cache, which combines SRAM and eDRAM cells in L1 data caches to provide 100% SRAM hard-error fault coverage. Results show that, compared to a conventional cache and assuming 50% failure probability at low-power mode, leakage and dynamic energy savings are by 85% and 62%, respectively, with a minimal impact on performance.
---
paper_title: i2WAP: Improving non-volatile cache lifetime by reducing inter- and intra-set write variations

paper_content:

Modern computers require large on-chip caches, but the scalability of traditional SRAM and eDRAM caches is constrained by leakage and cell density. Emerging non-volatile memory (NVM) is a promising alternative to build large on-chip caches. However, limited write endurance is a common problem for non-volatile memory technologies. In addition, today's cache management might result in unbalanced write traffic to cache blocks causing heavily-written cache blocks to fail much earlier than others. Unfortunately, existing wear-leveling techniques for NVM-based main memories cannot be simply applied to NVM-based on-chip caches because cache writes have intra-set variations as well as inter-set variations. To solve this problem, we propose i2WAP, a new cache management policy that can reduce both inter- and intra-set write variations. i2WAP has two features: (1) Swap-Shift, an enhancement based on previous main memory wear-leveling to reduce cache inter-set write variations; (2) Probabilistic Set Line Flush, a novel technique to reduce cache intra-set write variations. Implementing i2WAP only needs two global counters and two global registers. By adopting i2WAP, we can improve the lifetime of on-chip non-volatile caches by 75% on average and up to 224%.
---
paper_title: Constructing large and fast multi-level cell STT-MRAM based cache for embedded processors

paper_content:

MLC STT-MRAM (Multi-level Cell Spin-Transfer Torque Magnetic RAM), an emerging non-volatile memory technology, has become a promising candidate to construct L2 caches for high-end embedded processors. However, the long write latency limits the effectiveness of MLC STT-MRAM based L2 caches. In this paper, we address this limitation with two novel designs: Line Pairing (LP) and Line Swapping (LS). LP forms fast cachelines by re-organizing MLC soft bits which are faster to write. LS dynamically stores frequently written data into these fast cachelines. Our experimental results show that LP and LS improve system performance by 15% and reduce energy consumption by 21%.
---
paper_title: Reducing cache power with low-cost, multi-bit error-correcting codes

paper_content:

Technology advancements have enabled the integration of large on-die embedded DRAM (eDRAM) caches. eDRAM is significantly denser than traditional SRAMs, but must be periodically refreshed to retain data. Like SRAM, eDRAM is susceptible to device variations, which play a role in determining refresh time for eDRAM cells. Refresh power potentially represents a large fraction of overall system power, particularly during low-power states when the CPU is idle. Future designs need to reduce cache power without incurring the high cost of flushing cache data when entering low-power states. In this paper, we show the significant impact of variations on refresh time and cache power consumption for large eDRAM caches. We propose Hi-ECC, a technique that incorporates multi-bit error-correcting codes to significantly reduce refresh rate. Multi-bit error-correcting codes usually have a complex decoder design and high storage cost. Hi-ECC avoids the decoder complexity by using strong ECC codes to identify and disable sections of the cache with multi-bit failures, while providing efficient single-bit error correction for the common case. Hi-ECC includes additional optimizations that allow us to amortize the storage cost of the code over large data words, providing the benefit of multi-bit correction at same storage cost as a single-bit error-correcting (SECDED) code (2% overhead). Our proposal achieves a 93% reduction in refresh power vs. a baseline eDRAM cache without error correcting capability, and a 66% reduction in refresh power vs. a system using SECDED codes.
---
paper_title: Performance, Power, and Reliability Tradeoffs of STT-RAM Cell Subject to Architecture-Level Requirement

paper_content:

Large switching current and long switching time have significantly limited the adoption of spin-transfer torque random access memory (STT-RAM). Technology scaling, moreover, makes it very challenging to reduce the switching current while maintaining the reliability of magnetic tunneling junction (MTJ) to be similar to that of the earlier generations. In this work, we shall exploit a key insight that in the most on-chip caches where STT-RAM is most likely to be deployed, the lifespan of the data stored in the memory cells is much shorter than the data retention time requirement assumed in STT-RAM development, namely, 4~10 years. We also quantitatively investigated the possibility of trading off MTJ nonvolatility for improved switching performance, e.g., the switching time and/or current, under architectural level guidance. We further proposed and evaluated a hybrid memory design technique that partitions the on-chip STT-RAM cache into two parts with different nonvolatility performances so as to better fit the diverse retention time requirements of different data sets.
---
paper_title: Cache coherence enabled adaptive refresh for volatile STT-RAM

paper_content:

Spin-Transfer Torque RAM (STT-RAM) is extensively studied in recent years. Recent work proposed to improve the write performance of STT-RAM through relaxing the retention time of STT-RAM cell, magnetic tunnel junction (MTJ). Unfortunately, frequent refresh operations of volatile STT-RAM could dissipate significantly extra energy. In addition, refresh operations can severely conflict with normal read/write operations and results in degraded cache performance. This paper proposes Cache Coherence Enabled Adaptive Refresh (CCear) to minimize refresh operations for volatile STT-RAM. Through novel modifications to cache coherence protocol, CCear can effectively minimize the number of refresh operations on volatile STT-RAM. Full-system simulation results show that CCear approaches the performance of the ideal refresh policy with negligible overhead.
---
paper_title: Avoiding unnecessary write operations in STT-MRAM for low power implementation

paper_content:

Spin Transfer Torque (STT) is a promising emerging memory technology because of its various advantages such as non-volatility, high density, virtually infinite endurance, scalability and CMOS compatibility. Despite all these features, high write current is still a challenge for its widespread use. When writing a value that is already stored, a significant current flows through the Magnetic Tunnel Junction (MTJ) cell which is almost the same as that required to flip the stored data. This increases the total power consumption of the memory. To address this issue, we propose a technique which can avoid unnecessary write operations with bit-level granularity. Our technique can save 68.9% of the total write power consumption with a minor area overhead (0.68 %) and only a small timing penalty (1.33 %).
---
paper_title: Dynamically reconfigurable hybrid cache: An energy-efficient last-level cache design

paper_content:

The recent development of non-volatile memory (NVM), such as spin-torque transfer magnetoresistive RAM (STT-RAM) and phase-change RAM (PRAM), with the advantage of low leakage and high density, provides an energy-efficient alternative to traditional SRAM in cache systems. We propose a novel reconfigurable hybrid cache architecture (RHC), in which NVM is incorporated in the last-level cache together with SRAM. RHC can be reconfigured by powering on/off SRAM/NVM arrays in a way-based manner. In this work, we discuss both the architecture and circuit design issues for RHC. Furthermore, we provide hardware-based mechanisms to dynamically reconfigure RHC on-the-fly based on the cache demand. Experimental results on a wide range of benchmarks show that the proposed RHC achieves an average 63%, 48% and 25% energy saving over non-reconfigurable SRAM-based cache, non-reconfigurable hybrid cache, and reconfigurable SRAM-based cache, while maintaining the system performance (at most 4% performance overhead).
---
paper_title: C1C: A configurable, compiler-guided STT-RAM L1 cache

paper_content:

Spin-Transfer Torque RAM (STT-RAM), a promising alternative to SRAM for reducing leakage power consumption, has been widely studied to mitigate the impact of its asymmetrically long write latency. Recently, STT-RAM has been proposed for L1 caches by relaxing the data retention time to improve write performance and dynamic energy. However, as the technology scales down from 65nm to 22nm, the performance of the read operation scales poorly due to reduced sense margins and sense amplifier delays. In this article, we leverage a dual-mode STT memory cell to design a configurable L1 cache architecture termed C1C to mitigate read performance barriers with technology scaling. Guided by application access characteristics discovered through novel compiler analyses, the proposed cache adaptively switches between a high performance and a low-power access mode. Our evaluation demonstrates that the proposed cache with compiler guidance outperforms a state-of-the-art STT-RAM cache design by 9p with high dynamic energy efficiency, leading to significant performance/watt improvements over several competing approaches.
---
paper_title: A software approach for combating asymmetries of non-volatile memories

paper_content:

The recent advances in non-volatile memory technologies promise the delivery of future high performance and low power computing systems. While these technologies provide attractive features, they exhibit different degrees of asymmetric read/write behavior, resulting in under utilization of the technology benefits in many scenarios. Future systems should be aware of this asymmetry to fully utilize these memory technologies. In this paper, we propose software dispatch, a cross-layer approach to distribute data to appropriate memory resources based on an application's data access characteristics. We demonstrate the application of the proposed technique through a case study system with hybrid memory caches, on which we achieve over 5% performance improvement and 9.8% power savings compared to the state-of-the-art technique.
---
paper_title: Lower-bits cache for low power STT-RAM caches

paper_content:

As power-efficient design becomes more important, spin-transfer torque RAM (STT-RAM) has drawn a lot of attention due to its ability to meet both high performance and low power consumption. However, its high write energy incurs an increase of dynamic power consumption and may offset power saving due to its low static power. This paper proposes a novel technique called lower-bits caches for reducing write activities of STT-RAM L2 caches. Based on the observation that upper bits of data are not changed as frequently as lower bits in most applications, the technique tries to hide frequent bit changes in lower bits from the L2 cache. Experimental results show that our architecture reduced 25 percent of energy consumed by the L2 cache and slightly improved performance at the same time compared to the STT-RAM baseline.
---
paper_title: OAP: an obstruction-aware cache management policy for STT-RAM last-level caches

paper_content:

Emerging memory technologies are explored as potential alternatives to traditional SRAM/DRAM-based memory architecture in future microprocessor designs. Among various emerging memory technologies, Spin-Torque Transfer RAM (STT-RAM) has the benefits of fast read latency, low leakage power, and high density, and therefore has been investigated as a promising candidate for last-level cache (LLC)1. One of the major disadvantages for STT-RAM is the latency and energy overhead associated with the write operations. In particular, a long-latency write operation to STT-RAM cache may obstruct other cache accesses and result in severe performance degradation. Consequently, mitigation techniques to minimize the write overhead are required in order to successfully adopt this new technology for cache design. In this paper, we propose an obstruction-aware cache management policy called OAP. OAP monitors the cache to periodically detect LLC-obstruction processes, and manage the cache accesses from different processes. The experimental results on a 4-core architecture with an 8MB STT-RAM L3 cache shows that the performance can be improved by 14% on average and up to 42%, with a reduction of energy consumption by 64%2.
---
paper_title: AWARE (Asymmetric Write Architecture With REdundant Blocks): A High Write Speed STT-MRAM Cache Architecture

paper_content:

Spin-transfer torque magnetic RAM (STT-MRAM) is a promising memory technology for lower level caches because of its high density and nonvolatile nature. However, the high write latency is a bottleneck to its widespread adoption as the future on-chip memory. In this paper, we propose a new cache architecture-asymmetric write architecture with redundant blocks (AWARE)-that can improve the write latency by taking advantage of the asymmetric write characteristics of 1T-1MTJ STT-MRAM bit-cells. Due to the nature of the storage element in STT-MRAM, the time required for the two-state transitions ( 1→ 0 and 0→ 1) is not identical. In other words, one of the state transitions is slower than the other direction. In conventional cache architecture, the overall write latency is limited by the slower transition. However, the AWARE cache design introduces redundant blocks in each row, and they are preset to the initial state that enables the faster transition. Hence the write operations performed in these redundant blocks are much faster than the conventional write scheme. The write latency in AWARE is improved by 30% over conventional cache architecture with no area penalty in the data array. Moreover, the additional tag bits introduced in this technique result in penalty on the total cache area. In addition, the write energy increases modestly by 7% in the proposed cache design. However, this write-energy increase can be mitigated by sacrificing the cache capacity.
---
paper_title: Process variation aware data management for STT-RAM cache design

paper_content:

The spin-transfer torque random access memory (STT-RAM) has gained increasing attentions for its high density, fast read access, zero standby power, and good scalability. The recently proposed retention-relax design further improves STT-RAM write access performance and makes it even more promising as an on-chip memory technology. Nevertheless, the process variations could affect the writability of STT-RAM cells. The situation for retention-relax design is even more severe. In this paper, we comprehensively study the impact of process variations, including those from both CMOS and magnetic technologies, on key STT-RAM design parameters. Furthermore, we propose process variation aware nonuniform cache access (PVA-NUCA) technique for large STT-RAM cache design. Besides the varying interconnect latencies determined by memory locations, PVA-NUCA compensates write time variations of STT-RAM cells resulted by process variations. Two algorithms, namely, conservative promotion and aggressive prediction, have been introduced and evaluated. A conflict-reduction mechanism is utilized to degrade the data access miss rate caused by conflicts of access-intensive data blocks. Compared to the traditional STT-RAM dynamic nonuniform cache access (DNUCA), our proposed dynamic PVA-NUCA can improve 25.29% of IPC performance and reduce 26.4% of STT-RAM cache energy consumption, with
---
paper_title: An energy efficient cache design using spin torque transfer (STT) RAM

paper_content:

The on-chip memory is a dominant source of power and energy consumption in modern and future processors. This paper explores the use of a new emerging non-volatile memory technology as a replacement for SRAM based lower level caches - Spin Torque Transfer(STT) RAM. While STTRAM achieves a reduction in leakage energy of 90% compared to SRAM, the dynamic energy for a write operation is 2X that of SRAM. Consequently, we propose additional microarchitectural optimizations to reduce overall dynamic energy which achieve an average reduction in dynamic energy over the base case of 30% with a range of 16% to 60% across 10 benchmarks.
---
paper_title: Compiler-assisted refresh minimization for volatile STT-RAM cache

paper_content:

Spin-Transfer Torque RAM (STT-RAM) has been proposed to build on-chip caches because of its attractive features: high storage density and negligible leakage power. Recently, researchers propose to improve the write performance of STT-RAM by relaxing its non-volatility property. To avoid data loss resulting from volatility, refresh schemes are proposed. However, refresh operations consume additional energy. In this paper, we propose to reduce the number of refresh operations through re-arranging program data layout at compilation time. An N-refresh scheme is also proposed. Experimental results show that, on average, the proposedmethods can reduce the number of refresh operations by 73.3%, and reduce the dynamic energy consumption by 27.6%.
---
paper_title: Energy reduction for STT-RAM using early write termination

paper_content:

The emerging Spin Torque Transfer memory (STT-RAM) is a promising candidate for future on-chip caches due to STT-RAM's high density, low leakage, long endurance and high access speed. However, one of the major challenges of STT-RAM is its high write current, which is disadvantageous when used as an on-chip cache since the dynamic power generated is too high. In this paper, we propose Early Write Termination (EWT), a novel technique to significantly reduce write energy with no performance penalty. EWT can be implemented with low complexity and low energy overhead. Our evaluation shows that up to 80% of write energy reduction can be achieved through EWT, resulting 33% less total energy consumption, and 34% reduction in ED2. These results indicate that EWT is an effective and practical scheme to improve the energy efficiency of a STT-RAM cache.
---
paper_title: Implementing a hybrid SRAM / eDRAM NUCA architecture

paper_content:

Advances in technology allowed for integrating DRAM-like structures into the chip, called embedded DRAM (eDRAM). This technology has already been successfully implemented in some GPUs and other graphic-intensive SoC, like game consoles. The most recent processor from IBM®, POWER7, is the first general-purpose processor that integrates an eDRAM module on the chip. In this paper, we propose a hybrid cache architecture that exploits the main features of both memory technologies, speed of SRAM and high density of eDRAM. We demonstrate, that due to the high locality found in emerging applications, a high percentage of data that enters to the on-chip last-level cache are not accessed again before they are evicted. Based on that observation, we propose a placement scheme where re-accessed data blocks are stored in fast, but costly in terms of area and power, SRAM banks, while eDRAM banks store data blo cks that just arrive to the NUCA cache or were demoted from a SRAM bank. We show that a well-balanced SRAM / eDRAM NUCA cache can achieve similar performance results than using a NUCA cache composed of only SRAM banks, but reduces area by 15% and power consumed by 10%. Furthermore, we also explore several alternatives to exploit the area reduction we gain by using the hybrid architecture, resulting in an overall performance improvement of 4%.
---
paper_title: Optimizing bandwidth and power of graphics memory with hybrid memory technologies and adaptive data migration

paper_content:

While GPUs are designed to hide memory latency with massive multi-threading, the tremendous demands for memory bandwidth and power consumption constrain the system performance scaling. In this paper, we propose a hybrid graphics memory architecture with different memory technologies (DRAM, STT-RAM, and RRAM), to improve the memory bandwidth and reduce the power consumption. In addition, we present an adaptive data migration mechanism that exploits various memory access patterns of GPGPU applications for further memory power reduction. We evaluate our design with a set of multi-threaded GPU workloads. Compared to traditional GDDR5 memory, our design leads to 16% of GPU system power reduction, and improves the system throughput and energy efficiency by 12% and 33%.
---
paper_title: High-endurance hybrid cache design in CMP architecture with cache partitioning and access-aware policy

paper_content:

In recent years, NVM (non-volatile memory) technologies, such as STT-RAM (spin transfer torque RAM) and PRAM (phase change RAM), have drawn a lot of attention due to their low leakage and high density. However, both NVMs suffer from high write latency and limited endurance problems. To overcome these problems, the SRAM/NVM hybrid cache architecture has been proposed, and the write pressure on NVM can be mitigated with appropriate write management policy. Moreover, many wear leveling techniques have been proposed to extend the lifetime of NVM in the hybrid cache. In this paper, we proposed a hybrid cache design that includes SRAM cache, STT-RAM cache, and STT-RAM/SRAM hybrid cache banks for CMP (chip multi-processors) architecture. We also propose a partition-level wear leveling scheme and access-aware policies to mitigate unbalanced wear-out of STT-RAM lines within a partition and among different cache partitions. Experimental results show that, our proposed scheme and policies can achieve an average of 89 times improvement in cache lifetime and are able to save 58% power consumption compared to SRAM cache.
---
paper_title: MAC: migration-aware compilation for STT-RAM based hybrid cache in embedded systems

paper_content:

Hybrid caches consisting of both STT-RAM and SRAM have been proposed recently for energy efficiency. To explore the advantages of hybrid cache, most work on hybrid caches employs migration based strategies to dynamically move write-intensive data from STT-RAM to SRAM. Migrations require additional read and write operations for data movement and may lead to significant overheads. To address this issue, this paper proposes a Migration-Aware Compilation (MAC) approach to improve the energy efficiency and performance of STT-RAM based hybrid cache. By re-arranging data layout, the data access pattern in memory blocks is changed such that the number of migrations is reduced without any hardware modification. The reduction of migration overheads in turn improves energy efficiency and performance. The experimental results show that with the proposed approach, on average, the number of write operations on STT-RAM is reduced by 13.4%, the number of migrations is reduced by 16.1%, the total dynamic energy is reduced by 8.5%, and the total latency is reduced by 12.1%.
---
paper_title: Improving energy efficiency of embedded DRAM caches for high-end computing systems

paper_content:

The number of cores in a single chip in the nodes of high-end computing systems is on rise, due, in part, to a number of constraints, such as power consumption. With this, the size of the last level cache (LLC) has also increased significantly. Since LLCs built with SRAM consume high leakage power, power consumption of LLCs is becoming a significant fraction of processor power consumption. To address this issue, researchers have used embedded DRAM (eDRAM) LLCs which consume low leakage power. However, eDRAM caches consume a significant amount of energy in the form of refresh energy. In this paper, we propose ESTEEM, an energy saving technique for embedded DRAM caches. ESTEEM uses dynamic cache reconfiguration to turn off a portion of the cache to save both leakage and refresh energy. It logically divides the cache sets into multiple modules and turns off possibly different number of ways in each module. Microarchitectural simulations confirm that ESTEEM is effective in improving performance and energy efficiency and provides better results compared to a recently-proposed eDRAM cache energy saving technique, namely Refrint. For single and dual-core simulations, the average energy saving in memory subsystem (LLC+main memory) with ESTEEM is 25.8% and 32.6% respectively, and the average weighted speedup is 1.09x and 1.22x respectively. Additional experiments confirm that ESTEEM works well for a wide-range of system and algorithm parameters.
---
paper_title: Design space exploration of workload-specific last-level caches

paper_content:

Leakage power of last-level caches constitute a significant part of overall power consumption. Various circuit-level and technology-based methods have been proposed to reduce cache leakage. However, from a system designer's perspective, for a particular configuration and workload, it is not clear which method will be most effective. In this work, we make a detailed evaluation and comparison of cache energy reduction techniques. Our results show that when energy is very scarce and important, the best results are obtained with highly energy efficient Tunnel-FET caches. When the available energy increases and performance becomes a bigger concern, there is no single winner. While a small number of capacity sensitive workloads benefit from increased capacity of STT-RAM caches, latency sensitive workloads prefer solutions with smaller latency penalties such as drowsy caches.
---
paper_title: DWM-TAPESTRI - an energy efficient all-spin cache using domain wall shift based writes

paper_content:

Spin-based memories are promising candidates for future on-chip memories due to their high density, non-volatility, and very low leakage. However, the high energy and latency of write operations in these memories is a major challenge. In this work, we explore a new approach -- shift based write -- that offers a fast and energy-efficient alternative to performing writes in spin-based memories. We propose DWM-TAPESTRI, a new all-spin cache design that utilizes Domain Wall Memory (DWM) with shift based writes at all levels of the cache hierarchy. The proposed write scheme enables DWM to be used, for the first time, in L1 caches and in tag arrays, where the inefficiency of writes in spin memories has traditionally precluded their use. At the circuit level, we propose bit-cell designs utilizing shift-based writes, which are tailored to the differing requirements of different levels in the cache hierarchy. We also propose pre-shifting as an architectural technique to hide the latency of shift operations that is inherent to DWM. We performed a systematic device-circuit-architecture evaluation of the proposed design. Over a wide range of SPEC 2006 benchmarks, DWM-TAPESTRI achieves 8.2X improvement in energy and 4X improvement in area, with virtually identical performance, compared to an iso-capacity SRAM cache. Compared to an iso-capacity STT-MRAM cache, the proposed design achieves around 1.6X improvement in both area and energy under iso-performance conditions.
---
paper_title: Compiler-assisted preferred caching for embedded systems with STT-RAM based hybrid cache

paper_content:

As technology scales down, energy consumption is becoming a big problem for traditional SRAM-based cache hierarchies. The emerging Spin-Torque Transfer RAM (STT-RAM) is a promising replacement for large on-chip cache due to its ultra low leakage power and high storage density. However, write operations on STT-RAM suffer from considerably higher energy consumption and longer latency than SRAM. Hybrid cache consisting of both SRAM and STT-RAM has been proposed recently for both performance and energy efficiency. Most management strategies for hybrid caches employ migration-based techniques to dynamically move write-intensive data from STT-RAM to SRAM. These techniques lead to extra overheads. In this paper, we propose a compiler-assisted approach, preferred caching, to significantly reduce the migration overhead by giving migration-intensive memory blocks the preference for the SRAM part of the hybrid cache. Furthermore, a data assignment technique is proposed to improve the efficiency of preferred caching. The reduction of migration overhead can in turn improve the performance and energy efficiency of STT-RAM based hybrid cache. The experimental results show that, with the proposed techniques, on average, the number of migrations is reduced by 21.3%, the total latency is reduced by 8.0% and the total dynamic energy is reduced by 10.8%.
---
paper_title: A Hybrid PRAM and STT-RAM Cache Architecture for Extending the Lifetime of PRAM Caches

paper_content:

To extend the lifetime of phase change RAM (PRAM) caches, we propose a hybrid cache architecture that integrates a relatively small capacity of spin transfer torque RAM (STT-RAM) write buffer with a PRAM cache. Our hybrid cache improves the endurance limitation of the PRAM cache by judiciously redirecting the write traffic from an upper memory layer to the STT-RAM write buffer. We have demonstrated through simulation that the proposed hybrid cache outperforms existing write-traffic reduction schemes with the same area overhead. Moreover, our approach is orthogonal to the existing schemes, providing an effective way of investing die area for cache lifetime extension by being used in combination with them.
---
paper_title: Using Magnetic RAM to Build Low-Power and Soft Error-Resilient L1 Cache

paper_content:

Due to its great scalability, fast read access, low leakage power, and nonvolatility, magnetic random access memory (MRAM) appears to be a promising memory technology for on-chip cache memory in microprocessors. However, the write-to-MRAM process is relatively slow and results in high dynamic power consumption. Such inherent disadvantages of MRAM make researchers easily conclude that MRAM can only be used for low-level caches (e.g., L2 or L3 cache), where cache memories are less frequently accessed and slow write to MRAM can be more easily compensated using simple architectural techniques. By developing a hybrid cache architecture, this paper attempts to show that, with appropriate architecture design, MRAM can also be used in L1 cache to improve both the energy efficiency and soft error immunity. The basic idea is to supplement the MRAM L1 cache with several small SRAM buffers, which can substantially mitigate the performance degradation and dynamic energy overhead induced by MRAM write operations. Moreover, the proposed hybrid cache architecture is also an efficient solution to protect cache memory from radiation-induced soft errors, as MRAM is inherently invulnerable to emissive particles. Simulation results show that, with only less than 2% performance degradation, the proposed design approach can reduce the power consumption by up to 76.1% on average compared with the traditional SRAM L1 cache. In addition, the architectural vulnerability factor of L1 data cache is reduced from 28.3% to as low as 0.5%.
---
paper_title: Automatic Feedback Control of Shared Hybrid Caches in 3D Chip Multiprocessors

paper_content:

3D integration enables building caches from different types of technologies such as SRAM, Magnetic RAM (MRAM), DRAM, and Phase-change RAM (PRAM). Hybrid cache architectures (HCAs) have been proposed to take advantage of the benefits offered by these types of technologies. Employing this novel cache architecture to build shared caches in chip multiprocessors (CMPs) can lead to significant performance and power consumption improvements. In this paper, we focus on a 3D CMP design in which the shared last level L2 cache is composed of an MRAM layer and an SRAM layer stacked upon the processing cores and present a control theory centric approach designed to partition this shared hybrid L2 cache space dynamically among concurrently running applications in order to satisfy the application-level performance QoS targets. At each time interval, the two layers of the hybrid L2 cache are partitioned, based on the cache demands made by the controllers of the applications, to satisfy the specified performance targets. We evaluate our feedback control based scheme using various workloads. Our experimental evaluation shows that the proposed scheme is able to satisfy the specified performance QoS in most of the tested cases, by partitioning the hybrid cache space of the 3D CMP among co-runner applications.
---
paper_title: Versatile refresh: low complexity refresh scheduling for high-throughput multi-banked eDRAM

paper_content:

Multi-banked embedded DRAM (eDRAM) has become increasingly popular in high-performance systems. However, the data retention problem of eDRAM is exacerbated by the larger number of banks and the high-performance environment in which it is deployed: The data retention time of each memory cell decreases while the number of cells to be refreshed increases. For this, multi-bank designs offer a concurrent refresh mode, where idle banks can be refreshed concurrently during read and write operations. However, conventional techniques such as periodically scheduling refreshes---with priority given to refreshes in case of conflicts with reads or writes---have variable performance, increase read latency, and can perform poorly in worst case memory access patterns. We propose a novel refresh scheduling algorithm that is low-complexity, produces near-optimal throughput with universal guarantees, and is tolerant to bursty memory access patterns. The central idea is to decouple the scheduler into two simple-to-implement modules: one determines which cell to refresh next and the other determines when to force an idle cycle in all banks. We derive necessary and sufficient conditions to guarantee data integrity for all access patterns, with any given number of banks, rows per bank, read/write ports and data retention time. Our analysis shows that there is a tradeoff between refresh overhead and burst tolerance and characterizes this tradeoff precisely. The algorithm is shown to be near-optimal and achieves, for instance, 76.6% reduction in worst-case refresh overhead from the periodic refresh algorithm for a 250MHz eDRAM with 10us retention time and 16 banks each with 128 rows. Simulations with Apex-Map synthetic benchmarks and switch lookup table traffic show that VR can almost completely hide the refresh overhead for memory accesses with moderate-to-high multiplexing across memory banks.
---
paper_title: Multi-level magnetic RAM using domain wall shift for energy-efficient, high-density caches

paper_content:

Spin-based devices promise to revolutionize computing platforms by enabling high-density, low-leakage memories. However, stringent tradeoffs between critical design metrics such as read and write stability, reliability, density, performance and energy-efficiency limit the efficiency of conventional spin-transfer- torque devices and bit-cells. We propose a new multi-level cell design with domain wall magnets (DWM-MLC) that significantly improves upon the read/write performance, density, and write energy consumption of conventional spin memories. The fundamental design tradeoff between read and write operations are addressed in DWM-MLC by decoupling the read and write paths, thereby allowing separate optimization for reads and writes. A thicker tunneling oxide is used for higher readability, while a domain-wall-shift (DWS) based write mechanism is used to improve write speed and energy. The storage of multiple bits per cell and the ability to use smaller transistors lead to a net improvement in density compared to conventional spin memories. We perform a systematic evaluation of DWM-MLC at different levels of design abstraction. At the circuit level, DWM-MLC achieves 2X improvement in density, read energy and read latency over its 1-bit counterpart. We evaluate an "all-spin" cache hierarchy that uses DWM-MLC for both L1 and L2, resulting in 4.4X (1.7X) area improvement and 10X (2X) energy reduction at isoperformance over SRAM (STT-MRAM).
---
paper_title: Memories: Exploiting Them and Developing Them

paper_content:

The disciplines of process development, circuit design, and microarchitecture too often remain isolated. This paper on memories explores the benefit of their unification by taking a retrospective look at a L1 cache memory design, a current view of embedded DRAM, and a speculative peek at emerging memories (e.g. MTJ MRAM). In the midst, a novel refresh operation is proposed for DRAM that relies on read and write activity ongoing within a cache to refresh the DRAM.
---
paper_title: Analysis and Optimization of Thermal Effect on STT-RAM Based 3-D Stacked Cache Design

paper_content:

Spin-Transfer Torque Random Access Memory (STT-RAM) has been proved a promising emerging nonvolatile memory technology suitable for many applications such as cache memory of CPU. Simulation results show that the switching time of Magnetic Tunnel Junction (MTJ), which is the core element of the STT-RAM cell, varies when the temperature changes. In this paper, we study the thermal effect on switching time of STT-RAM cell, and it is showed that when temperature changes from 300K to 375K, the required write pulse period to achieve 1E-8 bit error rate (BER) increases from 10.02ns to 15.04ns under 45nm technology. When STT-RAM is used as 3-D stacked L3 cache, the required write pulse period ranges from 11.42ns to 14.68ns due to temperature variation caused by the CPU core layer. If the thermal effect is not considered, the BER of the hottest region will significantly increase to 1E-4. Based on these observations, an optimization design with Dynamic Temperature Aware Write Access is proposed, to increase the efficiency of accessing a 3-D stacked STT-RAM cache, as well as achieve the target BER. Compared to a conventional design, the proposed scheme can improve the CPU performance by 3.8% and reduce the write energy consumption of the STT-RAM cache by 4.8%.
---
paper_title: LASIC: Loop-Aware Sleepy Instruction Caches Based on STT-RAM Technology

paper_content:

This brief presents an approach to reduce static power consumption in peripheral circuits of spin-transfer torque RAM (STT-RAM) instruction caches. It is based on the key observation that only a small set of instructions is accessed inside a program loop. We propose to add a small static RAM cache called loop cache between the processor and the L1 instruction cache made of STT-RAM. When the loop cache has an entire loop cached, the L1 instruction cache can be turned off to save energy during the execution of the loop. Experimental results show that the proposed approach achieves 49% reduction in energy consumption over the STT-RAM baseline.
---
paper_title: Exploring the vulnerability of CMPs to soft errors with 3D stacked non-volatile memory

paper_content:

Spin-transfer Torque Random Access Memory (STT-RAM) emerges for on-chip memory in microprocessor architectures. Thanks to the magnetic field based storage STT-RAM cells have immunity to radiation induced soft errors that affect electrical charge based data storage, which is a major challenge in SRAM based caches in current microprocessors. In this study we explore the soft error resilience benefits and design trade offs of 3D-stacked STT-RAM for multi-core architectures. We use 3D stacking as an enabler for modular integration of STT-RAM caches with minimum disruption in the baseline processor design flow, while providing further interconnectivity and capacity advantages. We take an in-depth look at alternative replacement schemes in terms of performance, power, temperature, and reliability trade-offs to capture the multi-variable optimization challenges microprocessor architectures face. We analyze and compare the characteristics of STT-RAM, SRAM, and DRAM alternatives for various levels of the cache hierarchy in terms of reliability.
---
paper_title: Analyzing the optimal ratio of SRAM banks in hybrid caches

paper_content:

Cache memories have been typically implemented with Static Random Access Memory (SRAM) technology. This technology presents a fast access time but high energy consumption and low density. As opposite, the recently appeared embedded Dynamic RAM (eDRAM) technology allows caches to be built with lower energy and area, although with a slower access time. The eDRAM technology provides important leakage and area savings, especially in huge Last-Level Caches (LLCs), which occupy almost half the silicon area in some recent microprocessors. This paper proposes a novel hybrid LLC, which combines SRAM and eDRAM banks to address the trade-off among performance, energy, and area. To this end, we explore the optimal percentage of SRAM and eDRAM banks that achieves the best target trade-off. Architectural mechanisms have been devised to keep the most likely accessed blocks in fast SRAM banks as well as to avoid unnecessary destructive reads. Experimental results show that, compared to a conventional SRAM LLC with the same storage capacity, performance degradation does not surpass, on average, 2.9% (even with 12.5% of banks built with SRAM technology), whereas area savings can be as high as 46% for a 1MB-16way LLC. For a 45nm technology node, the energy-delay squared product confirms that a hybrid cache is a better design than the conventional SRAM cache regardless the number of eDRAM banks, and also better than a conventional eDRAM cache when the number of SRAM banks is a quarter or an eighth of the cache banks.
---
paper_title: Refrint: Intelligent refresh to minimize power in on-chip multiprocessor cache hierarchies

paper_content:

As manycores use dynamic energy ever more efficiently, static power consumption becomes a major concern. In particular, in a large manycore running at a low voltage, leakage in on-chip memory modules contributes substantially to the chip's power draw. This is unfortunate, given that, intuitively, the large multi-level cache hierarchy of a manycore is likely to contain a lot of useless data. An effective way to reduce this problem is to use a low-leakage technology such as embedded DRAM (eDRAM). However, eDRAM requires refresh. In this paper, we examine the opportunity of minimizing on-chip memory power further by intelligently refreshing on-chip eDRAM. We present Refrint, a simple approach to perform fine-grained, intelligent refresh of on-chip eDRAM multiprocessor cache hierarchies. We introduce the Refrint algorithms and microarchitecture. We evaluate Refrint in a simulated manycore running 16-threaded parallel applications. We show that an eDRAM-based memory hierarchy with Refrint consumes only 30% of the energy of a conventional SRAM-based memory hierarchy, and induces a slowdown of only 6%. In contrast, an eDRAM-based memory hierarchy without Refrint consumes 56% of the energy of the conventional memory hierarchy, inducing a slowdown of 25%.
---
paper_title: Cross-layer racetrack memory design for ultra high density and low power consumption

paper_content:

The racetrack memory technology utilizes magnetic domains along a nanoscopic wire to obtain ultra-high data storage density. The recent success in the planar racetrack nanowire promised its fabrication feasibility and future scalability, bringing more design challenges and opportunities. In this paper, we initialize the optimization of racetrack memory embracing design considerations across multiple layers, including cell design, array structure, architecture organization, and data management. Our evaluation shows that racetrack memory based cache can achieve 6.4× area reduction, 25% performance enhancement, and 62% energy saving, compared to STT-RAM cache design. The benefit over SRAM technology is even more significant.
---
paper_title: Multi retention level STT-RAM cache designs with a dynamic refresh scheme

paper_content:

Spin-transfer torque random access memory (STT-RAM) has received increasing attention because of its attractive features: good scalability, zero standby power, non-volatility and radiation hardness. The use of STT-RAM technology in the last level on-chip caches has been proposed as it minimizes cache leakage power with technology scaling down. Furthermore, the cell area of STT-RAM is only 1/9 ~ 1/3 that of SRAM. This allows for a much larger cache with the same die footprint, improving overall system performance through reducing cache misses. However, deploying STT-RAM technology in L1 caches is challenging because of the long and power-consuming write operations. In this paper, we propose both L1 and lower level cache designs that use STT-RAM. In particular, our designs use STT-RAM cells with various data retention time and write performances, made possible by different magnetic tunneling junction (MTJ) designs. For the fast STT-RAM bits with reduced data retention time, a counter controlled dynamic refresh scheme is proposed to maintain the data validity. Our dynamic scheme saves more than 80% refresh energy compared to the simple refresh scheme proposed in previous works. A L1 cache built with ultra low retention STT-RAM coupled with our proposed dynamic refresh scheme can achieve 9.2% in performance improvement, and saves up to 30% of the total energy when compared to one that uses traditional SRAM. For lower level caches with relative large cache capacity, we propose a data migration scheme that moves data between portions of the cache with different retention characteristics so as to maximize the performance and power benefits. Our experiments show that on the average, our proposed multi retention level STT-RAM cache reduces 30 ~ 70% of the total energy compared to previous works, while improving IPC performance for both 2-level and 3-level cache hierarchy.
---
paper_title: Coordinating prefetching and STT-RAM based last-level cache management for multicore systems

paper_content:

Data prefetching is a common mechanism to mitigate the bottleneck of off-chip memory bandwidth in modern computing systems. Unfortunately, the side effects of prefetching are an additional burden on off-chip communication and increased cache write operations. With the proposal of spin-transfer torque random access memory (STT-RAM) based last-level caches (LLCs) for their high density and low power consumption, the increase of write pressure to the cache from prefetching coupled with the characteristically long write access compared with traditional SRAM caches exacerbates the performance cost of prefetching schemes. In this work, we propose two orthogonal techniques to reduce the negative performance impact induced by aggressive prefetching on multicore systems employing STT-RAM based LLC. First, basic priority assignment prioritizes the different types of access requests of LLC by their criticality and responds to them based on priority. Second, priority boosting differentiates requests by application and prioritizes the relatively few requests from applications with non-intensive accesses to the LLC, which usually creates the most severe performance degradation in multi-core systems. Combining these two prioritization policies can alleviate the negative effect induced by aggressive prefetching. Our results show that these techniques can achieve an 8.3 average application speedup compared to a baseline, prefetch only design without prioritization.
---
paper_title: LastingNVCache: A Technique for Improving the Lifetime of Non-volatile Caches

paper_content:

Use of non-volatile memory (NVM) devices such as resistive RAM (ReRAM) and spin transfer torque RAM (STT-RAM) for designing on-chip caches holds the promise of providing a high-density, low-leakage alternative to SRAM. However, low write endurance of NVMs, along with the write-variation introduced by existing cache management schemes significantly limits the lifetime of NVM caches. We present LastingNVCache, a technique for improving the cache lifetime by mitigating the intra-set write variation. LastingNVCache works on the key idea that by periodically flushing a frequently-written data-item, next time the block can be made to load into a cold block in the set. Through this, the future writes to that data-item can be redirected from a hot block to a cold block, which leads to improvement in the cache lifetime. Microarchitectural simulations have shown that LastingNVCache provides 6.36X, 9.79X, and 10.94X improvement in lifetime for single, dual and quad-core systems, respectively. Also, its implementation overhead is small and it outperforms a recently proposed technique for improving lifetime of NVM caches.
---
paper_title: Low-current probabilistic writes for power-efficient STT-RAM caches

paper_content:

MRAM has emerged as one of the most attractive non-volatile solutions due to fast read access, low leakage power, high bit density, and long endurance. However, the high power consumption of write operations remains a barrier to the commercial adoption of MRAM technology. This paper addresses this problem by introducing low-current probabilistic writes (LCPW), a technique that reduces write access energy by lowering the amplitude of the write current pulse. Although low current pulses no longer guarantee successful bit write operations, we propose and evaluate a simple technique to ensure correctness and achieve significant power reduction over a typical MRAM implementation.
---
paper_title: Exploiting Heterogeneity for Energy Efficiency in Chip Multiprocessors

paper_content:

Heterogeneous multicores are envisioned to be a promising design paradigm to combat today's challenges of power, memory, and reliability walls that are impeding chip design using deep submicron technology. Future multicores are expected to integrate multiple different cores, including GPGPUs, custom accelerators and configurable cores. In this paper, we introduce an important dimension-technology-using which heterogeneity can be introduced in multicores to improve their energy-performance envelope. Specifically, we analyze the benefits of heterogenous technologies for processor cores and cache subsystems. We discuss two promising device candidates (Tunnel-FET and Magnetic-RAM) for introducing technological diversity in the multicores and analyze their integration in the processor and cache hierarchy in detail. Our analysis shows that introducing such a kind of heterogeneity can significantly enhance the performance and energy behavior of future multicore systems.
---
paper_title: Power-performance co-optimization of throughput core architecture using resistive memory

paper_content:

Massively parallel computing on throughput computers such as GPUs requires myriad memory accesses to register files, on-chip scratchpad, caches, and off-chip DRAM. Unlike CPUs, these processors have a large register file and on-chip scratchpad memory, which consume a significant portion of compute core power (35%-45%). In this paper, we introduce novel throughput architecture by integrating resistive memory (Spin Transfer Torque RAM) inside the compute core, which reduces leakage significantly, but introduces write power overhead and longer write latencies in GPU shared memory and register file accesses. We enhance the compute core by introducing register file organization with differential memory update mechanism to remove update redundancy during write operations. Furthermore, using merged register-write-mechanism and write-back buffer, we coalesce multithreaded GPU register write accesses to save write energy. In addition, we introduce hybrid shared memory design using SRAM and STT-MRAM that provides significant leakage/dynamic power savings without affecting performance. On average, across 23 GPGPU/graphics workloads, our schemes save 46% dynamic power due to register access (83% leakage power saving) with negligible performance degradation. On average, hybrid shared memory provides 10% reduction in dynamic power with maximum 1.6× performance improvement for the current workloads at no additional area overhead.
---
paper_title: An hybrid eDRAM/SRAM macrocell to implement first-level data caches

paper_content:

SRAM and DRAM cells have been the predominant technologies used to implement memory cells in computer systems, each one having its advantages and shortcomings. SRAM cells are faster and require no refresh since reads are not destructive. In contrast, DRAM cells provide higher density and minimal leakage energy since there are no paths within the cell from Vdd to ground. Recently, DRAM cells have been embedded in logic-based technology, thus overcoming the speed limit of typical DRAM cells. In this paper we propose an n-bit macrocell that implements one static cell, and n-1 dynamic cells. This cell is aimed at being used in an n-way set-associative first-level data cache. Our study shows that in a four-way set-associative cache with this macrocell compared to an SRAM based with the same capacity, leakage is reduced by about 75% and area more than half with a minimal impact on performance. Architectural mechanisms have also been devised to avoid refresh logic. Experimental results show that no performance is lost when the retention time is larger than 50K processor cycles. In addition, the proposed delayed writeback policy that avoids refreshing performs a similar amount of writebacks than a conventional cache with the same organization, so no power wasting is incurred.
---
paper_title: A Cache Reconfiguration Approach for Saving Leakage and Refresh Energy in Embedded DRAM Caches

paper_content:

In recent years, the size and leakage energy consumption of large last level caches (LLCs) has increased. To address this, embedded DRAM (eDRAM) caches have been considered which have lower leakage energy consumption; however eDRAM caches consume a significant amount of energy in the form of refresh energy. In this paper, we present a technique for saving both leakage and refresh energy in eDRAM caches. We use dynamic cache reconfiguration approach to intelligently turn-off part of the cache to save leakage energy and refresh only valid data of the active (i.e. not turned-off) cache to save refresh energy. We evaluate our technique using an x86-64 simulator and SPEC2006 benchmarks and compare it with a recently proposed technique for saving refresh energy, named Refrint. The experiments have shown that our technique provides better performance and energy efficiency than Refrint. Using our technique, for a 2MB LLC and 40 micro-seconds eDRAM refresh period, the average saving in energy over eDRAM baseline (which periodically refreshes all cache lines) is 22.8%.
---
paper_title: Using Cache-coloring to Mitigate Inter-set Write Variation in Non-volatile Caches

paper_content:

In recent years, researchers have explored use of non-volatile devices such as STT-RAM (spin torque transfer RAM) for designing on-chip caches, since they provide high density and consume low leakage power. A common limitation of all non-volatile devices is their limited write endurance. Further, since existing cache management policies are write-variation unaware, excessive writes to a few blocks may lead to a quick failure of the whole cache. We propose an architectural technique for wear-leveling of non-volatile last level caches (LLCs). Our technique uses cache-coloring approach which adds a software-controlled mapping layer between groups of physical pages and cache sets. Periodically the mapping is altered to ensure that write-traffic can be spread uniformly to different sets of the cache to achieve wear-leveling. Simulations performed with an x86-64 simulator and SPEC2006 benchmarks show that our technique reduces the worst-case writes to cache blocks and thus improves the cache lifetime by 4.07X.
---
paper_title: Mosaic: Exploiting the spatial locality of process variation to reduce refresh energy in on-chip eDRAM modules

paper_content:

EDRAM cells require periodic refresh, which ends up consuming substantial energy for large last-level caches. In practice, it is well known that different eDRAM cells can exhibit very different charge-retention properties. Unfortunately, current systems pessimistically assume worst-case retention times, and end up refreshing all the cells at a conservatively-high rate. In this paper, we propose an alternative approach. We use known facts about the factors that determine the retention properties of cells to build a new model of eDRAM retention times. The model is called Mosaic. The model shows that the retention times of cells in large eDRAM modules exhibit spatial correlation. Therefore, we logically divide the eDRAM module into regions or tiles, profile the retention properties of each tile, and program their refresh requirements in small counters in the cache controller. With this architecture, also called Mosaic, we refresh each tile at a different rate. The result is a 20x reduction in the number of refreshes in large eDRAM modules — practically eliminating refresh as a source of energy consumption.
---
paper_title: Bandwidth-aware reconfigurable cache design with hybrid memory technologies

paper_content:

In chip-multiprocessor (CMP) designs, limited memory bandwidth is a potential bottleneck of the system performance. New memory technologies, such as spin-torque-transfer memory (STT-RAM), resistive memory (RRAM), and embedded DRAM (eDRAM), are promising on-chip memory solutions for CMPs. In this paper, we propose a bandwidth-aware re-configurable cache hierarchy (BARCH) with hybrid memory technologies. BARCH consists of a hybrid cache hierarchy, a reconfiguration mechanism, and a statistical prediction engine. Our hybrid cache hierarchy chooses different memory technologies to configure each level so that the bandwidth provided by the overall hierarchy is optimized. Furthermore, we present a reconfiguration mechanism to dynamically adapt the cache space of each level based on the predicted bandwidth demands of different applications, which is guaranteed by our prediction engine. We evaluate the system performance gain obtained by our method with a set of multithreaded and multiprogrammed applications. Compared to traditional SRAM-based cache designs, our proposed design improves the system throughput by 58% and 14% for multithreaded and multiprogrammed applications, respectively.1
---
paper_title: A low-power phase change memory based hybrid cache architecture

paper_content:

Sub-threshold leakage in SRAM based cache memories is becoming a predominant source of power consumption in deep-sub micron CMOS designs. Phase Change Random Access Memory (PRAM), a high density, fast access, non-volatile memory is being considered as a candidate for future universal memory technologies. In this paper, we investigate the architectural challenges in integrating a PRAM based memory into the conventional cache hierarchy. First, we develop PRAM cache delay and energy models. We then propose a hybrid PRAM architecture for L1 instruction caches on embedded processors. We also propose a PRAM based unified cache architecture for L2 caches on high-end microprocessors. Finally, we evaluate the proposed architectures, in terms of area, performance, and energy. The experimental results show that the PRAM based cache architectures achieve close to 80% reduction in the leakage energy consumption of a L1-L2 cache hierarchy.
---
paper_title: Static and dynamic co-optimizations for blocks mapping in hybrid caches

paper_content:

In this paper, a combined static and dynamic scheme is proposed to optimize the block placement for endurance and energy-efficiency in a hybrid SRAM and STT-RAM cache. With the proposed scheme, STT-RAM endurance is maximized while performance is maintained. We use the compiler to provide static hints to guide initial data placement, and use the hardware to correct the hints based on the run-time cache behavior. Experimental results show that the combined scheme improves the endurance by 23.9x and 5.9x compared to pure static and pure dynamic optimizations respectively. Furthermore, the system energy can be reduced by 17% compared to pure dynamic optimization through minimizing STT-RAM writes.
---
paper_title: Write intensity prediction for energy-efficient non-volatile caches

paper_content:

This paper presents a novel concept called write intensity prediction for energy-efficient non-volatile caches as well as the architecture that implements the concept. The key idea is to correlate write intensity of cache blocks with addresses of memory access instructions that incur cache misses of those blocks. The predictor keeps track of instructions that tend to load write-intensive blocks and utilizes that information to predict write intensity of blocks. Based on this concept, we propose a block placement strategy driven by write intensity prediction for SRAM/STT-RAM hybrid caches. Experimental results show that the proposed approach reduces write energy consumption by 55% on average compared to the existing hybrid cache architecture.
---
paper_title: Relaxing non-volatility for fast and energy-efficient STT-RAM caches

paper_content:

Spin-Transfer Torque RAM (STT-RAM) is an emerging non-volatile memory technology that is a potential universal memory that could replace SRAM in processor caches. This paper presents a novel approach for redesigning STT-RAM memory cells to reduce the high dynamic energy and slow write latencies. We lower the retention time by reducing the planar area of the cell, thereby reducing the write current, which we then use with CACTI to design caches and memories. We simulate quad-core processor designs using a combination of SRAM- and STT-RAM-based caches. Since ultra-low retention STT-RAM may lose data, we also provide a preliminary evaluation for a simple, DRAMstyle refresh policy. We found that a pure STT-RAM cache hierarchy provides the best energy efficiency, though a hybrid design of SRAM-based L1 caches with reduced-retention STT-RAM L2 and L3 caches eliminates performance loss while still reducing the energy-delay product by more than 70%.
---
paper_title: TapeCache: a high density, energy efficient cache based on domain wall memory

paper_content:

Domain Wall Memory (DWM) is a recently developed spin-based memory technology in which several bits of data are densely packed into the domains of a ferromagnetic wire. DWM has shown great promise in enabling non-volatile memory with unprecedented density and high energy efficiency. In this work, we propose TapeCache, a first attempt to employ DWMs as last-level caches in general purpose computing platforms. DWMs enable much higher density compared to SRAM, DRAM, and other spin-based memory technologies such as STT-MRAM. However, they also pose unique challenges such as serial access to the bits stored in a DWM cell, leading to variable access latencies. We propose a novel circuit-architecture co-design for TapeCache, consisting of (i) a multi-port DWM macro-cell optimized for read operations considering the asymmetry in applications' read/write characteristics, and (ii) a new cache organization and suitable management policies that mitigate the performance penalty arising from serial access to bits in a macro-cell. Over a wide range of SPEC 2006 benchmarks, TapeCache achieves 7.8X improvement in area, an average energy improvement of 7.3X, and an average performance improvement of 1.2% compared to an iso-capacity SRAM cache. Compared to an iso-capacity STT-MRAM cache, TapeCache obtains 2.3X improvement in area and 1.4X average energy savings with virtually identical performance.
---
paper_title: Energy-efficient Spin-Transfer Torque RAM cache exploiting additional all-zero-data flags

paper_content:

Large on-chip caches account for a considerable fraction of the total energy consumption in modern microprocessors. In this context, emerging Spin-Transfer Torque RAM (STT-RAM) has been regarded as a promising candidate to replace large on-chip SRAM caches in virtue of its nature of the zero leakage. However, large energy requirement of STT-RAM on write operations, resulting in a huge amount of dynamic energy consumption, precludes it from application to on-chip cache designs. In order to reduce the write energy of the STT-RAM cache thereby the total energy consumption, this paper provides an architectural technique which exploits the fact that many applications process a large number of zero data. The proposed design appends additional flags in cache tag arrays and set these additional bits if the corresponding data in the cache line is the zero-valued data in which all data bits are zero. Our experimental results show that the proposed cache design can reduce 73.78% and 69.30% of the dynamic energy on write operations at the byte and word granularities, respectively; total energy consumption reduced by 36.18% and 42.51%, respectively. In addition to the energy reduction, performance evaluation results indicate that the proposed cache improves the processor performance by 5.44% on average.
---
paper_title: Coding Last Level STT-RAM Cache for High Endurance and Low Power

paper_content:

STT-RAM technology has recently emerged as one of the most promising memory technologies. However, its major problems, limited write endurance and high write energy, are still preventing it from being used as a drop-in replacement of SRAM cache. In this paper, we propose a novel coding scheme for STT-RAM last level cache based on the concept of value locality. We reduce switching probability in cache by swapping common patterns with limited weight codes (LWC) to make writes less often as well as more uniform. We also define some policies for swapping these patterns. Our evaluation shows that bit write variance in memory cells can be reduced by about 20% on average resulting in a more uniform wear-out directly enhancing lifetime and improving cell reliability. In addition, writes in cache lines can be reduced by about 12% compared to one of the most effective circuit level techniques known as early write termination (EWT) [12]. Our method increases memory hierarchy access time by about 0.08% on average, which is negligible. We have shown that our method doesn't adversely affect last level cache energy-delay2. The non-uniformity caused by the coding scheme can be used for another coding scheme at main memory or L1 cache depending on their technologies.
---
paper_title: Circuit and microarchitecture evaluation of 3D stacking magnetic RAM (MRAM) as a universal memory replacement

paper_content:

Magnetic random access memory (MRAM) has been considered as a promising memory technology due to many attractive properties. Integrating MRAM with CMOS logic may incur extra manufacture cost, due to its hybrid magnetic-CMOS fabrication process. Stacking MRAM on top of CMOS logics using 3D integration is a way to minimize this cost overhead. In this paper, we discuss the circuit design issues for MRAM, and present the MRAM cache model. Based on the model, we compare MRAM against SRAM and DRAM in terms of area, performance, and energy. Finally we conduct architectural evaluation for 3D microprocessor stacking with MRAM. The experimental results show that MRAM stacking offers competitive IPC performance with a large reduction in power consumption compared to SRAM and DRAM counterparts.
---
paper_title: Prediction table based management policy for STT-RAM and SRAM hybrid cache

paper_content:

STT-RAM has emerged as one of the most promising candidates for future on-chip cache due to STT-RAM's high density and low leakage. However, the STT-RAM has many disadvantages, such as: high write energy and long writing latency. To alleviate the problems, many STT-RAM and SRAM hybrid cache architectures have been proposed. In this paper, we propose a prediction table based hybrid L2 cache management policy named PTHCM. By adding a prediction table to the hybrid cache, the access information of each cache line can be recorded and we can predict the cache lines that have become dead in SRAM, lines that will be less-written and frequently-written in advance. Based on these information provided by prediction table, we propose a new cache replacement and management policy for hybrid cache architecture which replaces the dead cache lines in SRAM to improve the utilization ratio of SRAM, places frequently-written lines into SRAM and swaps out less-written lines from SRAM as early as possible. Compared with a baseline hybrid cache design, our design reduces 23.3% of the write operations to STT-RAM, and achieves 8.2% power reduction.
---
paper_title: Hybrid cache architecture with disparate memory technologies

paper_content:

Caching techniques have been an efficient mechanism for mitigating the effects of the processor-memory speed gap. Traditional multi-level SRAM-based cache hierarchies, especially in the context of chip multiprocessors (CMPs), present many challenges in area requirements, core-to-cache balance, power consumption, and design complexity. New advancements in technology enable caches to be built from other technologies, such as Embedded DRAM (EDRAM), Magnetic RAM (MRAM), and Phase-change RAM (PRAM), in both 2D chips or 3D stacked chips. Caches fabricated in these technologies offer dramatically different power and performance characteristics when compared with SRAM-based caches, particularly in the areas of access latency, cell density, and overall power consumption. In this paper, we propose to take advantage of the best characteristics that each technology offers, through the use of Hybrid Cache Architecture (HCA) designs. We discuss and evaluate two types of hybrid cache architectures: inter cache Level HCA (LHCA), in which the levels in a cache hierarchy can be made of disparate memory technologies; and intra cache level or cache Region based HCA (RHCA), where a single level of cache can be partitioned into multiple regions, each of a different memory technology. We have studied a number of different HCA architectures and explored the potential of hardware support for intra-cache data movement and power consumption management within HCA caches. Utilizing a full-system simulator that has been validated against real hardware, we demonstrate that an LHCA design can provide a geometric mean 7% IPC improvement over a baseline 3-level SRAM cache design under the same area constraint across a collection of 25 workloads. A more aggressive RHCA-based design provides 12% IPC improvement over the baseline. Finally, a 2-layer 3D cache stack (3DHCA) of high density memory technology within the same chip footprint gives 18% IPC improvement over the baseline. Furthermore, up to 70% reduction in power consumption over a baseline SRAM-only design is achieved.
---
paper_title: On-chip caches built on multilevel spin-transfer torque RAM cells and its optimizations

paper_content:

It has been predicted that a processor's caches could occupy as much as 90p of chip area a few technology nodes from the current ones. In this article, we investigate the use of multilevel spin-transfer torque RAM (STT-RAM) cells in the design of processor caches. We start with examining the access (read and write) scheme for multilevel cell (MLC) STT-RAM from a circuit design perspective, detailing the read and write circuits. Compared to traditional SRAM caches, a multilevel cell (MLC) STT-RAM cache design is denser, fast, and requires less energy. However, a number of critical architecture-level issues remain to be solved before MLC STT-RAM technology can be deployed in processor caches. We shall offer solutions to the issue of bit encoding as well as tackle the write endurance problem. In particular, the latter has been neglected in previous works on STT-RAM caches. We propose a set remapping scheme that can potentially prolong the lifetime of a MLC STT-RAM cache by 80× on average. Furthermore, a method for recovering the performance that may be lost in some applications due to set remapping is proposed. The impacts of process variations of the MLC STT-RAM cell on the robustness of the memory hierarchy is also discussed, together with various enhancement techniques, namely, ECC and design redundancy.
---
paper_title: Technology comparison for large last-level caches (L3Cs): Low-leakage SRAM, low write-energy STT-RAM, and refresh-optimized eDRAM

paper_content:

Large last-level caches (L3Cs) are frequently used to bridge the performance and power gap between processor and memory. Although traditional processors implement caches as SRAMs, technologies such as STT-RAM (MRAM), and eDRAM have been used and/or considered for the implementation of L3Cs. Each of these technologies has inherent weaknesses: SRAM is relatively low density and has high leakage current; STT-RAM has high write latency and write energy consumption; and eDRAM requires refresh operations. As future processors are expected to have larger last-level caches, the goal of this paper is to study the trade-offs associated with using each of these technologies to implement L3Cs. In order to make useful comparisons between SRAM, STTRAM, and eDRAM L3Cs, we model them in detail and apply low power techniques to each of these technologies to address their respective weaknesses. We optimize SRAM for low leakage and optimize STT-RAM for low write energy. Moreover, we classify eDRAM refresh-reduction schemes into two categories and demonstrate the effectiveness of using dead-line prediction to eliminate unnecessary refreshes. A comparison of these technologies through full-system simulation shows that the proposed refresh-reduction method makes eDRAM a viable, energy-efficient technology for implementing L3Cs.
---
paper_title: Energy- and endurance-aware design of phase change memory caches

paper_content:

Phase change memory (PCM) is one of the most promising technology among emerging non-volatile random access memory technologies. Implementing a cache memory using PCM provides many benefits such as high density, non-volatility, low leakage power, and high immunity to soft error. However, its disadvantages such as high write latency, high write energy, and limited write endurance prevent it from being used as a drop-in replacement of an SRAM cache. In this paper, we study a set of techniques to design an energy- and endurance-aware PCM cache. We also modeled the timing, energy, endurance, and area of PCM caches and integrated them into a PCM cache simulator to evaluate the techniques. Experiments show that our PCM cache design can achieve 8% of energy saving and 3.8 years of lifetime compared with a baseline PCM cache having less than a hour of lifetime.
---
paper_title: High-endurance and performance-efficient design of hybrid cache architectures through adaptive line replacement

paper_content:

In this paper, we propose a run-time strategy for managing writes onto last level cache in chip multiprocessors where STT-RAM memory is used as baseline technology. To this end, we assume that each cache set is decomposed into limited SRAM lines and large number of STT-RAM lines. SRAM lines are target of frequently-written data and rarely-written or read-only ones are pushed into STT-RAM. As a novel contribution, a low-overhead, fully-hardware technique is utilized to detect write-intensive data blocks of working set and place them into SRAM lines while the remaining data blocks are candidates to be remapped onto STT-RAM blocks during system operation. Therefore, the achieved cache architecture has large capacity and consumes near zero leakage energy using STT-RAM array; while dynamic write energy, acceptable write latency, and long lifetime is guaranteed via SRAM array. Results of full-system simulation for a quad-core CMP running PARSEC-2 benchmark suit confirm an average of 49 times improvement in cache lifetime and more than 50% reduction in cache power consumption when compared to baseline configurations.
---
paper_title: Exploiting set-level write non-uniformity for energy-efficient NVM-based hybrid cache

paper_content:

Hybrid cache architectures have been proposed to mitigate the increasing on-chip power dissipation through the exploitation of the emerging non-volatile memories (NVMs). To overcome the high energy and long latency associated with write operations of NVMs, a small SRAM is typically incorporated into the hybrid cache for accommodating write-intensive cache blocks. How to efficiently manage this SRAM and manipulate the write operations are crucial to the performance of the hybrid cache. In this paper, we first present our observation that the intensity of write operations on different cache sets is usually non-uniform for real applications, such as multimedia, multi-programmed, multithreaded applications. The previously proposed hybrid cache schemes can not efficiently and symmetrically utilize the small SRAM to accommodate such widely-existing non-uniform writes on cache sets. Based on this observation, we propose a novel hybrid cache design, Dual Associative Hybrid Cache (denoted as DAHYC), as well as the corresponding cache management policy. By organizing the SRAM blocks in the hybrid cache as a semi-independent set-associative cache, several hybrid cache sets can efficiently share and cooperatively utilize their SRAM blocks, instead of exclusively utilizing the SRAM blocks in each cache set in previous hybrid cache schemes, to boost power-efficiency. Through prudently manipulating the locality information of SRAM blocks in both the NVM sets and the SRAM sets, the proposed cache management policy also delivers high-performance. Experimental results show that, compared with previous works, the DAHYC can reduce the dynamic power of the hybrid cache by 24.8% on average and up to 54% for SPEC2000 INT benchmarks, while at the same time improving the performance of the hybrid cache by 1.16% on average.
---
paper_title: Future cache design using STT MRAMs for improved energy efficiency: devices, circuits and architecture

paper_content:

Spin-transfer torque magnetic RAM (STT MRAM) has emerged as a promising candidate for on-chip memory in future computing platforms. We present a cross-layer (device-circuit-architecture) approach to energy-efficient cache design using STT MRAM. At the device and circuit levels, we consider different genres of MTJs and bitcells, and evaluate their impact on the area, energy and performance of caches. In addition, we propose micro-architectural techniques viz. sequential cache read and partial cache line update, which exploit the non-volatility of STT MRAM to further improve energy efficiency of STT MRAM caches. A detailed comparison of STT MRAM caches with SRAM-based caches is also presented. Our results indicate that the proposed optimizations significantly enhance the efficiency of STT MRAM for designing lower level caches.
---
paper_title: STT-RAM based energy-efficiency hybrid cache for CMPs

paper_content:

Modern high performance Chip Multiprocessor (CMP) systems rely on large on-chip cache hierarchy. As technology scales down, the leakage power of present SRAM based cache gradually dominates the on-chip power consumption, which can severely jeopardize system performance. The emerging nonvolatile Spin Transfer Torque RAM (STT-RAM) is a promising candidate for large on-chip cache because of the ultra low leakage power. However, the write operations on STT-RAM suffer from considerably higher energy as well as longer latency compared with SRAM which will make STT-RAM in trouble for write-intensive workloads. In this paper, we propose to integrate SRAM with STT-RAM to construct a novel hybrid cache architecture for CMPs. We also propose dedicated microarchitectural mechanisms to make the hybrid cache robust to workloads with different write patterns. Extensive simulation results demonstrate that the proposed hybrid scheme is adaptive to variations of workloads. Overall power consumption is reduced by 37.1% and performance is improved by 23.6% on average compared with SRAM based static NUCA under the same area configuration.
---
paper_title: Future memory and interconnect technologies

paper_content:

The improvement of the computer system performance is constrained by the well-known memory wall and power wall. It has been recognized that the memory architecture and the interconnect architecture are becoming the overwhelming bottleneck in computer performance. Disruptive technologies, such as emerging non-volatile memory (NVM) technologies, 3D integration, and optical interconnects, are envisioned as promising future memory and interconnect technologies that can fundamentally change the landscape of the future computer architecture design with profound impact. This invited survey paper gives a brief introduction of these future memory and interconnect technologies, discusses the opportunities and challenges of these new technologies for future computer system designs.
---
paper_title: Cross-Layer Techniques for Optimizing Systems Utilizing Memories with Asymmetric Access Characteristics

paper_content:

Non-volatile memory technologies promise a variety of advantages for memory architectures of next generation computing systems. However, these capabilities come at the cost of some inefficiencies governing the operation of these memories. The most well understood is the asymmetry of access. In order to most effectively take advantage of the benefits of these memory technologies in terms of density and reduced static power in systems while mitigating access complexity an one-size fits all method is not sufficient for all types of applications. Instead, cross-layer techniques that include the compiler, operating system, and hardware layer can extract characteristics from the application that can be used to deliver the highest possible performance while minimizing power consumption for systems using these memories.
---
paper_title: Exploiting reuse information to reduce refresh energy in on-chip eDRAM caches

paper_content:

This work introduces a novel refresh mechanism that leverages reuse information to decide which blocks should be refreshed in an energy-aware eDRAM last-level cache. Experimental results show that, compared to a conventional eDRAM cache, the energy-aware approach achieves refresh energy savings up to 71%, while the reduction on the overall dynamic energy is by 65% with negligible performance losses.
---
paper_title: Cache revive: architecting volatile STT-RAM caches for enhanced performance in CMPs

paper_content:

High density, low leakage and non-volatility are the attractive features of Spin-Transfer-Torque-RAM (STT-RAM), which has made it a strong competitor against SRAM as a universal memory replacement in multi-core systems. However, STT-RAM suffers from high write latency and energy which has impeded its widespread adoption. To this end, we look at trading-off STT-RAM's non-volatility property (data-retention-time) to overcome these problems. We formulate the relationship between retention-time and write-latency, and find optimal retention-time for architecting an efficient cache hierarchy using STT-RAM. Our results show that, compared to SRAM-based design, our proposal can improve performance and energy consumption by 18% and 60%, respectively.
---
paper_title: A novel architecture of the 3D stacked MRAM L2 cache for CMPs

paper_content:

Magnetic random access memory (MRAM) is a promising memory technology, which has fast read access, high density, and non-volatility. Using 3D heterogeneous integrations, it becomes feasible and cost-efficient to stack MRAM atop conventional chip multiprocessors (CMPs). However, one disadvantage of MRAM is its long write latency and its high write energy. In this paper, we first stackMRAM-based L2 caches directly atop CMPs and compare it against SRAM counterparts in terms of performance and energy. We observe that the direct MRAM stacking might harm the chip performance due to the aforementioned long write latency and high write energy. To solve this problem, we then propose two architectural techniques: read-preemptive write buffer and SRAM-MRAM hybrid L2 cache. The simulation result shows that our optimized MRAM L2 cache improves performance by 4.91% and reduces power by 73.5%compared to the conventional SRAM L2 cache with the similar area.
---
paper_title: A circuit-architecture co-optimization framework for evaluating emerging memory hierarchies

paper_content:

Energy-efficient and cost-effective memory hierarchies are needed in the next era of computing. Currently, many emerging non-volatile memory technologies such as PCRAM, STTRAM, and ReRAM, can potentially meet this requirement. It is necessary to have a framework that can quickly find the optimal memory technology choice and the corresponding circuit design style in terms of performance, energy, or cost. In this work, we first build a circuit-architecture joint design space exploration framework by combining RC circuit analysis and ANN-based performance modeling. Then, we use this framework to design and optimize different memory hierarchy levels by adopting new memory technologies such as ReRAM.
---
paper_title: WriteSmoothing: improving lifetime of non-volatile caches using intra-set wear-leveling

paper_content:

Driven by the trends of increasing core-count and bandwidth-wall problem, the size of last level caches (LLCs) has greatly increased. Since SRAM consumes high leakage power, researchers have explored use of non-volatile memories (NVMs) for designing caches as they provide high density and consume low leakage power. However, since NVMs have low write-endurance and the existing cache management policies are write variation-unaware, effective wear-leveling techniques are required for achieving reasonable cache lifetimes using NVMs. We present WriteSmoothing, a technique for mitigating intra-set write variation in NVM caches. WriteSmoothing logically divides the cache-sets into multiple modules. For each module, WriteSmoothing collectively records number of writes in each way for any of the sets. It then periodically makes most frequently written ways in a module unavailable to shift the write-pressure to other ways in the sets of the module. Extensive simulation results have shown that on average, for single and dual-core system configurations, WriteSmoothing improves cache lifetime by 2.17X and 2.75X, respectively. Also, its implementation overhead is small and it works well for a wide range of algorithm and system parameters.
---
paper_title: Embedded memory hierarchy exploration based on magnetic RAM

paper_content:

SRAM, DRAM and FLASH are the three main employed technologies in design of on-chip processor memories. However, manufacturing constraints for this technologies in the most advanced nodes compromises further evolution. MRAM (Magnetic memory) presents itself as an attractive alternative for these technologies, as it has reasonable timing and power characteristics. Last results in the state of the art demonstrate that MRAM access time is can be less than 5ns and read/write energy per bit in same order of magnitude as SRAM, also it can evolve with the manufacturing process. One important feature of MRAM is the non-volatility, allowing to define new instant on/off policies and mainly optimizing leakage current. In this paper we demonstrate how MRAM can be used into memory hierarchy of embedded systems. The main objective is to demonstrate the interest to use MRAM for Level-1 & 2 cache and to better understand the architectural choice in order to minimize the impact of the higher write latency of MRAMs.
---
paper_title: Combining RAM technologies for hard-error recovery in L1 data caches working at very-low power modes

paper_content:

Low-power modes in modern microprocessors rely on low frequencies and low voltages to reduce the energy budget. Nevertheless, manufacturing induced parameter variations can make SRAM cells unreliable producing hard errors at supply voltages below Vccmin. Recent proposals provide a rather low fault-coverage due to the fault coverage/overhead trade-off. We propose a new fault-tolerant L1 cache, which combines SRAM and eDRAM cells in L1 data caches to provide 100% SRAM hard-error fault coverage. Results show that, compared to a conventional cache and assuming 50% failure probability at low-power mode, leakage and dynamic energy savings are by 85% and 62%, respectively, with a minimal impact on performance.
---
paper_title: i2WAP: Improving non-volatile cache lifetime by reducing inter- and intra-set write variations

paper_content:

Modern computers require large on-chip caches, but the scalability of traditional SRAM and eDRAM caches is constrained by leakage and cell density. Emerging non-volatile memory (NVM) is a promising alternative to build large on-chip caches. However, limited write endurance is a common problem for non-volatile memory technologies. In addition, today's cache management might result in unbalanced write traffic to cache blocks causing heavily-written cache blocks to fail much earlier than others. Unfortunately, existing wear-leveling techniques for NVM-based main memories cannot be simply applied to NVM-based on-chip caches because cache writes have intra-set variations as well as inter-set variations. To solve this problem, we propose i2WAP, a new cache management policy that can reduce both inter- and intra-set write variations. i2WAP has two features: (1) Swap-Shift, an enhancement based on previous main memory wear-leveling to reduce cache inter-set write variations; (2) Probabilistic Set Line Flush, a novel technique to reduce cache intra-set write variations. Implementing i2WAP only needs two global counters and two global registers. By adopting i2WAP, we can improve the lifetime of on-chip non-volatile caches by 75% on average and up to 224%.
---
paper_title: Constructing large and fast multi-level cell STT-MRAM based cache for embedded processors

paper_content:

MLC STT-MRAM (Multi-level Cell Spin-Transfer Torque Magnetic RAM), an emerging non-volatile memory technology, has become a promising candidate to construct L2 caches for high-end embedded processors. However, the long write latency limits the effectiveness of MLC STT-MRAM based L2 caches. In this paper, we address this limitation with two novel designs: Line Pairing (LP) and Line Swapping (LS). LP forms fast cachelines by re-organizing MLC soft bits which are faster to write. LS dynamically stores frequently written data into these fast cachelines. Our experimental results show that LP and LS improve system performance by 15% and reduce energy consumption by 21%.
---
paper_title: Unleashing the potential of MLC STT-RAM caches

paper_content:

In this paper, we study the use of multi-level cell (MLC) spin-transfer torque RAM (STT-RAM) in cache design of embedded systems and microprocessors. Compared to the single level cell (SLC) design, a MLC STT-RAM cache is expected to offer higher density and faster system performance. However, the cell design constrains, such as the switching current requirement and asymmetry in write operations, severely limit the density benefit of the conventional MLC STT-RAM. The two-step read/write accesses and inflexible data mapping strategy in the existing MLC STT-RAM cache architecture may even result in system performance degradation. To unleash the real potential of MLC STT-RAM cache, we propose a cross-layer solution. First, we introduce the reverse magnetic junction tunneling (MTJ) into MLC cell design, which offers a more balanced device and design tradeoff and enables 2x storage density than SLC. At architectural level, we propose a cell split mapping method to divide cache lines into fast and slow regions and data migration policies to allocate the frequently-used data to fast regions. Furthermore, an application-aware speed enhancement mode is utilized to adaptively trade-off cache capacity and speed, satisfying different requirements of various applications. Simulation results show that the proposed techniques can improve the system performance by 10.3% and reduce the energy consumption on cache by 26.0% compared with conventional MLC STT-RAM.
---
paper_title: Performance, Power, and Reliability Tradeoffs of STT-RAM Cell Subject to Architecture-Level Requirement

paper_content:

Large switching current and long switching time have significantly limited the adoption of spin-transfer torque random access memory (STT-RAM). Technology scaling, moreover, makes it very challenging to reduce the switching current while maintaining the reliability of magnetic tunneling junction (MTJ) to be similar to that of the earlier generations. In this work, we shall exploit a key insight that in the most on-chip caches where STT-RAM is most likely to be deployed, the lifespan of the data stored in the memory cells is much shorter than the data retention time requirement assumed in STT-RAM development, namely, 4~10 years. We also quantitatively investigated the possibility of trading off MTJ nonvolatility for improved switching performance, e.g., the switching time and/or current, under architectural level guidance. We further proposed and evaluated a hybrid memory design technique that partitions the on-chip STT-RAM cache into two parts with different nonvolatility performances so as to better fit the diverse retention time requirements of different data sets.
---
paper_title: Cache coherence enabled adaptive refresh for volatile STT-RAM

paper_content:

Spin-Transfer Torque RAM (STT-RAM) is extensively studied in recent years. Recent work proposed to improve the write performance of STT-RAM through relaxing the retention time of STT-RAM cell, magnetic tunnel junction (MTJ). Unfortunately, frequent refresh operations of volatile STT-RAM could dissipate significantly extra energy. In addition, refresh operations can severely conflict with normal read/write operations and results in degraded cache performance. This paper proposes Cache Coherence Enabled Adaptive Refresh (CCear) to minimize refresh operations for volatile STT-RAM. Through novel modifications to cache coherence protocol, CCear can effectively minimize the number of refresh operations on volatile STT-RAM. Full-system simulation results show that CCear approaches the performance of the ideal refresh policy with negligible overhead.
---
paper_title: Asynchronous Asymmetrical Write Termination (AAWT) for a low power STT-MRAM

paper_content:

Spin Transfer Torque (STT) memory is an emerging and promising non-volatile storage technology. However, the high write current is still a major challenge which leads to a huge power consumption of the memory. Due to an inherent torque asymmetry of the Magnetic Tunnel Junction (MTJ) device employed in STT memories, the switching time between parallel to anti-parallel and anti-parallel to parallel magnetization is significantly different. Hence, the write latencies for writing `0' and `1' are also considerably different. In this paper, we propose a technique called Asynchronous Asymmetrical Write Termination (AAWT) which utilizes this asymmetrical behavior to terminate the write operations asynchronously and as a result significantly reduces the write power consumption. Furthermore, we present two different AAWT implementations to determine the actual write termination times. The first one makes use of a clock signal and the second one employs a self-timing approach based on an internal delay element. As shown by our experimental results, AAWT can reduce the total write energy by 30% in average with a negligible area overhead.
---
paper_title: Avoiding unnecessary write operations in STT-MRAM for low power implementation

paper_content:

Spin Transfer Torque (STT) is a promising emerging memory technology because of its various advantages such as non-volatility, high density, virtually infinite endurance, scalability and CMOS compatibility. Despite all these features, high write current is still a challenge for its widespread use. When writing a value that is already stored, a significant current flows through the Magnetic Tunnel Junction (MTJ) cell which is almost the same as that required to flip the stored data. This increases the total power consumption of the memory. To address this issue, we propose a technique which can avoid unnecessary write operations with bit-level granularity. Our technique can save 68.9% of the total write power consumption with a minor area overhead (0.68 %) and only a small timing penalty (1.33 %).
---
paper_title: Dynamically reconfigurable hybrid cache: An energy-efficient last-level cache design

paper_content:

The recent development of non-volatile memory (NVM), such as spin-torque transfer magnetoresistive RAM (STT-RAM) and phase-change RAM (PRAM), with the advantage of low leakage and high density, provides an energy-efficient alternative to traditional SRAM in cache systems. We propose a novel reconfigurable hybrid cache architecture (RHC), in which NVM is incorporated in the last-level cache together with SRAM. RHC can be reconfigured by powering on/off SRAM/NVM arrays in a way-based manner. In this work, we discuss both the architecture and circuit design issues for RHC. Furthermore, we provide hardware-based mechanisms to dynamically reconfigure RHC on-the-fly based on the cache demand. Experimental results on a wide range of benchmarks show that the proposed RHC achieves an average 63%, 48% and 25% energy saving over non-reconfigurable SRAM-based cache, non-reconfigurable hybrid cache, and reconfigurable SRAM-based cache, while maintaining the system performance (at most 4% performance overhead).
---
paper_title: C1C: A configurable, compiler-guided STT-RAM L1 cache

paper_content:

Spin-Transfer Torque RAM (STT-RAM), a promising alternative to SRAM for reducing leakage power consumption, has been widely studied to mitigate the impact of its asymmetrically long write latency. Recently, STT-RAM has been proposed for L1 caches by relaxing the data retention time to improve write performance and dynamic energy. However, as the technology scales down from 65nm to 22nm, the performance of the read operation scales poorly due to reduced sense margins and sense amplifier delays. In this article, we leverage a dual-mode STT memory cell to design a configurable L1 cache architecture termed C1C to mitigate read performance barriers with technology scaling. Guided by application access characteristics discovered through novel compiler analyses, the proposed cache adaptively switches between a high performance and a low-power access mode. Our evaluation demonstrates that the proposed cache with compiler guidance outperforms a state-of-the-art STT-RAM cache design by 9p with high dynamic energy efficiency, leading to significant performance/watt improvements over several competing approaches.
---
paper_title: A software approach for combating asymmetries of non-volatile memories

paper_content:

The recent advances in non-volatile memory technologies promise the delivery of future high performance and low power computing systems. While these technologies provide attractive features, they exhibit different degrees of asymmetric read/write behavior, resulting in under utilization of the technology benefits in many scenarios. Future systems should be aware of this asymmetry to fully utilize these memory technologies. In this paper, we propose software dispatch, a cross-layer approach to distribute data to appropriate memory resources based on an application's data access characteristics. We demonstrate the application of the proposed technique through a case study system with hybrid memory caches, on which we achieve over 5% performance improvement and 9.8% power savings compared to the state-of-the-art technique.
---
paper_title: Lower-bits cache for low power STT-RAM caches

paper_content:

As power-efficient design becomes more important, spin-transfer torque RAM (STT-RAM) has drawn a lot of attention due to its ability to meet both high performance and low power consumption. However, its high write energy incurs an increase of dynamic power consumption and may offset power saving due to its low static power. This paper proposes a novel technique called lower-bits caches for reducing write activities of STT-RAM L2 caches. Based on the observation that upper bits of data are not changed as frequently as lower bits in most applications, the technique tries to hide frequent bit changes in lower bits from the L2 cache. Experimental results show that our architecture reduced 25 percent of energy consumed by the L2 cache and slightly improved performance at the same time compared to the STT-RAM baseline.
---
paper_title: OAP: an obstruction-aware cache management policy for STT-RAM last-level caches

paper_content:

Emerging memory technologies are explored as potential alternatives to traditional SRAM/DRAM-based memory architecture in future microprocessor designs. Among various emerging memory technologies, Spin-Torque Transfer RAM (STT-RAM) has the benefits of fast read latency, low leakage power, and high density, and therefore has been investigated as a promising candidate for last-level cache (LLC)1. One of the major disadvantages for STT-RAM is the latency and energy overhead associated with the write operations. In particular, a long-latency write operation to STT-RAM cache may obstruct other cache accesses and result in severe performance degradation. Consequently, mitigation techniques to minimize the write overhead are required in order to successfully adopt this new technology for cache design. In this paper, we propose an obstruction-aware cache management policy called OAP. OAP monitors the cache to periodically detect LLC-obstruction processes, and manage the cache accesses from different processes. The experimental results on a 4-core architecture with an 8MB STT-RAM L3 cache shows that the performance can be improved by 14% on average and up to 42%, with a reduction of energy consumption by 64%2.
---
paper_title: Wear-Resistant Hybrid Cache Architecture with Phase Change Memory

paper_content:

Phase-change Random Access Memory (PRAM) is one of the most promising technologies among emerging non-volatile memory technologies, which provides many benefits, such as high density, non-volatility and low leakage power. However, the limited write endurance of PRAM prevents it from being used as a drop-in replacement of SRAM cache. Moreover, the inherent high latency and power dissipation of write operations are both hindrances that PRAM faces. In this paper, we study the L2 cache write operations incurred by different types of data, and accordingly, propose Wear-Resistant Hybrid Cache Architecture (WRHCA), in which the write access behavior of the hybrid L2 cache, that is composed of SRAM and PRAM, is optimized. Through the prediction of data access patterns, the proposed WRHCA prevents write-prone data from entering PRAM L2 cache, and consequently, the wear-out issue of PRAM is alleviated efficiently. Experimental results on the basis of the trace-driven simulator demonstrate that, as compared to the baseline system with pure PRAM L2 cache, our optimized WRHCA saved 85.5% write operations to PRAM on average, and boosted the performance by the averaged 6.4% CPI reduction. Last but not least, as compared with the primitive 3-level SRAM cache with the same chip area, our WRHCA achieved 60.9% reduction in terms of power consumption.
---
paper_title: AWARE (Asymmetric Write Architecture With REdundant Blocks): A High Write Speed STT-MRAM Cache Architecture

paper_content:

Spin-transfer torque magnetic RAM (STT-MRAM) is a promising memory technology for lower level caches because of its high density and nonvolatile nature. However, the high write latency is a bottleneck to its widespread adoption as the future on-chip memory. In this paper, we propose a new cache architecture-asymmetric write architecture with redundant blocks (AWARE)-that can improve the write latency by taking advantage of the asymmetric write characteristics of 1T-1MTJ STT-MRAM bit-cells. Due to the nature of the storage element in STT-MRAM, the time required for the two-state transitions ( 1→ 0 and 0→ 1) is not identical. In other words, one of the state transitions is slower than the other direction. In conventional cache architecture, the overall write latency is limited by the slower transition. However, the AWARE cache design introduces redundant blocks in each row, and they are preset to the initial state that enables the faster transition. Hence the write operations performed in these redundant blocks are much faster than the conventional write scheme. The write latency in AWARE is improved by 30% over conventional cache architecture with no area penalty in the data array. Moreover, the additional tag bits introduced in this technique result in penalty on the total cache area. In addition, the write energy increases modestly by 7% in the proposed cache design. However, this write-energy increase can be mitigated by sacrificing the cache capacity.
---
paper_title: An energy efficient cache design using spin torque transfer (STT) RAM

paper_content:

The on-chip memory is a dominant source of power and energy consumption in modern and future processors. This paper explores the use of a new emerging non-volatile memory technology as a replacement for SRAM based lower level caches - Spin Torque Transfer(STT) RAM. While STTRAM achieves a reduction in leakage energy of 90% compared to SRAM, the dynamic energy for a write operation is 2X that of SRAM. Consequently, we propose additional microarchitectural optimizations to reduce overall dynamic energy which achieve an average reduction in dynamic energy over the base case of 30% with a range of 16% to 60% across 10 benchmarks.
---
paper_title: Compiler-assisted refresh minimization for volatile STT-RAM cache

paper_content:

Spin-Transfer Torque RAM (STT-RAM) has been proposed to build on-chip caches because of its attractive features: high storage density and negligible leakage power. Recently, researchers propose to improve the write performance of STT-RAM by relaxing its non-volatility property. To avoid data loss resulting from volatility, refresh schemes are proposed. However, refresh operations consume additional energy. In this paper, we propose to reduce the number of refresh operations through re-arranging program data layout at compilation time. An N-refresh scheme is also proposed. Experimental results show that, on average, the proposedmethods can reduce the number of refresh operations by 73.3%, and reduce the dynamic energy consumption by 27.6%.
---
paper_title: Energy reduction for STT-RAM using early write termination

paper_content:

The emerging Spin Torque Transfer memory (STT-RAM) is a promising candidate for future on-chip caches due to STT-RAM's high density, low leakage, long endurance and high access speed. However, one of the major challenges of STT-RAM is its high write current, which is disadvantageous when used as an on-chip cache since the dynamic power generated is too high. In this paper, we propose Early Write Termination (EWT), a novel technique to significantly reduce write energy with no performance penalty. EWT can be implemented with low complexity and low energy overhead. Our evaluation shows that up to 80% of write energy reduction can be achieved through EWT, resulting 33% less total energy consumption, and 34% reduction in ED2. These results indicate that EWT is an effective and practical scheme to improve the energy efficiency of a STT-RAM cache.
---
paper_title: Implementing a hybrid SRAM / eDRAM NUCA architecture

paper_content:

Advances in technology allowed for integrating DRAM-like structures into the chip, called embedded DRAM (eDRAM). This technology has already been successfully implemented in some GPUs and other graphic-intensive SoC, like game consoles. The most recent processor from IBM®, POWER7, is the first general-purpose processor that integrates an eDRAM module on the chip. In this paper, we propose a hybrid cache architecture that exploits the main features of both memory technologies, speed of SRAM and high density of eDRAM. We demonstrate, that due to the high locality found in emerging applications, a high percentage of data that enters to the on-chip last-level cache are not accessed again before they are evicted. Based on that observation, we propose a placement scheme where re-accessed data blocks are stored in fast, but costly in terms of area and power, SRAM banks, while eDRAM banks store data blo cks that just arrive to the NUCA cache or were demoted from a SRAM bank. We show that a well-balanced SRAM / eDRAM NUCA cache can achieve similar performance results than using a NUCA cache composed of only SRAM banks, but reduces area by 15% and power consumed by 10%. Furthermore, we also explore several alternatives to exploit the area reduction we gain by using the hybrid architecture, resulting in an overall performance improvement of 4%.
---
paper_title: Resistive computation: avoiding the power wall with low-leakage, STT-MRAM based computing

paper_content:

As CMOS scales beyond the 45nm technology node, leakage concerns are starting to limit microprocessor performance growth. To keep dynamic power constant across process generations, traditional MOSFET scaling theory prescribes reducing supply and threshold voltages in proportion to device dimensions, a practice that induces an exponential increase in subthreshold leakage. As a result, leakage power has become comparable to dynamic power in current-generation processes, and will soon exceed it in magnitude if voltages are scaled down any further. Beyond this inflection point, multicore processors will not be able to afford keeping more than a small fraction of all cores active at any given moment. Multicore scaling will soon hit a power wall. This paper presents resistive computation, a new technique that aims at avoiding the power wall by migrating most of the functionality of a modern microprocessor from CMOS to spin-torque transfer magnetoresistive RAM (STT-MRAM)---a CMOS-compatible, leakage-resistant, non-volatile resistive memory technology. By implementing much of the on-chip storage and combinational logic using leakage-resistant, scalable RAM blocks and lookup tables, and by carefully re-architecting the pipeline, an STT-MRAM based implementation of an eight-core Sun Niagara-like CMT processor reduces chip-wide power dissipation by 1.7× and leakage power by 2.1× at the 32nm technology node, while maintaining 93% of the system throughput of a CMOS-based design.
---
paper_title: FlexiWay: A cache energy saving technique using fine-grained cache reconfiguration

paper_content:

Recent trends of CMOS scaling and use of large last level caches (LLCs) have led to significant increase in the leakage energy consumption of LLCs and hence, managing their energy consumption has become extremely important in modern processor design. The conventional cache energy saving techniques require offline profiling or provide only coarse granularity of cache allocation. We present FlexiWay, a cache energy saving technique which uses dynamic cache reconfiguration. FlexiWay logically divides the cache sets into multiple (e.g. 16) modules and dynamically turns off suitable and possibly different number of cache ways in each module. FlexiWay has very small implementation overhead and it provides fine-grain cache allocation even with caches of typical associativity, e.g. an 8-way cache. Microarchitectural simulations have been performed using an x86-64 simulator and workloads from SPEC2006 suite. Also, FlexiWay has been compared with two conventional energy saving techniques. The results show that FlexiWay provides largest energy saving and incurs only small loss in performance. For single, dual and quad core systems, the average energy saving using FlexiWay are 26.2%, 25.7% and 22.4%, respectively.
---
paper_title: High-endurance hybrid cache design in CMP architecture with cache partitioning and access-aware policy

paper_content:

In recent years, NVM (non-volatile memory) technologies, such as STT-RAM (spin transfer torque RAM) and PRAM (phase change RAM), have drawn a lot of attention due to their low leakage and high density. However, both NVMs suffer from high write latency and limited endurance problems. To overcome these problems, the SRAM/NVM hybrid cache architecture has been proposed, and the write pressure on NVM can be mitigated with appropriate write management policy. Moreover, many wear leveling techniques have been proposed to extend the lifetime of NVM in the hybrid cache. In this paper, we proposed a hybrid cache design that includes SRAM cache, STT-RAM cache, and STT-RAM/SRAM hybrid cache banks for CMP (chip multi-processors) architecture. We also propose a partition-level wear leveling scheme and access-aware policies to mitigate unbalanced wear-out of STT-RAM lines within a partition and among different cache partitions. Experimental results show that, our proposed scheme and policies can achieve an average of 89 times improvement in cache lifetime and are able to save 58% power consumption compared to SRAM cache.
---
paper_title: MAC: migration-aware compilation for STT-RAM based hybrid cache in embedded systems

paper_content:

Hybrid caches consisting of both STT-RAM and SRAM have been proposed recently for energy efficiency. To explore the advantages of hybrid cache, most work on hybrid caches employs migration based strategies to dynamically move write-intensive data from STT-RAM to SRAM. Migrations require additional read and write operations for data movement and may lead to significant overheads. To address this issue, this paper proposes a Migration-Aware Compilation (MAC) approach to improve the energy efficiency and performance of STT-RAM based hybrid cache. By re-arranging data layout, the data access pattern in memory blocks is changed such that the number of migrations is reduced without any hardware modification. The reduction of migration overheads in turn improves energy efficiency and performance. The experimental results show that with the proposed approach, on average, the number of write operations on STT-RAM is reduced by 13.4%, the number of migrations is reduced by 16.1%, the total dynamic energy is reduced by 8.5%, and the total latency is reduced by 12.1%.
---
paper_title: Improving energy efficiency of embedded DRAM caches for high-end computing systems

paper_content:

The number of cores in a single chip in the nodes of high-end computing systems is on rise, due, in part, to a number of constraints, such as power consumption. With this, the size of the last level cache (LLC) has also increased significantly. Since LLCs built with SRAM consume high leakage power, power consumption of LLCs is becoming a significant fraction of processor power consumption. To address this issue, researchers have used embedded DRAM (eDRAM) LLCs which consume low leakage power. However, eDRAM caches consume a significant amount of energy in the form of refresh energy. In this paper, we propose ESTEEM, an energy saving technique for embedded DRAM caches. ESTEEM uses dynamic cache reconfiguration to turn off a portion of the cache to save both leakage and refresh energy. It logically divides the cache sets into multiple modules and turns off possibly different number of ways in each module. Microarchitectural simulations confirm that ESTEEM is effective in improving performance and energy efficiency and provides better results compared to a recently-proposed eDRAM cache energy saving technique, namely Refrint. For single and dual-core simulations, the average energy saving in memory subsystem (LLC+main memory) with ESTEEM is 25.8% and 32.6% respectively, and the average weighted speedup is 1.09x and 1.22x respectively. Additional experiments confirm that ESTEEM works well for a wide-range of system and algorithm parameters.
---
paper_title: Design space exploration of workload-specific last-level caches

paper_content:

Leakage power of last-level caches constitute a significant part of overall power consumption. Various circuit-level and technology-based methods have been proposed to reduce cache leakage. However, from a system designer's perspective, for a particular configuration and workload, it is not clear which method will be most effective. In this work, we make a detailed evaluation and comparison of cache energy reduction techniques. Our results show that when energy is very scarce and important, the best results are obtained with highly energy efficient Tunnel-FET caches. When the available energy increases and performance becomes a bigger concern, there is no single winner. While a small number of capacity sensitive workloads benefit from increased capacity of STT-RAM caches, latency sensitive workloads prefer solutions with smaller latency penalties such as drowsy caches.
---
paper_title: DWM-TAPESTRI - an energy efficient all-spin cache using domain wall shift based writes

paper_content:

Spin-based memories are promising candidates for future on-chip memories due to their high density, non-volatility, and very low leakage. However, the high energy and latency of write operations in these memories is a major challenge. In this work, we explore a new approach -- shift based write -- that offers a fast and energy-efficient alternative to performing writes in spin-based memories. We propose DWM-TAPESTRI, a new all-spin cache design that utilizes Domain Wall Memory (DWM) with shift based writes at all levels of the cache hierarchy. The proposed write scheme enables DWM to be used, for the first time, in L1 caches and in tag arrays, where the inefficiency of writes in spin memories has traditionally precluded their use. At the circuit level, we propose bit-cell designs utilizing shift-based writes, which are tailored to the differing requirements of different levels in the cache hierarchy. We also propose pre-shifting as an architectural technique to hide the latency of shift operations that is inherent to DWM. We performed a systematic device-circuit-architecture evaluation of the proposed design. Over a wide range of SPEC 2006 benchmarks, DWM-TAPESTRI achieves 8.2X improvement in energy and 4X improvement in area, with virtually identical performance, compared to an iso-capacity SRAM cache. Compared to an iso-capacity STT-MRAM cache, the proposed design achieves around 1.6X improvement in both area and energy under iso-performance conditions.
---
paper_title: Compiler-assisted preferred caching for embedded systems with STT-RAM based hybrid cache

paper_content:

As technology scales down, energy consumption is becoming a big problem for traditional SRAM-based cache hierarchies. The emerging Spin-Torque Transfer RAM (STT-RAM) is a promising replacement for large on-chip cache due to its ultra low leakage power and high storage density. However, write operations on STT-RAM suffer from considerably higher energy consumption and longer latency than SRAM. Hybrid cache consisting of both SRAM and STT-RAM has been proposed recently for both performance and energy efficiency. Most management strategies for hybrid caches employ migration-based techniques to dynamically move write-intensive data from STT-RAM to SRAM. These techniques lead to extra overheads. In this paper, we propose a compiler-assisted approach, preferred caching, to significantly reduce the migration overhead by giving migration-intensive memory blocks the preference for the SRAM part of the hybrid cache. Furthermore, a data assignment technique is proposed to improve the efficiency of preferred caching. The reduction of migration overhead can in turn improve the performance and energy efficiency of STT-RAM based hybrid cache. The experimental results show that, with the proposed techniques, on average, the number of migrations is reduced by 21.3%, the total latency is reduced by 8.0% and the total dynamic energy is reduced by 10.8%.
---
paper_title: Using Magnetic RAM to Build Low-Power and Soft Error-Resilient L1 Cache

paper_content:

Due to its great scalability, fast read access, low leakage power, and nonvolatility, magnetic random access memory (MRAM) appears to be a promising memory technology for on-chip cache memory in microprocessors. However, the write-to-MRAM process is relatively slow and results in high dynamic power consumption. Such inherent disadvantages of MRAM make researchers easily conclude that MRAM can only be used for low-level caches (e.g., L2 or L3 cache), where cache memories are less frequently accessed and slow write to MRAM can be more easily compensated using simple architectural techniques. By developing a hybrid cache architecture, this paper attempts to show that, with appropriate architecture design, MRAM can also be used in L1 cache to improve both the energy efficiency and soft error immunity. The basic idea is to supplement the MRAM L1 cache with several small SRAM buffers, which can substantially mitigate the performance degradation and dynamic energy overhead induced by MRAM write operations. Moreover, the proposed hybrid cache architecture is also an efficient solution to protect cache memory from radiation-induced soft errors, as MRAM is inherently invulnerable to emissive particles. Simulation results show that, with only less than 2% performance degradation, the proposed design approach can reduce the power consumption by up to 76.1% on average compared with the traditional SRAM L1 cache. In addition, the architectural vulnerability factor of L1 data cache is reduced from 28.3% to as low as 0.5%.
---
paper_title: Automatic Feedback Control of Shared Hybrid Caches in 3D Chip Multiprocessors

paper_content:

3D integration enables building caches from different types of technologies such as SRAM, Magnetic RAM (MRAM), DRAM, and Phase-change RAM (PRAM). Hybrid cache architectures (HCAs) have been proposed to take advantage of the benefits offered by these types of technologies. Employing this novel cache architecture to build shared caches in chip multiprocessors (CMPs) can lead to significant performance and power consumption improvements. In this paper, we focus on a 3D CMP design in which the shared last level L2 cache is composed of an MRAM layer and an SRAM layer stacked upon the processing cores and present a control theory centric approach designed to partition this shared hybrid L2 cache space dynamically among concurrently running applications in order to satisfy the application-level performance QoS targets. At each time interval, the two layers of the hybrid L2 cache are partitioned, based on the cache demands made by the controllers of the applications, to satisfy the specified performance targets. We evaluate our feedback control based scheme using various workloads. Our experimental evaluation shows that the proposed scheme is able to satisfy the specified performance QoS in most of the tested cases, by partitioning the hybrid cache space of the 3D CMP among co-runner applications.
---
paper_title: Versatile refresh: low complexity refresh scheduling for high-throughput multi-banked eDRAM

paper_content:

Multi-banked embedded DRAM (eDRAM) has become increasingly popular in high-performance systems. However, the data retention problem of eDRAM is exacerbated by the larger number of banks and the high-performance environment in which it is deployed: The data retention time of each memory cell decreases while the number of cells to be refreshed increases. For this, multi-bank designs offer a concurrent refresh mode, where idle banks can be refreshed concurrently during read and write operations. However, conventional techniques such as periodically scheduling refreshes---with priority given to refreshes in case of conflicts with reads or writes---have variable performance, increase read latency, and can perform poorly in worst case memory access patterns. We propose a novel refresh scheduling algorithm that is low-complexity, produces near-optimal throughput with universal guarantees, and is tolerant to bursty memory access patterns. The central idea is to decouple the scheduler into two simple-to-implement modules: one determines which cell to refresh next and the other determines when to force an idle cycle in all banks. We derive necessary and sufficient conditions to guarantee data integrity for all access patterns, with any given number of banks, rows per bank, read/write ports and data retention time. Our analysis shows that there is a tradeoff between refresh overhead and burst tolerance and characterizes this tradeoff precisely. The algorithm is shown to be near-optimal and achieves, for instance, 76.6% reduction in worst-case refresh overhead from the periodic refresh algorithm for a 250MHz eDRAM with 10us retention time and 16 banks each with 128 rows. Simulations with Apex-Map synthetic benchmarks and switch lookup table traffic show that VR can almost completely hide the refresh overhead for memory accesses with moderate-to-high multiplexing across memory banks.
---
paper_title: Multi-level magnetic RAM using domain wall shift for energy-efficient, high-density caches

paper_content:

Spin-based devices promise to revolutionize computing platforms by enabling high-density, low-leakage memories. However, stringent tradeoffs between critical design metrics such as read and write stability, reliability, density, performance and energy-efficiency limit the efficiency of conventional spin-transfer- torque devices and bit-cells. We propose a new multi-level cell design with domain wall magnets (DWM-MLC) that significantly improves upon the read/write performance, density, and write energy consumption of conventional spin memories. The fundamental design tradeoff between read and write operations are addressed in DWM-MLC by decoupling the read and write paths, thereby allowing separate optimization for reads and writes. A thicker tunneling oxide is used for higher readability, while a domain-wall-shift (DWS) based write mechanism is used to improve write speed and energy. The storage of multiple bits per cell and the ability to use smaller transistors lead to a net improvement in density compared to conventional spin memories. We perform a systematic evaluation of DWM-MLC at different levels of design abstraction. At the circuit level, DWM-MLC achieves 2X improvement in density, read energy and read latency over its 1-bit counterpart. We evaluate an "all-spin" cache hierarchy that uses DWM-MLC for both L1 and L2, resulting in 4.4X (1.7X) area improvement and 10X (2X) energy reduction at isoperformance over SRAM (STT-MRAM).
---
paper_title: Memories: Exploiting Them and Developing Them

paper_content:

The disciplines of process development, circuit design, and microarchitecture too often remain isolated. This paper on memories explores the benefit of their unification by taking a retrospective look at a L1 cache memory design, a current view of embedded DRAM, and a speculative peek at emerging memories (e.g. MTJ MRAM). In the midst, a novel refresh operation is proposed for DRAM that relies on read and write activity ongoing within a cache to refresh the DRAM.
---
paper_title: Analysis and Optimization of Thermal Effect on STT-RAM Based 3-D Stacked Cache Design

paper_content:

Spin-Transfer Torque Random Access Memory (STT-RAM) has been proved a promising emerging nonvolatile memory technology suitable for many applications such as cache memory of CPU. Simulation results show that the switching time of Magnetic Tunnel Junction (MTJ), which is the core element of the STT-RAM cell, varies when the temperature changes. In this paper, we study the thermal effect on switching time of STT-RAM cell, and it is showed that when temperature changes from 300K to 375K, the required write pulse period to achieve 1E-8 bit error rate (BER) increases from 10.02ns to 15.04ns under 45nm technology. When STT-RAM is used as 3-D stacked L3 cache, the required write pulse period ranges from 11.42ns to 14.68ns due to temperature variation caused by the CPU core layer. If the thermal effect is not considered, the BER of the hottest region will significantly increase to 1E-4. Based on these observations, an optimization design with Dynamic Temperature Aware Write Access is proposed, to increase the efficiency of accessing a 3-D stacked STT-RAM cache, as well as achieve the target BER. Compared to a conventional design, the proposed scheme can improve the CPU performance by 3.8% and reduce the write energy consumption of the STT-RAM cache by 4.8%.
---
paper_title: LASIC: Loop-Aware Sleepy Instruction Caches Based on STT-RAM Technology

paper_content:

This brief presents an approach to reduce static power consumption in peripheral circuits of spin-transfer torque RAM (STT-RAM) instruction caches. It is based on the key observation that only a small set of instructions is accessed inside a program loop. We propose to add a small static RAM cache called loop cache between the processor and the L1 instruction cache made of STT-RAM. When the loop cache has an entire loop cached, the L1 instruction cache can be turned off to save energy during the execution of the loop. Experimental results show that the proposed approach achieves 49% reduction in energy consumption over the STT-RAM baseline.
---
paper_title: Exploring the vulnerability of CMPs to soft errors with 3D stacked non-volatile memory

paper_content:

Spin-transfer Torque Random Access Memory (STT-RAM) emerges for on-chip memory in microprocessor architectures. Thanks to the magnetic field based storage STT-RAM cells have immunity to radiation induced soft errors that affect electrical charge based data storage, which is a major challenge in SRAM based caches in current microprocessors. In this study we explore the soft error resilience benefits and design trade offs of 3D-stacked STT-RAM for multi-core architectures. We use 3D stacking as an enabler for modular integration of STT-RAM caches with minimum disruption in the baseline processor design flow, while providing further interconnectivity and capacity advantages. We take an in-depth look at alternative replacement schemes in terms of performance, power, temperature, and reliability trade-offs to capture the multi-variable optimization challenges microprocessor architectures face. We analyze and compare the characteristics of STT-RAM, SRAM, and DRAM alternatives for various levels of the cache hierarchy in terms of reliability.
---
paper_title: Analyzing the optimal ratio of SRAM banks in hybrid caches

paper_content:

Cache memories have been typically implemented with Static Random Access Memory (SRAM) technology. This technology presents a fast access time but high energy consumption and low density. As opposite, the recently appeared embedded Dynamic RAM (eDRAM) technology allows caches to be built with lower energy and area, although with a slower access time. The eDRAM technology provides important leakage and area savings, especially in huge Last-Level Caches (LLCs), which occupy almost half the silicon area in some recent microprocessors. This paper proposes a novel hybrid LLC, which combines SRAM and eDRAM banks to address the trade-off among performance, energy, and area. To this end, we explore the optimal percentage of SRAM and eDRAM banks that achieves the best target trade-off. Architectural mechanisms have been devised to keep the most likely accessed blocks in fast SRAM banks as well as to avoid unnecessary destructive reads. Experimental results show that, compared to a conventional SRAM LLC with the same storage capacity, performance degradation does not surpass, on average, 2.9% (even with 12.5% of banks built with SRAM technology), whereas area savings can be as high as 46% for a 1MB-16way LLC. For a 45nm technology node, the energy-delay squared product confirms that a hybrid cache is a better design than the conventional SRAM cache regardless the number of eDRAM banks, and also better than a conventional eDRAM cache when the number of SRAM banks is a quarter or an eighth of the cache banks.
---
paper_title: Refrint: Intelligent refresh to minimize power in on-chip multiprocessor cache hierarchies

paper_content:

As manycores use dynamic energy ever more efficiently, static power consumption becomes a major concern. In particular, in a large manycore running at a low voltage, leakage in on-chip memory modules contributes substantially to the chip's power draw. This is unfortunate, given that, intuitively, the large multi-level cache hierarchy of a manycore is likely to contain a lot of useless data. An effective way to reduce this problem is to use a low-leakage technology such as embedded DRAM (eDRAM). However, eDRAM requires refresh. In this paper, we examine the opportunity of minimizing on-chip memory power further by intelligently refreshing on-chip eDRAM. We present Refrint, a simple approach to perform fine-grained, intelligent refresh of on-chip eDRAM multiprocessor cache hierarchies. We introduce the Refrint algorithms and microarchitecture. We evaluate Refrint in a simulated manycore running 16-threaded parallel applications. We show that an eDRAM-based memory hierarchy with Refrint consumes only 30% of the energy of a conventional SRAM-based memory hierarchy, and induces a slowdown of only 6%. In contrast, an eDRAM-based memory hierarchy without Refrint consumes 56% of the energy of the conventional memory hierarchy, inducing a slowdown of 25%.
---
paper_title: Cross-layer racetrack memory design for ultra high density and low power consumption

paper_content:

The racetrack memory technology utilizes magnetic domains along a nanoscopic wire to obtain ultra-high data storage density. The recent success in the planar racetrack nanowire promised its fabrication feasibility and future scalability, bringing more design challenges and opportunities. In this paper, we initialize the optimization of racetrack memory embracing design considerations across multiple layers, including cell design, array structure, architecture organization, and data management. Our evaluation shows that racetrack memory based cache can achieve 6.4× area reduction, 25% performance enhancement, and 62% energy saving, compared to STT-RAM cache design. The benefit over SRAM technology is even more significant.
---
paper_title: Multi retention level STT-RAM cache designs with a dynamic refresh scheme

paper_content:

Spin-transfer torque random access memory (STT-RAM) has received increasing attention because of its attractive features: good scalability, zero standby power, non-volatility and radiation hardness. The use of STT-RAM technology in the last level on-chip caches has been proposed as it minimizes cache leakage power with technology scaling down. Furthermore, the cell area of STT-RAM is only 1/9 ~ 1/3 that of SRAM. This allows for a much larger cache with the same die footprint, improving overall system performance through reducing cache misses. However, deploying STT-RAM technology in L1 caches is challenging because of the long and power-consuming write operations. In this paper, we propose both L1 and lower level cache designs that use STT-RAM. In particular, our designs use STT-RAM cells with various data retention time and write performances, made possible by different magnetic tunneling junction (MTJ) designs. For the fast STT-RAM bits with reduced data retention time, a counter controlled dynamic refresh scheme is proposed to maintain the data validity. Our dynamic scheme saves more than 80% refresh energy compared to the simple refresh scheme proposed in previous works. A L1 cache built with ultra low retention STT-RAM coupled with our proposed dynamic refresh scheme can achieve 9.2% in performance improvement, and saves up to 30% of the total energy when compared to one that uses traditional SRAM. For lower level caches with relative large cache capacity, we propose a data migration scheme that moves data between portions of the cache with different retention characteristics so as to maximize the performance and power benefits. Our experiments show that on the average, our proposed multi retention level STT-RAM cache reduces 30 ~ 70% of the total energy compared to previous works, while improving IPC performance for both 2-level and 3-level cache hierarchy.
---
paper_title: Coordinating prefetching and STT-RAM based last-level cache management for multicore systems

paper_content:

Data prefetching is a common mechanism to mitigate the bottleneck of off-chip memory bandwidth in modern computing systems. Unfortunately, the side effects of prefetching are an additional burden on off-chip communication and increased cache write operations. With the proposal of spin-transfer torque random access memory (STT-RAM) based last-level caches (LLCs) for their high density and low power consumption, the increase of write pressure to the cache from prefetching coupled with the characteristically long write access compared with traditional SRAM caches exacerbates the performance cost of prefetching schemes. In this work, we propose two orthogonal techniques to reduce the negative performance impact induced by aggressive prefetching on multicore systems employing STT-RAM based LLC. First, basic priority assignment prioritizes the different types of access requests of LLC by their criticality and responds to them based on priority. Second, priority boosting differentiates requests by application and prioritizes the relatively few requests from applications with non-intensive accesses to the LLC, which usually creates the most severe performance degradation in multi-core systems. Combining these two prioritization policies can alleviate the negative effect induced by aggressive prefetching. Our results show that these techniques can achieve an 8.3 average application speedup compared to a baseline, prefetch only design without prioritization.
---
paper_title: LastingNVCache: A Technique for Improving the Lifetime of Non-volatile Caches

paper_content:

Use of non-volatile memory (NVM) devices such as resistive RAM (ReRAM) and spin transfer torque RAM (STT-RAM) for designing on-chip caches holds the promise of providing a high-density, low-leakage alternative to SRAM. However, low write endurance of NVMs, along with the write-variation introduced by existing cache management schemes significantly limits the lifetime of NVM caches. We present LastingNVCache, a technique for improving the cache lifetime by mitigating the intra-set write variation. LastingNVCache works on the key idea that by periodically flushing a frequently-written data-item, next time the block can be made to load into a cold block in the set. Through this, the future writes to that data-item can be redirected from a hot block to a cold block, which leads to improvement in the cache lifetime. Microarchitectural simulations have shown that LastingNVCache provides 6.36X, 9.79X, and 10.94X improvement in lifetime for single, dual and quad-core systems, respectively. Also, its implementation overhead is small and it outperforms a recently proposed technique for improving lifetime of NVM caches.
---
paper_title: Low-current probabilistic writes for power-efficient STT-RAM caches

paper_content:

MRAM has emerged as one of the most attractive non-volatile solutions due to fast read access, low leakage power, high bit density, and long endurance. However, the high power consumption of write operations remains a barrier to the commercial adoption of MRAM technology. This paper addresses this problem by introducing low-current probabilistic writes (LCPW), a technique that reduces write access energy by lowering the amplitude of the write current pulse. Although low current pulses no longer guarantee successful bit write operations, we propose and evaluate a simple technique to ensure correctness and achieve significant power reduction over a typical MRAM implementation.
---
paper_title: Power-performance co-optimization of throughput core architecture using resistive memory

paper_content:

Massively parallel computing on throughput computers such as GPUs requires myriad memory accesses to register files, on-chip scratchpad, caches, and off-chip DRAM. Unlike CPUs, these processors have a large register file and on-chip scratchpad memory, which consume a significant portion of compute core power (35%-45%). In this paper, we introduce novel throughput architecture by integrating resistive memory (Spin Transfer Torque RAM) inside the compute core, which reduces leakage significantly, but introduces write power overhead and longer write latencies in GPU shared memory and register file accesses. We enhance the compute core by introducing register file organization with differential memory update mechanism to remove update redundancy during write operations. Furthermore, using merged register-write-mechanism and write-back buffer, we coalesce multithreaded GPU register write accesses to save write energy. In addition, we introduce hybrid shared memory design using SRAM and STT-MRAM that provides significant leakage/dynamic power savings without affecting performance. On average, across 23 GPGPU/graphics workloads, our schemes save 46% dynamic power due to register access (83% leakage power saving) with negligible performance degradation. On average, hybrid shared memory provides 10% reduction in dynamic power with maximum 1.6× performance improvement for the current workloads at no additional area overhead.
---
paper_title: An hybrid eDRAM/SRAM macrocell to implement first-level data caches

paper_content:

SRAM and DRAM cells have been the predominant technologies used to implement memory cells in computer systems, each one having its advantages and shortcomings. SRAM cells are faster and require no refresh since reads are not destructive. In contrast, DRAM cells provide higher density and minimal leakage energy since there are no paths within the cell from Vdd to ground. Recently, DRAM cells have been embedded in logic-based technology, thus overcoming the speed limit of typical DRAM cells. In this paper we propose an n-bit macrocell that implements one static cell, and n-1 dynamic cells. This cell is aimed at being used in an n-way set-associative first-level data cache. Our study shows that in a four-way set-associative cache with this macrocell compared to an SRAM based with the same capacity, leakage is reduced by about 75% and area more than half with a minimal impact on performance. Architectural mechanisms have also been devised to avoid refresh logic. Experimental results show that no performance is lost when the retention time is larger than 50K processor cycles. In addition, the proposed delayed writeback policy that avoids refreshing performs a similar amount of writebacks than a conventional cache with the same organization, so no power wasting is incurred.
---
paper_title: A Cache Reconfiguration Approach for Saving Leakage and Refresh Energy in Embedded DRAM Caches

paper_content:

In recent years, the size and leakage energy consumption of large last level caches (LLCs) has increased. To address this, embedded DRAM (eDRAM) caches have been considered which have lower leakage energy consumption; however eDRAM caches consume a significant amount of energy in the form of refresh energy. In this paper, we present a technique for saving both leakage and refresh energy in eDRAM caches. We use dynamic cache reconfiguration approach to intelligently turn-off part of the cache to save leakage energy and refresh only valid data of the active (i.e. not turned-off) cache to save refresh energy. We evaluate our technique using an x86-64 simulator and SPEC2006 benchmarks and compare it with a recently proposed technique for saving refresh energy, named Refrint. The experiments have shown that our technique provides better performance and energy efficiency than Refrint. Using our technique, for a 2MB LLC and 40 micro-seconds eDRAM refresh period, the average saving in energy over eDRAM baseline (which periodically refreshes all cache lines) is 22.8%.
---
paper_title: Using Cache-coloring to Mitigate Inter-set Write Variation in Non-volatile Caches

paper_content:

In recent years, researchers have explored use of non-volatile devices such as STT-RAM (spin torque transfer RAM) for designing on-chip caches, since they provide high density and consume low leakage power. A common limitation of all non-volatile devices is their limited write endurance. Further, since existing cache management policies are write-variation unaware, excessive writes to a few blocks may lead to a quick failure of the whole cache. We propose an architectural technique for wear-leveling of non-volatile last level caches (LLCs). Our technique uses cache-coloring approach which adds a software-controlled mapping layer between groups of physical pages and cache sets. Periodically the mapping is altered to ensure that write-traffic can be spread uniformly to different sets of the cache to achieve wear-leveling. Simulations performed with an x86-64 simulator and SPEC2006 benchmarks show that our technique reduces the worst-case writes to cache blocks and thus improves the cache lifetime by 4.07X.
---
paper_title: Reducing cache power with low-cost, multi-bit error-correcting codes

paper_content:

Technology advancements have enabled the integration of large on-die embedded DRAM (eDRAM) caches. eDRAM is significantly denser than traditional SRAMs, but must be periodically refreshed to retain data. Like SRAM, eDRAM is susceptible to device variations, which play a role in determining refresh time for eDRAM cells. Refresh power potentially represents a large fraction of overall system power, particularly during low-power states when the CPU is idle. Future designs need to reduce cache power without incurring the high cost of flushing cache data when entering low-power states. In this paper, we show the significant impact of variations on refresh time and cache power consumption for large eDRAM caches. We propose Hi-ECC, a technique that incorporates multi-bit error-correcting codes to significantly reduce refresh rate. Multi-bit error-correcting codes usually have a complex decoder design and high storage cost. Hi-ECC avoids the decoder complexity by using strong ECC codes to identify and disable sections of the cache with multi-bit failures, while providing efficient single-bit error correction for the common case. Hi-ECC includes additional optimizations that allow us to amortize the storage cost of the code over large data words, providing the benefit of multi-bit correction at same storage cost as a single-bit error-correcting (SECDED) code (2% overhead). Our proposal achieves a 93% reduction in refresh power vs. a baseline eDRAM cache without error correcting capability, and a 66% reduction in refresh power vs. a system using SECDED codes.
---
paper_title: Technology comparison for large last-level caches (L3Cs): Low-leakage SRAM, low write-energy STT-RAM, and refresh-optimized eDRAM

paper_content:

Large last-level caches (L3Cs) are frequently used to bridge the performance and power gap between processor and memory. Although traditional processors implement caches as SRAMs, technologies such as STT-RAM (MRAM), and eDRAM have been used and/or considered for the implementation of L3Cs. Each of these technologies has inherent weaknesses: SRAM is relatively low density and has high leakage current; STT-RAM has high write latency and write energy consumption; and eDRAM requires refresh operations. As future processors are expected to have larger last-level caches, the goal of this paper is to study the trade-offs associated with using each of these technologies to implement L3Cs. In order to make useful comparisons between SRAM, STTRAM, and eDRAM L3Cs, we model them in detail and apply low power techniques to each of these technologies to address their respective weaknesses. We optimize SRAM for low leakage and optimize STT-RAM for low write energy. Moreover, we classify eDRAM refresh-reduction schemes into two categories and demonstrate the effectiveness of using dead-line prediction to eliminate unnecessary refreshes. A comparison of these technologies through full-system simulation shows that the proposed refresh-reduction method makes eDRAM a viable, energy-efficient technology for implementing L3Cs.
---
paper_title: MASTER: A Multicore Cache Energy-Saving Technique Using Dynamic Cache Reconfiguration

paper_content:

With increasing number of on-chip cores and CMOS scaling, the size of last-level caches (LLCs) is on the rise and hence, managing their leakage energy consumption has become vital for continuing to scale performance. In multicore systems, the locality of memory access stream is significantly reduced because of multiplexing of access streams from different running programs and hence, leakage energy-saving techniques such as decay cache, which rely on memory access locality, do not save a large amount of energy. The techniques based on way level allocation provide very coarse granularity and the techniques based on offline profiling become infeasible to use for large number of cores. We present a multicore cache energy saving technique using dynamic cache reconfiguration (MASTER) that uses online profiling to predict energy consumption of running programs at multiple LLC sizes. Using these estimates, suitable cache quotas are allocated to different programs using cache coloring scheme and the unused LLC space is turned off to save energy. Even for four core systems, the implementation overhead of MASTER is only 0.8% of L2 size. We evaluate MASTER using out-of-order simulations with multiprogrammed workloads from SPEC2006 and compare it with conventional cache leakage energy-saving techniques. The results show that MASTER gives the highest saving in energy and does not harm performance or cause unfairness. For twoand four-core simulations, the average savings in memory subsystem (which includes LLC and main memory) energy over shared baseline LLC are 15% and 11%, respectively. Also, the average values of weighted speedup and fair speedup are close to one (≥0.98).
---
paper_title: CASHIER: A Cache Energy Saving Technique for QoS Systems

paper_content:

With each CMOS technology generation, leakage energy has been increasing at an exponential rate and hence, managing the energy consumption of large, last-level caches is becoming a critical research issue in modern chip design. Saving cache energy in QoS systems is especially challenging, since, to avoid missing deadlines, a suitable balance needs to be made between energy saving and performance loss. We present CASHIER, a Cache Energy Saving Technique for Quality of Service Systems. Cashier uses dynamic profiling to estimate the memory subsystem energy and execution time of the program under multiple last level cache (LLC) configurations. It then reconfigures LLC to an energy efficient configuration with a view to meet the deadline. In QoS systems, allowed slack may be specified either as percentage of baseline execution time or as absolute slack and Cashier can work for both these cases. The experiments show the effectiveness of Cashier in saving cache energy. For example, for an L2 cache size of 2MB and 5% allowed-slack over baseline, the average saving in memory subsystem energy by using Cashier is 23.6%.
---
paper_title: Versatile refresh: low complexity refresh scheduling for high-throughput multi-banked eDRAM

paper_content:

Multi-banked embedded DRAM (eDRAM) has become increasingly popular in high-performance systems. However, the data retention problem of eDRAM is exacerbated by the larger number of banks and the high-performance environment in which it is deployed: The data retention time of each memory cell decreases while the number of cells to be refreshed increases. For this, multi-bank designs offer a concurrent refresh mode, where idle banks can be refreshed concurrently during read and write operations. However, conventional techniques such as periodically scheduling refreshes---with priority given to refreshes in case of conflicts with reads or writes---have variable performance, increase read latency, and can perform poorly in worst case memory access patterns. We propose a novel refresh scheduling algorithm that is low-complexity, produces near-optimal throughput with universal guarantees, and is tolerant to bursty memory access patterns. The central idea is to decouple the scheduler into two simple-to-implement modules: one determines which cell to refresh next and the other determines when to force an idle cycle in all banks. We derive necessary and sufficient conditions to guarantee data integrity for all access patterns, with any given number of banks, rows per bank, read/write ports and data retention time. Our analysis shows that there is a tradeoff between refresh overhead and burst tolerance and characterizes this tradeoff precisely. The algorithm is shown to be near-optimal and achieves, for instance, 76.6% reduction in worst-case refresh overhead from the periodic refresh algorithm for a 250MHz eDRAM with 10us retention time and 16 banks each with 128 rows. Simulations with Apex-Map synthetic benchmarks and switch lookup table traffic show that VR can almost completely hide the refresh overhead for memory accesses with moderate-to-high multiplexing across memory banks.
---
paper_title: Memories: Exploiting Them and Developing Them

paper_content:

The disciplines of process development, circuit design, and microarchitecture too often remain isolated. This paper on memories explores the benefit of their unification by taking a retrospective look at a L1 cache memory design, a current view of embedded DRAM, and a speculative peek at emerging memories (e.g. MTJ MRAM). In the midst, a novel refresh operation is proposed for DRAM that relies on read and write activity ongoing within a cache to refresh the DRAM.
---
paper_title: Refrint: Intelligent refresh to minimize power in on-chip multiprocessor cache hierarchies

paper_content:

As manycores use dynamic energy ever more efficiently, static power consumption becomes a major concern. In particular, in a large manycore running at a low voltage, leakage in on-chip memory modules contributes substantially to the chip's power draw. This is unfortunate, given that, intuitively, the large multi-level cache hierarchy of a manycore is likely to contain a lot of useless data. An effective way to reduce this problem is to use a low-leakage technology such as embedded DRAM (eDRAM). However, eDRAM requires refresh. In this paper, we examine the opportunity of minimizing on-chip memory power further by intelligently refreshing on-chip eDRAM. We present Refrint, a simple approach to perform fine-grained, intelligent refresh of on-chip eDRAM multiprocessor cache hierarchies. We introduce the Refrint algorithms and microarchitecture. We evaluate Refrint in a simulated manycore running 16-threaded parallel applications. We show that an eDRAM-based memory hierarchy with Refrint consumes only 30% of the energy of a conventional SRAM-based memory hierarchy, and induces a slowdown of only 6%. In contrast, an eDRAM-based memory hierarchy without Refrint consumes 56% of the energy of the conventional memory hierarchy, inducing a slowdown of 25%.
---
paper_title: A Cache Reconfiguration Approach for Saving Leakage and Refresh Energy in Embedded DRAM Caches

paper_content:

In recent years, the size and leakage energy consumption of large last level caches (LLCs) has increased. To address this, embedded DRAM (eDRAM) caches have been considered which have lower leakage energy consumption; however eDRAM caches consume a significant amount of energy in the form of refresh energy. In this paper, we present a technique for saving both leakage and refresh energy in eDRAM caches. We use dynamic cache reconfiguration approach to intelligently turn-off part of the cache to save leakage energy and refresh only valid data of the active (i.e. not turned-off) cache to save refresh energy. We evaluate our technique using an x86-64 simulator and SPEC2006 benchmarks and compare it with a recently proposed technique for saving refresh energy, named Refrint. The experiments have shown that our technique provides better performance and energy efficiency than Refrint. Using our technique, for a 2MB LLC and 40 micro-seconds eDRAM refresh period, the average saving in energy over eDRAM baseline (which periodically refreshes all cache lines) is 22.8%.
---
paper_title: Mosaic: Exploiting the spatial locality of process variation to reduce refresh energy in on-chip eDRAM modules

paper_content:

EDRAM cells require periodic refresh, which ends up consuming substantial energy for large last-level caches. In practice, it is well known that different eDRAM cells can exhibit very different charge-retention properties. Unfortunately, current systems pessimistically assume worst-case retention times, and end up refreshing all the cells at a conservatively-high rate. In this paper, we propose an alternative approach. We use known facts about the factors that determine the retention properties of cells to build a new model of eDRAM retention times. The model is called Mosaic. The model shows that the retention times of cells in large eDRAM modules exhibit spatial correlation. Therefore, we logically divide the eDRAM module into regions or tiles, profile the retention properties of each tile, and program their refresh requirements in small counters in the cache controller. With this architecture, also called Mosaic, we refresh each tile at a different rate. The result is a 20x reduction in the number of refreshes in large eDRAM modules — practically eliminating refresh as a source of energy consumption.
---
paper_title: Relaxing non-volatility for fast and energy-efficient STT-RAM caches

paper_content:

Spin-Transfer Torque RAM (STT-RAM) is an emerging non-volatile memory technology that is a potential universal memory that could replace SRAM in processor caches. This paper presents a novel approach for redesigning STT-RAM memory cells to reduce the high dynamic energy and slow write latencies. We lower the retention time by reducing the planar area of the cell, thereby reducing the write current, which we then use with CACTI to design caches and memories. We simulate quad-core processor designs using a combination of SRAM- and STT-RAM-based caches. Since ultra-low retention STT-RAM may lose data, we also provide a preliminary evaluation for a simple, DRAMstyle refresh policy. We found that a pure STT-RAM cache hierarchy provides the best energy efficiency, though a hybrid design of SRAM-based L1 caches with reduced-retention STT-RAM L2 and L3 caches eliminates performance loss while still reducing the energy-delay product by more than 70%.
---
paper_title: Cache revive: architecting volatile STT-RAM caches for enhanced performance in CMPs

paper_content:

High density, low leakage and non-volatility are the attractive features of Spin-Transfer-Torque-RAM (STT-RAM), which has made it a strong competitor against SRAM as a universal memory replacement in multi-core systems. However, STT-RAM suffers from high write latency and energy which has impeded its widespread adoption. To this end, we look at trading-off STT-RAM's non-volatility property (data-retention-time) to overcome these problems. We formulate the relationship between retention-time and write-latency, and find optimal retention-time for architecting an efficient cache hierarchy using STT-RAM. Our results show that, compared to SRAM-based design, our proposal can improve performance and energy consumption by 18% and 60%, respectively.
---
paper_title: Resistive computation: avoiding the power wall with low-leakage, STT-MRAM based computing

paper_content:

As CMOS scales beyond the 45nm technology node, leakage concerns are starting to limit microprocessor performance growth. To keep dynamic power constant across process generations, traditional MOSFET scaling theory prescribes reducing supply and threshold voltages in proportion to device dimensions, a practice that induces an exponential increase in subthreshold leakage. As a result, leakage power has become comparable to dynamic power in current-generation processes, and will soon exceed it in magnitude if voltages are scaled down any further. Beyond this inflection point, multicore processors will not be able to afford keeping more than a small fraction of all cores active at any given moment. Multicore scaling will soon hit a power wall. This paper presents resistive computation, a new technique that aims at avoiding the power wall by migrating most of the functionality of a modern microprocessor from CMOS to spin-torque transfer magnetoresistive RAM (STT-MRAM)---a CMOS-compatible, leakage-resistant, non-volatile resistive memory technology. By implementing much of the on-chip storage and combinational logic using leakage-resistant, scalable RAM blocks and lookup tables, and by carefully re-architecting the pipeline, an STT-MRAM based implementation of an eight-core Sun Niagara-like CMT processor reduces chip-wide power dissipation by 1.7× and leakage power by 2.1× at the 32nm technology node, while maintaining 93% of the system throughput of a CMOS-based design.
---
paper_title: Multi retention level STT-RAM cache designs with a dynamic refresh scheme

paper_content:

Spin-transfer torque random access memory (STT-RAM) has received increasing attention because of its attractive features: good scalability, zero standby power, non-volatility and radiation hardness. The use of STT-RAM technology in the last level on-chip caches has been proposed as it minimizes cache leakage power with technology scaling down. Furthermore, the cell area of STT-RAM is only 1/9 ~ 1/3 that of SRAM. This allows for a much larger cache with the same die footprint, improving overall system performance through reducing cache misses. However, deploying STT-RAM technology in L1 caches is challenging because of the long and power-consuming write operations. In this paper, we propose both L1 and lower level cache designs that use STT-RAM. In particular, our designs use STT-RAM cells with various data retention time and write performances, made possible by different magnetic tunneling junction (MTJ) designs. For the fast STT-RAM bits with reduced data retention time, a counter controlled dynamic refresh scheme is proposed to maintain the data validity. Our dynamic scheme saves more than 80% refresh energy compared to the simple refresh scheme proposed in previous works. A L1 cache built with ultra low retention STT-RAM coupled with our proposed dynamic refresh scheme can achieve 9.2% in performance improvement, and saves up to 30% of the total energy when compared to one that uses traditional SRAM. For lower level caches with relative large cache capacity, we propose a data migration scheme that moves data between portions of the cache with different retention characteristics so as to maximize the performance and power benefits. Our experiments show that on the average, our proposed multi retention level STT-RAM cache reduces 30 ~ 70% of the total energy compared to previous works, while improving IPC performance for both 2-level and 3-level cache hierarchy.
---
paper_title: Energy-efficient Spin-Transfer Torque RAM cache exploiting additional all-zero-data flags

paper_content:

Large on-chip caches account for a considerable fraction of the total energy consumption in modern microprocessors. In this context, emerging Spin-Transfer Torque RAM (STT-RAM) has been regarded as a promising candidate to replace large on-chip SRAM caches in virtue of its nature of the zero leakage. However, large energy requirement of STT-RAM on write operations, resulting in a huge amount of dynamic energy consumption, precludes it from application to on-chip cache designs. In order to reduce the write energy of the STT-RAM cache thereby the total energy consumption, this paper provides an architectural technique which exploits the fact that many applications process a large number of zero data. The proposed design appends additional flags in cache tag arrays and set these additional bits if the corresponding data in the cache line is the zero-valued data in which all data bits are zero. Our experimental results show that the proposed cache design can reduce 73.78% and 69.30% of the dynamic energy on write operations at the byte and word granularities, respectively; total energy consumption reduced by 36.18% and 42.51%, respectively. In addition to the energy reduction, performance evaluation results indicate that the proposed cache improves the processor performance by 5.44% on average.
---
paper_title: Coding Last Level STT-RAM Cache for High Endurance and Low Power

paper_content:

STT-RAM technology has recently emerged as one of the most promising memory technologies. However, its major problems, limited write endurance and high write energy, are still preventing it from being used as a drop-in replacement of SRAM cache. In this paper, we propose a novel coding scheme for STT-RAM last level cache based on the concept of value locality. We reduce switching probability in cache by swapping common patterns with limited weight codes (LWC) to make writes less often as well as more uniform. We also define some policies for swapping these patterns. Our evaluation shows that bit write variance in memory cells can be reduced by about 20% on average resulting in a more uniform wear-out directly enhancing lifetime and improving cell reliability. In addition, writes in cache lines can be reduced by about 12% compared to one of the most effective circuit level techniques known as early write termination (EWT) [12]. Our method increases memory hierarchy access time by about 0.08% on average, which is negligible. We have shown that our method doesn't adversely affect last level cache energy-delay2. The non-uniformity caused by the coding scheme can be used for another coding scheme at main memory or L1 cache depending on their technologies.
---
paper_title: Energy- and endurance-aware design of phase change memory caches

paper_content:

Phase change memory (PCM) is one of the most promising technology among emerging non-volatile random access memory technologies. Implementing a cache memory using PCM provides many benefits such as high density, non-volatility, low leakage power, and high immunity to soft error. However, its disadvantages such as high write latency, high write energy, and limited write endurance prevent it from being used as a drop-in replacement of an SRAM cache. In this paper, we study a set of techniques to design an energy- and endurance-aware PCM cache. We also modeled the timing, energy, endurance, and area of PCM caches and integrated them into a PCM cache simulator to evaluate the techniques. Experiments show that our PCM cache design can achieve 8% of energy saving and 3.8 years of lifetime compared with a baseline PCM cache having less than a hour of lifetime.
---
paper_title: Future cache design using STT MRAMs for improved energy efficiency: devices, circuits and architecture

paper_content:

Spin-transfer torque magnetic RAM (STT MRAM) has emerged as a promising candidate for on-chip memory in future computing platforms. We present a cross-layer (device-circuit-architecture) approach to energy-efficient cache design using STT MRAM. At the device and circuit levels, we consider different genres of MTJs and bitcells, and evaluate their impact on the area, energy and performance of caches. In addition, we propose micro-architectural techniques viz. sequential cache read and partial cache line update, which exploit the non-volatility of STT MRAM to further improve energy efficiency of STT MRAM caches. A detailed comparison of STT MRAM caches with SRAM-based caches is also presented. Our results indicate that the proposed optimizations significantly enhance the efficiency of STT MRAM for designing lower level caches.
---
paper_title: A novel architecture of the 3D stacked MRAM L2 cache for CMPs

paper_content:

Magnetic random access memory (MRAM) is a promising memory technology, which has fast read access, high density, and non-volatility. Using 3D heterogeneous integrations, it becomes feasible and cost-efficient to stack MRAM atop conventional chip multiprocessors (CMPs). However, one disadvantage of MRAM is its long write latency and its high write energy. In this paper, we first stackMRAM-based L2 caches directly atop CMPs and compare it against SRAM counterparts in terms of performance and energy. We observe that the direct MRAM stacking might harm the chip performance due to the aforementioned long write latency and high write energy. To solve this problem, we then propose two architectural techniques: read-preemptive write buffer and SRAM-MRAM hybrid L2 cache. The simulation result shows that our optimized MRAM L2 cache improves performance by 4.91% and reduces power by 73.5%compared to the conventional SRAM L2 cache with the similar area.
---
paper_title: Asynchronous Asymmetrical Write Termination (AAWT) for a low power STT-MRAM

paper_content:

Spin Transfer Torque (STT) memory is an emerging and promising non-volatile storage technology. However, the high write current is still a major challenge which leads to a huge power consumption of the memory. Due to an inherent torque asymmetry of the Magnetic Tunnel Junction (MTJ) device employed in STT memories, the switching time between parallel to anti-parallel and anti-parallel to parallel magnetization is significantly different. Hence, the write latencies for writing `0' and `1' are also considerably different. In this paper, we propose a technique called Asynchronous Asymmetrical Write Termination (AAWT) which utilizes this asymmetrical behavior to terminate the write operations asynchronously and as a result significantly reduces the write power consumption. Furthermore, we present two different AAWT implementations to determine the actual write termination times. The first one makes use of a clock signal and the second one employs a self-timing approach based on an internal delay element. As shown by our experimental results, AAWT can reduce the total write energy by 30% in average with a negligible area overhead.
---
paper_title: Avoiding unnecessary write operations in STT-MRAM for low power implementation

paper_content:

Spin Transfer Torque (STT) is a promising emerging memory technology because of its various advantages such as non-volatility, high density, virtually infinite endurance, scalability and CMOS compatibility. Despite all these features, high write current is still a challenge for its widespread use. When writing a value that is already stored, a significant current flows through the Magnetic Tunnel Junction (MTJ) cell which is almost the same as that required to flip the stored data. This increases the total power consumption of the memory. To address this issue, we propose a technique which can avoid unnecessary write operations with bit-level granularity. Our technique can save 68.9% of the total write power consumption with a minor area overhead (0.68 %) and only a small timing penalty (1.33 %).
---
paper_title: Lower-bits cache for low power STT-RAM caches

paper_content:

As power-efficient design becomes more important, spin-transfer torque RAM (STT-RAM) has drawn a lot of attention due to its ability to meet both high performance and low power consumption. However, its high write energy incurs an increase of dynamic power consumption and may offset power saving due to its low static power. This paper proposes a novel technique called lower-bits caches for reducing write activities of STT-RAM L2 caches. Based on the observation that upper bits of data are not changed as frequently as lower bits in most applications, the technique tries to hide frequent bit changes in lower bits from the L2 cache. Experimental results show that our architecture reduced 25 percent of energy consumed by the L2 cache and slightly improved performance at the same time compared to the STT-RAM baseline.
---
paper_title: AWARE (Asymmetric Write Architecture With REdundant Blocks): A High Write Speed STT-MRAM Cache Architecture

paper_content:

Spin-transfer torque magnetic RAM (STT-MRAM) is a promising memory technology for lower level caches because of its high density and nonvolatile nature. However, the high write latency is a bottleneck to its widespread adoption as the future on-chip memory. In this paper, we propose a new cache architecture-asymmetric write architecture with redundant blocks (AWARE)-that can improve the write latency by taking advantage of the asymmetric write characteristics of 1T-1MTJ STT-MRAM bit-cells. Due to the nature of the storage element in STT-MRAM, the time required for the two-state transitions ( 1→ 0 and 0→ 1) is not identical. In other words, one of the state transitions is slower than the other direction. In conventional cache architecture, the overall write latency is limited by the slower transition. However, the AWARE cache design introduces redundant blocks in each row, and they are preset to the initial state that enables the faster transition. Hence the write operations performed in these redundant blocks are much faster than the conventional write scheme. The write latency in AWARE is improved by 30% over conventional cache architecture with no area penalty in the data array. Moreover, the additional tag bits introduced in this technique result in penalty on the total cache area. In addition, the write energy increases modestly by 7% in the proposed cache design. However, this write-energy increase can be mitigated by sacrificing the cache capacity.
---
paper_title: An energy efficient cache design using spin torque transfer (STT) RAM

paper_content:

The on-chip memory is a dominant source of power and energy consumption in modern and future processors. This paper explores the use of a new emerging non-volatile memory technology as a replacement for SRAM based lower level caches - Spin Torque Transfer(STT) RAM. While STTRAM achieves a reduction in leakage energy of 90% compared to SRAM, the dynamic energy for a write operation is 2X that of SRAM. Consequently, we propose additional microarchitectural optimizations to reduce overall dynamic energy which achieve an average reduction in dynamic energy over the base case of 30% with a range of 16% to 60% across 10 benchmarks.
---
paper_title: Energy reduction for STT-RAM using early write termination

paper_content:

The emerging Spin Torque Transfer memory (STT-RAM) is a promising candidate for future on-chip caches due to STT-RAM's high density, low leakage, long endurance and high access speed. However, one of the major challenges of STT-RAM is its high write current, which is disadvantageous when used as an on-chip cache since the dynamic power generated is too high. In this paper, we propose Early Write Termination (EWT), a novel technique to significantly reduce write energy with no performance penalty. EWT can be implemented with low complexity and low energy overhead. Our evaluation shows that up to 80% of write energy reduction can be achieved through EWT, resulting 33% less total energy consumption, and 34% reduction in ED2. These results indicate that EWT is an effective and practical scheme to improve the energy efficiency of a STT-RAM cache.
---
paper_title: Power-performance co-optimization of throughput core architecture using resistive memory

paper_content:

Massively parallel computing on throughput computers such as GPUs requires myriad memory accesses to register files, on-chip scratchpad, caches, and off-chip DRAM. Unlike CPUs, these processors have a large register file and on-chip scratchpad memory, which consume a significant portion of compute core power (35%-45%). In this paper, we introduce novel throughput architecture by integrating resistive memory (Spin Transfer Torque RAM) inside the compute core, which reduces leakage significantly, but introduces write power overhead and longer write latencies in GPU shared memory and register file accesses. We enhance the compute core by introducing register file organization with differential memory update mechanism to remove update redundancy during write operations. Furthermore, using merged register-write-mechanism and write-back buffer, we coalesce multithreaded GPU register write accesses to save write energy. In addition, we introduce hybrid shared memory design using SRAM and STT-MRAM that provides significant leakage/dynamic power savings without affecting performance. On average, across 23 GPGPU/graphics workloads, our schemes save 46% dynamic power due to register access (83% leakage power saving) with negligible performance degradation. On average, hybrid shared memory provides 10% reduction in dynamic power with maximum 1.6× performance improvement for the current workloads at no additional area overhead.
---
paper_title: Circuit and microarchitecture evaluation of 3D stacking magnetic RAM (MRAM) as a universal memory replacement

paper_content:

Magnetic random access memory (MRAM) has been considered as a promising memory technology due to many attractive properties. Integrating MRAM with CMOS logic may incur extra manufacture cost, due to its hybrid magnetic-CMOS fabrication process. Stacking MRAM on top of CMOS logics using 3D integration is a way to minimize this cost overhead. In this paper, we discuss the circuit design issues for MRAM, and present the MRAM cache model. Based on the model, we compare MRAM against SRAM and DRAM in terms of area, performance, and energy. Finally we conduct architectural evaluation for 3D microprocessor stacking with MRAM. The experimental results show that MRAM stacking offers competitive IPC performance with a large reduction in power consumption compared to SRAM and DRAM counterparts.
---
paper_title: OAP: an obstruction-aware cache management policy for STT-RAM last-level caches

paper_content:

Emerging memory technologies are explored as potential alternatives to traditional SRAM/DRAM-based memory architecture in future microprocessor designs. Among various emerging memory technologies, Spin-Torque Transfer RAM (STT-RAM) has the benefits of fast read latency, low leakage power, and high density, and therefore has been investigated as a promising candidate for last-level cache (LLC)1. One of the major disadvantages for STT-RAM is the latency and energy overhead associated with the write operations. In particular, a long-latency write operation to STT-RAM cache may obstruct other cache accesses and result in severe performance degradation. Consequently, mitigation techniques to minimize the write overhead are required in order to successfully adopt this new technology for cache design. In this paper, we propose an obstruction-aware cache management policy called OAP. OAP monitors the cache to periodically detect LLC-obstruction processes, and manage the cache accesses from different processes. The experimental results on a 4-core architecture with an 8MB STT-RAM L3 cache shows that the performance can be improved by 14% on average and up to 42%, with a reduction of energy consumption by 64%2.
---
paper_title: Coordinating prefetching and STT-RAM based last-level cache management for multicore systems

paper_content:

Data prefetching is a common mechanism to mitigate the bottleneck of off-chip memory bandwidth in modern computing systems. Unfortunately, the side effects of prefetching are an additional burden on off-chip communication and increased cache write operations. With the proposal of spin-transfer torque random access memory (STT-RAM) based last-level caches (LLCs) for their high density and low power consumption, the increase of write pressure to the cache from prefetching coupled with the characteristically long write access compared with traditional SRAM caches exacerbates the performance cost of prefetching schemes. In this work, we propose two orthogonal techniques to reduce the negative performance impact induced by aggressive prefetching on multicore systems employing STT-RAM based LLC. First, basic priority assignment prioritizes the different types of access requests of LLC by their criticality and responds to them based on priority. Second, priority boosting differentiates requests by application and prioritizes the relatively few requests from applications with non-intensive accesses to the LLC, which usually creates the most severe performance degradation in multi-core systems. Combining these two prioritization policies can alleviate the negative effect induced by aggressive prefetching. Our results show that these techniques can achieve an 8.3 average application speedup compared to a baseline, prefetch only design without prioritization.
---
paper_title: Asynchronous Asymmetrical Write Termination (AAWT) for a low power STT-MRAM

paper_content:

Spin Transfer Torque (STT) memory is an emerging and promising non-volatile storage technology. However, the high write current is still a major challenge which leads to a huge power consumption of the memory. Due to an inherent torque asymmetry of the Magnetic Tunnel Junction (MTJ) device employed in STT memories, the switching time between parallel to anti-parallel and anti-parallel to parallel magnetization is significantly different. Hence, the write latencies for writing `0' and `1' are also considerably different. In this paper, we propose a technique called Asynchronous Asymmetrical Write Termination (AAWT) which utilizes this asymmetrical behavior to terminate the write operations asynchronously and as a result significantly reduces the write power consumption. Furthermore, we present two different AAWT implementations to determine the actual write termination times. The first one makes use of a clock signal and the second one employs a self-timing approach based on an internal delay element. As shown by our experimental results, AAWT can reduce the total write energy by 30% in average with a negligible area overhead.
---
paper_title: AWARE (Asymmetric Write Architecture With REdundant Blocks): A High Write Speed STT-MRAM Cache Architecture

paper_content:

Spin-transfer torque magnetic RAM (STT-MRAM) is a promising memory technology for lower level caches because of its high density and nonvolatile nature. However, the high write latency is a bottleneck to its widespread adoption as the future on-chip memory. In this paper, we propose a new cache architecture-asymmetric write architecture with redundant blocks (AWARE)-that can improve the write latency by taking advantage of the asymmetric write characteristics of 1T-1MTJ STT-MRAM bit-cells. Due to the nature of the storage element in STT-MRAM, the time required for the two-state transitions ( 1→ 0 and 0→ 1) is not identical. In other words, one of the state transitions is slower than the other direction. In conventional cache architecture, the overall write latency is limited by the slower transition. However, the AWARE cache design introduces redundant blocks in each row, and they are preset to the initial state that enables the faster transition. Hence the write operations performed in these redundant blocks are much faster than the conventional write scheme. The write latency in AWARE is improved by 30% over conventional cache architecture with no area penalty in the data array. Moreover, the additional tag bits introduced in this technique result in penalty on the total cache area. In addition, the write energy increases modestly by 7% in the proposed cache design. However, this write-energy increase can be mitigated by sacrificing the cache capacity.
---
paper_title: On-chip caches built on multilevel spin-transfer torque RAM cells and its optimizations

paper_content:

It has been predicted that a processor's caches could occupy as much as 90p of chip area a few technology nodes from the current ones. In this article, we investigate the use of multilevel spin-transfer torque RAM (STT-RAM) cells in the design of processor caches. We start with examining the access (read and write) scheme for multilevel cell (MLC) STT-RAM from a circuit design perspective, detailing the read and write circuits. Compared to traditional SRAM caches, a multilevel cell (MLC) STT-RAM cache design is denser, fast, and requires less energy. However, a number of critical architecture-level issues remain to be solved before MLC STT-RAM technology can be deployed in processor caches. We shall offer solutions to the issue of bit encoding as well as tackle the write endurance problem. In particular, the latter has been neglected in previous works on STT-RAM caches. We propose a set remapping scheme that can potentially prolong the lifetime of a MLC STT-RAM cache by 80× on average. Furthermore, a method for recovering the performance that may be lost in some applications due to set remapping is proposed. The impacts of process variations of the MLC STT-RAM cell on the robustness of the memory hierarchy is also discussed, together with various enhancement techniques, namely, ECC and design redundancy.
---
paper_title: High-endurance and performance-efficient design of hybrid cache architectures through adaptive line replacement

paper_content:

In this paper, we propose a run-time strategy for managing writes onto last level cache in chip multiprocessors where STT-RAM memory is used as baseline technology. To this end, we assume that each cache set is decomposed into limited SRAM lines and large number of STT-RAM lines. SRAM lines are target of frequently-written data and rarely-written or read-only ones are pushed into STT-RAM. As a novel contribution, a low-overhead, fully-hardware technique is utilized to detect write-intensive data blocks of working set and place them into SRAM lines while the remaining data blocks are candidates to be remapped onto STT-RAM blocks during system operation. Therefore, the achieved cache architecture has large capacity and consumes near zero leakage energy using STT-RAM array; while dynamic write energy, acceptable write latency, and long lifetime is guaranteed via SRAM array. Results of full-system simulation for a quad-core CMP running PARSEC-2 benchmark suit confirm an average of 49 times improvement in cache lifetime and more than 50% reduction in cache power consumption when compared to baseline configurations.
---
paper_title: WriteSmoothing: improving lifetime of non-volatile caches using intra-set wear-leveling

paper_content:

Driven by the trends of increasing core-count and bandwidth-wall problem, the size of last level caches (LLCs) has greatly increased. Since SRAM consumes high leakage power, researchers have explored use of non-volatile memories (NVMs) for designing caches as they provide high density and consume low leakage power. However, since NVMs have low write-endurance and the existing cache management policies are write variation-unaware, effective wear-leveling techniques are required for achieving reasonable cache lifetimes using NVMs. We present WriteSmoothing, a technique for mitigating intra-set write variation in NVM caches. WriteSmoothing logically divides the cache-sets into multiple modules. For each module, WriteSmoothing collectively records number of writes in each way for any of the sets. It then periodically makes most frequently written ways in a module unavailable to shift the write-pressure to other ways in the sets of the module. Extensive simulation results have shown that on average, for single and dual-core system configurations, WriteSmoothing improves cache lifetime by 2.17X and 2.75X, respectively. Also, its implementation overhead is small and it works well for a wide range of algorithm and system parameters.
---
paper_title: i2WAP: Improving non-volatile cache lifetime by reducing inter- and intra-set write variations

paper_content:

Modern computers require large on-chip caches, but the scalability of traditional SRAM and eDRAM caches is constrained by leakage and cell density. Emerging non-volatile memory (NVM) is a promising alternative to build large on-chip caches. However, limited write endurance is a common problem for non-volatile memory technologies. In addition, today's cache management might result in unbalanced write traffic to cache blocks causing heavily-written cache blocks to fail much earlier than others. Unfortunately, existing wear-leveling techniques for NVM-based main memories cannot be simply applied to NVM-based on-chip caches because cache writes have intra-set variations as well as inter-set variations. To solve this problem, we propose i2WAP, a new cache management policy that can reduce both inter- and intra-set write variations. i2WAP has two features: (1) Swap-Shift, an enhancement based on previous main memory wear-leveling to reduce cache inter-set write variations; (2) Probabilistic Set Line Flush, a novel technique to reduce cache intra-set write variations. Implementing i2WAP only needs two global counters and two global registers. By adopting i2WAP, we can improve the lifetime of on-chip non-volatile caches by 75% on average and up to 224%.
---
paper_title: LastingNVCache: A Technique for Improving the Lifetime of Non-volatile Caches

paper_content:

Use of non-volatile memory (NVM) devices such as resistive RAM (ReRAM) and spin transfer torque RAM (STT-RAM) for designing on-chip caches holds the promise of providing a high-density, low-leakage alternative to SRAM. However, low write endurance of NVMs, along with the write-variation introduced by existing cache management schemes significantly limits the lifetime of NVM caches. We present LastingNVCache, a technique for improving the cache lifetime by mitigating the intra-set write variation. LastingNVCache works on the key idea that by periodically flushing a frequently-written data-item, next time the block can be made to load into a cold block in the set. Through this, the future writes to that data-item can be redirected from a hot block to a cold block, which leads to improvement in the cache lifetime. Microarchitectural simulations have shown that LastingNVCache provides 6.36X, 9.79X, and 10.94X improvement in lifetime for single, dual and quad-core systems, respectively. Also, its implementation overhead is small and it outperforms a recently proposed technique for improving lifetime of NVM caches.
---
paper_title: Using Cache-coloring to Mitigate Inter-set Write Variation in Non-volatile Caches

paper_content:

In recent years, researchers have explored use of non-volatile devices such as STT-RAM (spin torque transfer RAM) for designing on-chip caches, since they provide high density and consume low leakage power. A common limitation of all non-volatile devices is their limited write endurance. Further, since existing cache management policies are write-variation unaware, excessive writes to a few blocks may lead to a quick failure of the whole cache. We propose an architectural technique for wear-leveling of non-volatile last level caches (LLCs). Our technique uses cache-coloring approach which adds a software-controlled mapping layer between groups of physical pages and cache sets. Periodically the mapping is altered to ensure that write-traffic can be spread uniformly to different sets of the cache to achieve wear-leveling. Simulations performed with an x86-64 simulator and SPEC2006 benchmarks show that our technique reduces the worst-case writes to cache blocks and thus improves the cache lifetime by 4.07X.
---
paper_title: Power-performance co-optimization of throughput core architecture using resistive memory

paper_content:

Massively parallel computing on throughput computers such as GPUs requires myriad memory accesses to register files, on-chip scratchpad, caches, and off-chip DRAM. Unlike CPUs, these processors have a large register file and on-chip scratchpad memory, which consume a significant portion of compute core power (35%-45%). In this paper, we introduce novel throughput architecture by integrating resistive memory (Spin Transfer Torque RAM) inside the compute core, which reduces leakage significantly, but introduces write power overhead and longer write latencies in GPU shared memory and register file accesses. We enhance the compute core by introducing register file organization with differential memory update mechanism to remove update redundancy during write operations. Furthermore, using merged register-write-mechanism and write-back buffer, we coalesce multithreaded GPU register write accesses to save write energy. In addition, we introduce hybrid shared memory design using SRAM and STT-MRAM that provides significant leakage/dynamic power savings without affecting performance. On average, across 23 GPGPU/graphics workloads, our schemes save 46% dynamic power due to register access (83% leakage power saving) with negligible performance degradation. On average, hybrid shared memory provides 10% reduction in dynamic power with maximum 1.6× performance improvement for the current workloads at no additional area overhead.
---
paper_title: WriteSmoothing: improving lifetime of non-volatile caches using intra-set wear-leveling

paper_content:

Driven by the trends of increasing core-count and bandwidth-wall problem, the size of last level caches (LLCs) has greatly increased. Since SRAM consumes high leakage power, researchers have explored use of non-volatile memories (NVMs) for designing caches as they provide high density and consume low leakage power. However, since NVMs have low write-endurance and the existing cache management policies are write variation-unaware, effective wear-leveling techniques are required for achieving reasonable cache lifetimes using NVMs. We present WriteSmoothing, a technique for mitigating intra-set write variation in NVM caches. WriteSmoothing logically divides the cache-sets into multiple modules. For each module, WriteSmoothing collectively records number of writes in each way for any of the sets. It then periodically makes most frequently written ways in a module unavailable to shift the write-pressure to other ways in the sets of the module. Extensive simulation results have shown that on average, for single and dual-core system configurations, WriteSmoothing improves cache lifetime by 2.17X and 2.75X, respectively. Also, its implementation overhead is small and it works well for a wide range of algorithm and system parameters.
---
paper_title: i2WAP: Improving non-volatile cache lifetime by reducing inter- and intra-set write variations

paper_content:

Modern computers require large on-chip caches, but the scalability of traditional SRAM and eDRAM caches is constrained by leakage and cell density. Emerging non-volatile memory (NVM) is a promising alternative to build large on-chip caches. However, limited write endurance is a common problem for non-volatile memory technologies. In addition, today's cache management might result in unbalanced write traffic to cache blocks causing heavily-written cache blocks to fail much earlier than others. Unfortunately, existing wear-leveling techniques for NVM-based main memories cannot be simply applied to NVM-based on-chip caches because cache writes have intra-set variations as well as inter-set variations. To solve this problem, we propose i2WAP, a new cache management policy that can reduce both inter- and intra-set write variations. i2WAP has two features: (1) Swap-Shift, an enhancement based on previous main memory wear-leveling to reduce cache inter-set write variations; (2) Probabilistic Set Line Flush, a novel technique to reduce cache intra-set write variations. Implementing i2WAP only needs two global counters and two global registers. By adopting i2WAP, we can improve the lifetime of on-chip non-volatile caches by 75% on average and up to 224%.
---
paper_title: LastingNVCache: A Technique for Improving the Lifetime of Non-volatile Caches

paper_content:

Use of non-volatile memory (NVM) devices such as resistive RAM (ReRAM) and spin transfer torque RAM (STT-RAM) for designing on-chip caches holds the promise of providing a high-density, low-leakage alternative to SRAM. However, low write endurance of NVMs, along with the write-variation introduced by existing cache management schemes significantly limits the lifetime of NVM caches. We present LastingNVCache, a technique for improving the cache lifetime by mitigating the intra-set write variation. LastingNVCache works on the key idea that by periodically flushing a frequently-written data-item, next time the block can be made to load into a cold block in the set. Through this, the future writes to that data-item can be redirected from a hot block to a cold block, which leads to improvement in the cache lifetime. Microarchitectural simulations have shown that LastingNVCache provides 6.36X, 9.79X, and 10.94X improvement in lifetime for single, dual and quad-core systems, respectively. Also, its implementation overhead is small and it outperforms a recently proposed technique for improving lifetime of NVM caches.
---
paper_title: Energy- and endurance-aware design of phase change memory caches

paper_content:

Phase change memory (PCM) is one of the most promising technology among emerging non-volatile random access memory technologies. Implementing a cache memory using PCM provides many benefits such as high density, non-volatility, low leakage power, and high immunity to soft error. However, its disadvantages such as high write latency, high write energy, and limited write endurance prevent it from being used as a drop-in replacement of an SRAM cache. In this paper, we study a set of techniques to design an energy- and endurance-aware PCM cache. We also modeled the timing, energy, endurance, and area of PCM caches and integrated them into a PCM cache simulator to evaluate the techniques. Experiments show that our PCM cache design can achieve 8% of energy saving and 3.8 years of lifetime compared with a baseline PCM cache having less than a hour of lifetime.
---
paper_title: DWM-TAPESTRI - an energy efficient all-spin cache using domain wall shift based writes

paper_content:

Spin-based memories are promising candidates for future on-chip memories due to their high density, non-volatility, and very low leakage. However, the high energy and latency of write operations in these memories is a major challenge. In this work, we explore a new approach -- shift based write -- that offers a fast and energy-efficient alternative to performing writes in spin-based memories. We propose DWM-TAPESTRI, a new all-spin cache design that utilizes Domain Wall Memory (DWM) with shift based writes at all levels of the cache hierarchy. The proposed write scheme enables DWM to be used, for the first time, in L1 caches and in tag arrays, where the inefficiency of writes in spin memories has traditionally precluded their use. At the circuit level, we propose bit-cell designs utilizing shift-based writes, which are tailored to the differing requirements of different levels in the cache hierarchy. We also propose pre-shifting as an architectural technique to hide the latency of shift operations that is inherent to DWM. We performed a systematic device-circuit-architecture evaluation of the proposed design. Over a wide range of SPEC 2006 benchmarks, DWM-TAPESTRI achieves 8.2X improvement in energy and 4X improvement in area, with virtually identical performance, compared to an iso-capacity SRAM cache. Compared to an iso-capacity STT-MRAM cache, the proposed design achieves around 1.6X improvement in both area and energy under iso-performance conditions.
---
paper_title: Cross-layer racetrack memory design for ultra high density and low power consumption

paper_content:

The racetrack memory technology utilizes magnetic domains along a nanoscopic wire to obtain ultra-high data storage density. The recent success in the planar racetrack nanowire promised its fabrication feasibility and future scalability, bringing more design challenges and opportunities. In this paper, we initialize the optimization of racetrack memory embracing design considerations across multiple layers, including cell design, array structure, architecture organization, and data management. Our evaluation shows that racetrack memory based cache can achieve 6.4× area reduction, 25% performance enhancement, and 62% energy saving, compared to STT-RAM cache design. The benefit over SRAM technology is even more significant.
---
paper_title: Technology comparison for large last-level caches (L3Cs): Low-leakage SRAM, low write-energy STT-RAM, and refresh-optimized eDRAM

paper_content:

Large last-level caches (L3Cs) are frequently used to bridge the performance and power gap between processor and memory. Although traditional processors implement caches as SRAMs, technologies such as STT-RAM (MRAM), and eDRAM have been used and/or considered for the implementation of L3Cs. Each of these technologies has inherent weaknesses: SRAM is relatively low density and has high leakage current; STT-RAM has high write latency and write energy consumption; and eDRAM requires refresh operations. As future processors are expected to have larger last-level caches, the goal of this paper is to study the trade-offs associated with using each of these technologies to implement L3Cs. In order to make useful comparisons between SRAM, STTRAM, and eDRAM L3Cs, we model them in detail and apply low power techniques to each of these technologies to address their respective weaknesses. We optimize SRAM for low leakage and optimize STT-RAM for low write energy. Moreover, we classify eDRAM refresh-reduction schemes into two categories and demonstrate the effectiveness of using dead-line prediction to eliminate unnecessary refreshes. A comparison of these technologies through full-system simulation shows that the proposed refresh-reduction method makes eDRAM a viable, energy-efficient technology for implementing L3Cs.
---
paper_title: Implementing a hybrid SRAM / eDRAM NUCA architecture

paper_content:

Advances in technology allowed for integrating DRAM-like structures into the chip, called embedded DRAM (eDRAM). This technology has already been successfully implemented in some GPUs and other graphic-intensive SoC, like game consoles. The most recent processor from IBM®, POWER7, is the first general-purpose processor that integrates an eDRAM module on the chip. In this paper, we propose a hybrid cache architecture that exploits the main features of both memory technologies, speed of SRAM and high density of eDRAM. We demonstrate, that due to the high locality found in emerging applications, a high percentage of data that enters to the on-chip last-level cache are not accessed again before they are evicted. Based on that observation, we propose a placement scheme where re-accessed data blocks are stored in fast, but costly in terms of area and power, SRAM banks, while eDRAM banks store data blo cks that just arrive to the NUCA cache or were demoted from a SRAM bank. We show that a well-balanced SRAM / eDRAM NUCA cache can achieve similar performance results than using a NUCA cache composed of only SRAM banks, but reduces area by 15% and power consumed by 10%. Furthermore, we also explore several alternatives to exploit the area reduction we gain by using the hybrid architecture, resulting in an overall performance improvement of 4%.
---
paper_title: Analyzing the optimal ratio of SRAM banks in hybrid caches

paper_content:

Cache memories have been typically implemented with Static Random Access Memory (SRAM) technology. This technology presents a fast access time but high energy consumption and low density. As opposite, the recently appeared embedded Dynamic RAM (eDRAM) technology allows caches to be built with lower energy and area, although with a slower access time. The eDRAM technology provides important leakage and area savings, especially in huge Last-Level Caches (LLCs), which occupy almost half the silicon area in some recent microprocessors. This paper proposes a novel hybrid LLC, which combines SRAM and eDRAM banks to address the trade-off among performance, energy, and area. To this end, we explore the optimal percentage of SRAM and eDRAM banks that achieves the best target trade-off. Architectural mechanisms have been devised to keep the most likely accessed blocks in fast SRAM banks as well as to avoid unnecessary destructive reads. Experimental results show that, compared to a conventional SRAM LLC with the same storage capacity, performance degradation does not surpass, on average, 2.9% (even with 12.5% of banks built with SRAM technology), whereas area savings can be as high as 46% for a 1MB-16way LLC. For a 45nm technology node, the energy-delay squared product confirms that a hybrid cache is a better design than the conventional SRAM cache regardless the number of eDRAM banks, and also better than a conventional eDRAM cache when the number of SRAM banks is a quarter or an eighth of the cache banks.
---
paper_title: An hybrid eDRAM/SRAM macrocell to implement first-level data caches

paper_content:

SRAM and DRAM cells have been the predominant technologies used to implement memory cells in computer systems, each one having its advantages and shortcomings. SRAM cells are faster and require no refresh since reads are not destructive. In contrast, DRAM cells provide higher density and minimal leakage energy since there are no paths within the cell from Vdd to ground. Recently, DRAM cells have been embedded in logic-based technology, thus overcoming the speed limit of typical DRAM cells. In this paper we propose an n-bit macrocell that implements one static cell, and n-1 dynamic cells. This cell is aimed at being used in an n-way set-associative first-level data cache. Our study shows that in a four-way set-associative cache with this macrocell compared to an SRAM based with the same capacity, leakage is reduced by about 75% and area more than half with a minimal impact on performance. Architectural mechanisms have also been devised to avoid refresh logic. Experimental results show that no performance is lost when the retention time is larger than 50K processor cycles. In addition, the proposed delayed writeback policy that avoids refreshing performs a similar amount of writebacks than a conventional cache with the same organization, so no power wasting is incurred.
---
paper_title: Bandwidth-aware reconfigurable cache design with hybrid memory technologies

paper_content:

In chip-multiprocessor (CMP) designs, limited memory bandwidth is a potential bottleneck of the system performance. New memory technologies, such as spin-torque-transfer memory (STT-RAM), resistive memory (RRAM), and embedded DRAM (eDRAM), are promising on-chip memory solutions for CMPs. In this paper, we propose a bandwidth-aware re-configurable cache hierarchy (BARCH) with hybrid memory technologies. BARCH consists of a hybrid cache hierarchy, a reconfiguration mechanism, and a statistical prediction engine. Our hybrid cache hierarchy chooses different memory technologies to configure each level so that the bandwidth provided by the overall hierarchy is optimized. Furthermore, we present a reconfiguration mechanism to dynamically adapt the cache space of each level based on the predicted bandwidth demands of different applications, which is guaranteed by our prediction engine. We evaluate the system performance gain obtained by our method with a set of multithreaded and multiprogrammed applications. Compared to traditional SRAM-based cache designs, our proposed design improves the system throughput by 58% and 14% for multithreaded and multiprogrammed applications, respectively.1
---
paper_title: Moguls: A model to explore the memory hierarchy for bandwidth improvements

paper_content:

In recent years, the increasing number of processor cores and limited increases in main memory bandwidth have led to the problem of the bandwidth wall, where memory bandwidth is becoming a performance bottleneck. This is especially true for emerging latency-insensitive, bandwidth-sensitive applications. Designing the memory hierarchy for a platform with an emphasis on maximizing bandwidth within a fixed power budget becomes one of the key challenges. To facilitate architects to quickly explore the design space of memory hierarchies, we propose an analytical performance model called Moguls. The Moguls model estimates the performance of an application on a system, using the bandwidth demand of the application for a range of cache capacities and the bandwidth provided by the system with those capacities. We show how to extend this model with appropriate approximations to optimize a cache hierarchy under a power constraint. The results show how many levels of cache should be designed, and what the capacity, bandwidth, and technology of each level should be. In addition, we study memory hierarchy design with hybrid memory technologies, which shows the benefits of using multiple technologies for future computing systems.
---
paper_title: Static and dynamic co-optimizations for blocks mapping in hybrid caches

paper_content:

In this paper, a combined static and dynamic scheme is proposed to optimize the block placement for endurance and energy-efficiency in a hybrid SRAM and STT-RAM cache. With the proposed scheme, STT-RAM endurance is maximized while performance is maintained. We use the compiler to provide static hints to guide initial data placement, and use the hardware to correct the hints based on the run-time cache behavior. Experimental results show that the combined scheme improves the endurance by 23.9x and 5.9x compared to pure static and pure dynamic optimizations respectively. Furthermore, the system energy can be reduced by 17% compared to pure dynamic optimization through minimizing STT-RAM writes.
---
paper_title: Write intensity prediction for energy-efficient non-volatile caches

paper_content:

This paper presents a novel concept called write intensity prediction for energy-efficient non-volatile caches as well as the architecture that implements the concept. The key idea is to correlate write intensity of cache blocks with addresses of memory access instructions that incur cache misses of those blocks. The predictor keeps track of instructions that tend to load write-intensive blocks and utilizes that information to predict write intensity of blocks. Based on this concept, we propose a block placement strategy driven by write intensity prediction for SRAM/STT-RAM hybrid caches. Experimental results show that the proposed approach reduces write energy consumption by 55% on average compared to the existing hybrid cache architecture.
---
paper_title: Prediction table based management policy for STT-RAM and SRAM hybrid cache

paper_content:

STT-RAM has emerged as one of the most promising candidates for future on-chip cache due to STT-RAM's high density and low leakage. However, the STT-RAM has many disadvantages, such as: high write energy and long writing latency. To alleviate the problems, many STT-RAM and SRAM hybrid cache architectures have been proposed. In this paper, we propose a prediction table based hybrid L2 cache management policy named PTHCM. By adding a prediction table to the hybrid cache, the access information of each cache line can be recorded and we can predict the cache lines that have become dead in SRAM, lines that will be less-written and frequently-written in advance. Based on these information provided by prediction table, we propose a new cache replacement and management policy for hybrid cache architecture which replaces the dead cache lines in SRAM to improve the utilization ratio of SRAM, places frequently-written lines into SRAM and swaps out less-written lines from SRAM as early as possible. Compared with a baseline hybrid cache design, our design reduces 23.3% of the write operations to STT-RAM, and achieves 8.2% power reduction.
---
paper_title: Exploiting set-level write non-uniformity for energy-efficient NVM-based hybrid cache

paper_content:

Hybrid cache architectures have been proposed to mitigate the increasing on-chip power dissipation through the exploitation of the emerging non-volatile memories (NVMs). To overcome the high energy and long latency associated with write operations of NVMs, a small SRAM is typically incorporated into the hybrid cache for accommodating write-intensive cache blocks. How to efficiently manage this SRAM and manipulate the write operations are crucial to the performance of the hybrid cache. In this paper, we first present our observation that the intensity of write operations on different cache sets is usually non-uniform for real applications, such as multimedia, multi-programmed, multithreaded applications. The previously proposed hybrid cache schemes can not efficiently and symmetrically utilize the small SRAM to accommodate such widely-existing non-uniform writes on cache sets. Based on this observation, we propose a novel hybrid cache design, Dual Associative Hybrid Cache (denoted as DAHYC), as well as the corresponding cache management policy. By organizing the SRAM blocks in the hybrid cache as a semi-independent set-associative cache, several hybrid cache sets can efficiently share and cooperatively utilize their SRAM blocks, instead of exclusively utilizing the SRAM blocks in each cache set in previous hybrid cache schemes, to boost power-efficiency. Through prudently manipulating the locality information of SRAM blocks in both the NVM sets and the SRAM sets, the proposed cache management policy also delivers high-performance. Experimental results show that, compared with previous works, the DAHYC can reduce the dynamic power of the hybrid cache by 24.8% on average and up to 54% for SPEC2000 INT benchmarks, while at the same time improving the performance of the hybrid cache by 1.16% on average.
---
paper_title: Dynamically reconfigurable hybrid cache: An energy-efficient last-level cache design

paper_content:

The recent development of non-volatile memory (NVM), such as spin-torque transfer magnetoresistive RAM (STT-RAM) and phase-change RAM (PRAM), with the advantage of low leakage and high density, provides an energy-efficient alternative to traditional SRAM in cache systems. We propose a novel reconfigurable hybrid cache architecture (RHC), in which NVM is incorporated in the last-level cache together with SRAM. RHC can be reconfigured by powering on/off SRAM/NVM arrays in a way-based manner. In this work, we discuss both the architecture and circuit design issues for RHC. Furthermore, we provide hardware-based mechanisms to dynamically reconfigure RHC on-the-fly based on the cache demand. Experimental results on a wide range of benchmarks show that the proposed RHC achieves an average 63%, 48% and 25% energy saving over non-reconfigurable SRAM-based cache, non-reconfigurable hybrid cache, and reconfigurable SRAM-based cache, while maintaining the system performance (at most 4% performance overhead).
---
paper_title: A software approach for combating asymmetries of non-volatile memories

paper_content:

The recent advances in non-volatile memory technologies promise the delivery of future high performance and low power computing systems. While these technologies provide attractive features, they exhibit different degrees of asymmetric read/write behavior, resulting in under utilization of the technology benefits in many scenarios. Future systems should be aware of this asymmetry to fully utilize these memory technologies. In this paper, we propose software dispatch, a cross-layer approach to distribute data to appropriate memory resources based on an application's data access characteristics. We demonstrate the application of the proposed technique through a case study system with hybrid memory caches, on which we achieve over 5% performance improvement and 9.8% power savings compared to the state-of-the-art technique.
---
paper_title: MAC: migration-aware compilation for STT-RAM based hybrid cache in embedded systems

paper_content:

Hybrid caches consisting of both STT-RAM and SRAM have been proposed recently for energy efficiency. To explore the advantages of hybrid cache, most work on hybrid caches employs migration based strategies to dynamically move write-intensive data from STT-RAM to SRAM. Migrations require additional read and write operations for data movement and may lead to significant overheads. To address this issue, this paper proposes a Migration-Aware Compilation (MAC) approach to improve the energy efficiency and performance of STT-RAM based hybrid cache. By re-arranging data layout, the data access pattern in memory blocks is changed such that the number of migrations is reduced without any hardware modification. The reduction of migration overheads in turn improves energy efficiency and performance. The experimental results show that with the proposed approach, on average, the number of write operations on STT-RAM is reduced by 13.4%, the number of migrations is reduced by 16.1%, the total dynamic energy is reduced by 8.5%, and the total latency is reduced by 12.1%.
---
paper_title: Automatic Feedback Control of Shared Hybrid Caches in 3D Chip Multiprocessors

paper_content:

3D integration enables building caches from different types of technologies such as SRAM, Magnetic RAM (MRAM), DRAM, and Phase-change RAM (PRAM). Hybrid cache architectures (HCAs) have been proposed to take advantage of the benefits offered by these types of technologies. Employing this novel cache architecture to build shared caches in chip multiprocessors (CMPs) can lead to significant performance and power consumption improvements. In this paper, we focus on a 3D CMP design in which the shared last level L2 cache is composed of an MRAM layer and an SRAM layer stacked upon the processing cores and present a control theory centric approach designed to partition this shared hybrid L2 cache space dynamically among concurrently running applications in order to satisfy the application-level performance QoS targets. At each time interval, the two layers of the hybrid L2 cache are partitioned, based on the cache demands made by the controllers of the applications, to satisfy the specified performance targets. We evaluate our feedback control based scheme using various workloads. Our experimental evaluation shows that the proposed scheme is able to satisfy the specified performance QoS in most of the tested cases, by partitioning the hybrid cache space of the 3D CMP among co-runner applications.
---
paper_title: Hybrid cache architecture with disparate memory technologies

paper_content:

Caching techniques have been an efficient mechanism for mitigating the effects of the processor-memory speed gap. Traditional multi-level SRAM-based cache hierarchies, especially in the context of chip multiprocessors (CMPs), present many challenges in area requirements, core-to-cache balance, power consumption, and design complexity. New advancements in technology enable caches to be built from other technologies, such as Embedded DRAM (EDRAM), Magnetic RAM (MRAM), and Phase-change RAM (PRAM), in both 2D chips or 3D stacked chips. Caches fabricated in these technologies offer dramatically different power and performance characteristics when compared with SRAM-based caches, particularly in the areas of access latency, cell density, and overall power consumption. In this paper, we propose to take advantage of the best characteristics that each technology offers, through the use of Hybrid Cache Architecture (HCA) designs. We discuss and evaluate two types of hybrid cache architectures: inter cache Level HCA (LHCA), in which the levels in a cache hierarchy can be made of disparate memory technologies; and intra cache level or cache Region based HCA (RHCA), where a single level of cache can be partitioned into multiple regions, each of a different memory technology. We have studied a number of different HCA architectures and explored the potential of hardware support for intra-cache data movement and power consumption management within HCA caches. Utilizing a full-system simulator that has been validated against real hardware, we demonstrate that an LHCA design can provide a geometric mean 7% IPC improvement over a baseline 3-level SRAM cache design under the same area constraint across a collection of 25 workloads. A more aggressive RHCA-based design provides 12% IPC improvement over the baseline. Finally, a 2-layer 3D cache stack (3DHCA) of high density memory technology within the same chip footprint gives 18% IPC improvement over the baseline. Furthermore, up to 70% reduction in power consumption over a baseline SRAM-only design is achieved.
---
paper_title: Wear-Resistant Hybrid Cache Architecture with Phase Change Memory

paper_content:

Phase-change Random Access Memory (PRAM) is one of the most promising technologies among emerging non-volatile memory technologies, which provides many benefits, such as high density, non-volatility and low leakage power. However, the limited write endurance of PRAM prevents it from being used as a drop-in replacement of SRAM cache. Moreover, the inherent high latency and power dissipation of write operations are both hindrances that PRAM faces. In this paper, we study the L2 cache write operations incurred by different types of data, and accordingly, propose Wear-Resistant Hybrid Cache Architecture (WRHCA), in which the write access behavior of the hybrid L2 cache, that is composed of SRAM and PRAM, is optimized. Through the prediction of data access patterns, the proposed WRHCA prevents write-prone data from entering PRAM L2 cache, and consequently, the wear-out issue of PRAM is alleviated efficiently. Experimental results on the basis of the trace-driven simulator demonstrate that, as compared to the baseline system with pure PRAM L2 cache, our optimized WRHCA saved 85.5% write operations to PRAM on average, and boosted the performance by the averaged 6.4% CPI reduction. Last but not least, as compared with the primitive 3-level SRAM cache with the same chip area, our WRHCA achieved 60.9% reduction in terms of power consumption.
---
paper_title: TapeCache: a high density, energy efficient cache based on domain wall memory

paper_content:

Domain Wall Memory (DWM) is a recently developed spin-based memory technology in which several bits of data are densely packed into the domains of a ferromagnetic wire. DWM has shown great promise in enabling non-volatile memory with unprecedented density and high energy efficiency. In this work, we propose TapeCache, a first attempt to employ DWMs as last-level caches in general purpose computing platforms. DWMs enable much higher density compared to SRAM, DRAM, and other spin-based memory technologies such as STT-MRAM. However, they also pose unique challenges such as serial access to the bits stored in a DWM cell, leading to variable access latencies. We propose a novel circuit-architecture co-design for TapeCache, consisting of (i) a multi-port DWM macro-cell optimized for read operations considering the asymmetry in applications' read/write characteristics, and (ii) a new cache organization and suitable management policies that mitigate the performance penalty arising from serial access to bits in a macro-cell. Over a wide range of SPEC 2006 benchmarks, TapeCache achieves 7.8X improvement in area, an average energy improvement of 7.3X, and an average performance improvement of 1.2% compared to an iso-capacity SRAM cache. Compared to an iso-capacity STT-MRAM cache, TapeCache obtains 2.3X improvement in area and 1.4X average energy savings with virtually identical performance.
---

Title: A Survey Of Architectural Approaches for Managing Embedded DRAM and Non-Volatile On-Chip Caches

Section 1: INTRODUCTION
Description 1: In this section, introduce the scope of the survey and the motivation behind exploring different architectural approaches for managing embedded DRAM and non-volatile on-chip caches.

Section 2: A COMPARATIVE EVALUATION OF EMERGING MEMORY TECHNOLOGIES
Description 2: Provide a brief background on various emerging memory technologies, their features, and the challenges in their use, along with a comparative evaluation.

Section 3: CLASSIFICATION AND OVERVIEW
Description 3: Discuss the basics and essential ideas of architecture-level cache management and present a classification of the different techniques based on memory technology, application domain, levels of cache hierarchy, optimization goals, and essential approaches.

Section 4: CACHE MANAGEMENT APPROACHES
Description 4: Delve into detailed discussions on specific cache management techniques such as using error-correcting codes, minimizing refresh operations, reducing the number of writes, and utilizing hybrid caches.

Section 5: Using Error-Correcting Codes
Description 5: Explore techniques involving error-correcting codes to improve reliability and reduce refresh requirements in emerging memory technologies.

Section 6: Minimizing Refresh Operations
Description 6: Discuss various approaches that can be employed to minimize the refresh operations in eDRAM caches to enhance performance and reduce power consumption.

Section 7: Relaxing Non-Volatility of STT-RAM
Description 7: Provide insights into techniques that relax the retention time of STT-RAM to improve performance and energy efficiency.

Section 8: Minimizing or Avoiding Writes
Description 8: Discuss strategies to minimize or avoid write operations to non-volatile memories, thereby improving endurance and performance.

Section 9: Addressing High Write Latency of STT-RAM
Description 9: Explore solutions to mitigate the high write latency of STT-RAM caches which affects performance.

Section 10: Addressing 0/1 Write-Asymmetry
Description 10: Examine techniques that address the asymmetry in write operations (logical 1 to 0 transitions) to reduce latency and power consumption in STT-RAM caches.

Section 11: Improving Cache Lifetime by Wear-Leveling
Description 11: Discuss wear-leveling techniques aimed at distributing write traffic evenly to prolong the lifetime of non-volatile memory caches.

Section 12: Use in GPUs
Description 12: Investigate the use and impact of emerging memory technologies in GPU cache designs and their performance implications.

Section 13: Approaches Using RRAM
Description 13: Provide an overview of techniques specifically designed for managing RRAM-based caches and their advantages and limitations.

Section 14: Approaches Using PCM
Description 14: Discuss schemes to mitigate the limited write endurance and high write latency issues in PCM caches, and strategies to improve their viability for on-chip caches.

Section 15: Approaches Using DWM
Description 15: Examine architectural strategies and optimizations for managing DWM-based caches.

Section 16: HYBRID APPROACHES
Description 16: Discuss the design and benefits of hybrid caches that combine multiple memory technologies to leverage their combined strengths.

Section 17: CONCLUSION
Description 17: Summarize the key findings from the survey, the potential of emerging memory technologies, existing challenges, and future directions for research and development.