Solid-state storage finds its niche




Still extremely expensive vs. traditional media, solid-state disks can make a convincing economical case for some applications requiring very high IOPS.

With the cost of hard disk storage falling to $0.50 per gigabyte or less, the International Securities Exchange (ISE) in New York City decided to buy a 128GB storage system that lists for approximately $85,000 or $664/GB. While that sounds shocking, the company considers it a smart move.

"We have a very latency-sensitive application [that handles stock trading], and the storage gives us a distinct competitive advantage," says John Ryan, ISE's technology architecture officer. The solid-state disk (SSD) system from Texas Memory Systems Inc. handles reads and writes in 0.02 milliseconds (20 microseconds), which is orders of magnitude better than the fastest hard disk drives (HDDs).

The SSD system that ISE selected is based on double data rate (DDR) DRAM technology and comes integrated with battery backup, a Fibre Channel (FC) interface and conventional hard drives. This SSD technology has been around for years and has established itself in a niche market that serves large processing-intensive government projects and companies involved in high-volume, high-speed/low-latency transactions such as ISE's stock trading system.

Organizations use SSD as large cache to boost performance by pulling data from what amounts to memory, rather than from much slower spinning disk. The SSD industry regularly takes advantage of the falling prices of memory and storage, which allows organizations like ISE to purchase 128GB of SSD storage at a price that's high but justifiable. However, even SSD proponents admit that DRAM-based SSD systems are far too expensive for use in terabyte-sized systems.



Many SSD options
But there are other SSD options. The rapid rise of NAND flash-memory technology promises to make SSD a viable storage option in the mainstream corporate IT storage environment at some point. Because flash is nonvolatile, it's suitable for long-term data storage; in addition, systems built on flash technology follow an entirely different price curve because they don't have to incorporate the power protection and battery backup required by volatile DRAM-based SSD (see "The many flavors of SSD," below).

As flash storage becomes more feasible--thanks to the benefits of Moore's Law and the SSD industry's improving economies of scale--even companies that don't calculate their competitive advantage in microseconds may turn to SSD in the not-so-distant future. The advantages of NAND flash storage in terms of high performance, low energy usage and reliability may eventually offset SSD's high cost per gigabyte.


The many flavors of SSD
The solid-state disk (SSD) market is rapidly evolving into an alphabet soup of acronyms as companies try to find a nonvolatile storage technology--ferroelectric memories, magnetic memories, phase-change memory, polymer memories, nanotechnology memories, resistivity change memories, micromechanical probe memories, one-time programmable ROM memories--that delivers the high performance of double data rate with the low cost of hard disk drives, but without the wear-out problems of flash. In addition to NOR and NAND these include: The problem with these technologies is that they're years away, if ever, from mainstream, large-scale storage production use. "They are unproven, won't scale or can't achieve the density that NAND will have in, say, three years when they might be widely available," says Alan Niebel, founder and CEO, Web-Feet Research Inc., Monterey, CA. "At best, some may replace NOR flash in cell phones or something like that."

"Initially," adds Tom Coughlin, president of Coughlin Associates, Atascadero, CA, "you will see these [technologies] show up in the military or some highly specialized industrial niche."



With fundamental technology changes in store for HDD as the industry struggles to increase areal density (the amount of data that can be packed onto the disk) beyond what can be achieved with the latest perpendicular disk technology, disk drive vendors find themselves looking at SSD as a potential competitor.

"We've seen a growing focus on flash at HDD companies in the last six to nine months," says Joel Weiss, president, International Disk Drive, Equipment and Materials Association (IDEMA). For example, Seagate Technology LLC, an HDD leader, is already deep into flash and hybrid technologies (see "Hybrid storage," below). As a sign of change, IDEMA has opened its membership to flash manufacturers.


hybrid storage
Hybrid storage combines flash with hard disk drives (HDDs). The first devices add 256MB of flash to a conventional disk drive. "The flash sits between the system's RAM and hard drive," explains Josh Tinker, market development manager, personal computer business unit at Seagate Technology LLC. The benefits of hybrid storage are minimal until Microsoft Windows Vista enters the picture.

"Vista has SuperFetch, which manages the memory architecture to take advantage of the flash," says Tinker. Having 256MB of nonvolatile memory enables a drive manufacturer to do more in terms of energy reduction and reliability. For example, it can stop spinning the disk for periods of time and rely completely on the flash portion, which saves energy and wear and tear on the drive.

At this point, Seagate's hybrid drive will show up first in high-end laptops (running Vista), where conserving battery is a big concern. However, it demonstrates what might be done when enterprises combine solid-state disk and HDD storage.

Flash advantages
Flash is nonvolatile storage, which means the data remains even if the power cuts out. Everybody has seen flash, especially NAND flash. It's the storage behind Apple's iPod nano and is widely used in portable USB storage devices. Although flash storage comes in NOR and NAND flavors, this article refers only to the NAND version of flash. "When people talk about flash memory today, they are referring to NAND flash," says Krishna Chander, senior analyst, storage systems at iSuppli Corp., a research firm in El Segundo, CA. The terms NOR and NAND refer to the logical electronic gates (OR and AND) that make up the memory chip. NOR means not OR and NAND means not AND.



"NOR is too costly and too slow at writing," adds Alan Niebel, founder and CEO, Web-Feet Research Inc., Monterey, CA. It's not considered a viable enterprise storage technology.

The main advantages of flash storage are high performance, reliability, low power consumption and small size. Flash-drive performance is 1,000 times better than hard disk. Flash's sturdiness gives the technology another advantage over HDD. The military is one of the earliest adopters of flash storage for ruggedized PCs used in the field and in military aircraft. "They put it inside the computer in a jet fighter, which vibrates like hell. A hard disk drive can't take that kind of shock," says Jim Handy, director at Objective Analysis, an analyst firm in Los Gatos, CA.

For corporate IT, ruggedization isn't usually an issue, but reliability is. The big appeal of SSD, both flash and DRAM, is extremely high performance. "This is where you have a huge volume of transactions and need fast IO," says Jeff Janukowicz, research manager for hard disk drive components and solid-state disk drives at IDC, Framingham, MA. Of the two, flash delivers somewhat less performance and costs less, but it's still far faster than even the fastest 15K rpm hard disk.

Another appealing aspect of flash storage is its lower power consumption and low heat. "Power, heat and cooling have become huge issues, and flash SSD can cut down [on] power usage in the data center," says Niebel. Flash memory has no moving parts and requires negligible amounts--in the area of 10 milliamperes (mA)--of power for reads and writes.

Flash drawbacks
Flash SSD has its drawbacks, beginning with its high price. Flash proponents cite a current price of $10/GB. "In the past, the cost ratio of flash SSD to hard disk was 100:1. That has come down to 40:1 and will drop even further," says Niebel. But the $10/GB price is deceiving, as it's the raw cost of low-quality flash chips to the OEM. IT organizations will likely want higher quality flash systems delivered by OEMs, and these systems currently run about $60/GB or more. The question is how much of a premium IT managers are willing to pay for the advantages of flash storage. If only a few gigabytes are needed for an extremely high-performance cache, they might be able to justify the premium, as ISE did.

Related to its high price is the limited capacity flash SSD systems offer. Most systems are configured for 16GB, 32GB, maybe even 64GB, although STEC Inc., a Santa Ana, CA-based SSD system vendor, offers flash storage up to 256GB. "Price is an issue in the enterprise. If you need large capacity, a hard drive is way cheaper," says iSuppli's Chander.



Another drawback is long-term reliability or "wear out." Flash SSD may be nonvolatile, but individual flash-memory cells have a limited usage life. "You don't want to hammer one cell too much. If you can avoid writing to a particular cell for days or weeks, you can stretch the life of the cell," explains Niebel. Currently, a flash cell has a write-cycle life of 100,000 writes. That may seem like a lot of writes until you start thinking about a high-volume transaction system.

Flash SSD systems try to overcome this problem by building wear-leveling algorithms into the controllers. These algorithms spread the writes across different cells so no cell gets written to excessively. By combining wear-leveling with RAID-like striping, proponents argue that they can push the effective life of a flash SSD system beyond 100,000 writes.

Flash is delivered in two formats: single-level cell (SLC) and multi-level cell (MLC). The wear-out and cost problem may be partly addressed through a combination of SLC and MLC flash. MLC, which layers more data on each cell, is cheaper than SLC. OEMs can use it to add capacity to SSD systems. However, MLC has a much shorter life than SLC. Because it's the write process and not the read process that shortens cell life, the trick for OEMs becomes "trying to figure what portion needs to be read-only and make that portion out of MLC. They can use SLC for the rest," says Chander. OEMs will have to build intelligence into the controller to direct predominantly read-only data to the MLC cells in a SLC/ MLC SSD storage device.

In addition to write wear out, there's another concern with writes. "NAND flash has a slower write speed," explains IDC's Janukowicz. That and the wear-out problem suggest flash SSD is best suited for applications that write once or occasionally but read many times.

SSD economics
The economics of SSD will force storage managers to think differently about storage pricing. In the past, storage managers focused on the cost of capacity as measured in cost per gigabyte (cost/GB). With SSD, the big benefit is performance and throughput. Instead of cost/GB, SSD proponents talk about IO operations per second (IOPS).

While HDD has an overwhelming advantage in terms of cost/GB, the situation is reversed when it comes to cost per IOPS. But the problem goes beyond comparing one HDD system with one SSD system. "There are several things going on here. First, the hard disk is not keeping up with the performance of the server," says Alan Fitzgerald, CTO at Adtron Corp., a Phoenix-based SSD manufacturer.



Because servers can consume stored data faster than hard disks can provide it, companies are driven to add more disks to boost performance. They may not actually need this additional disk capacity, but more spindles increase the throughput of the system.

"You end up buying more hard drives to get the same level of IOPS as one SSD, and those drives require more space, power and cooling," says Janukowicz.

For example, STEC compared HDD and SSD in a high IOPS application. It found that a company would need 450 73GB disk drives to generate the same level of IOPS as 15 146GB SSD drives.

The cost for those 450 hard drives, according to STEC, would be $175,000. STEC pegged the cost of the 15 SSD drives at $187,500. But if you throw in the cost of the multiple enclosures, racks, cables, transceivers, controllers and SAN switches necessary to house and connect all of those hard drives, the total HDD cost comes to almost $413,000 in STEC's calculation. The total SSD cost--which requires only one enclosure, rack, cable, transceiver, controller and switch--comes to a bit more than $201,000. In STEC's analysis, the SSD solution costs half that of the HDD solution. Of course, the HDD solution delivers substantially more storage capacity in the process.

Texas Memory Systems is also targeting the enterprise storage market with its RamSan-500 SSD device, which delivers 2TB of SSD storage with two FC controllers for $300,000. It boasts 100,000 IOPS, but its best performance comes from its 16GB of very high-performance DDR cache. The rest of the 2TB capacity consists of less-costly flash SSD.

The issue actually goes beyond storage. Organizations are buying servers that are poorly utilized in part because the storage can't feed data to the server fast enough.

"This results from the latency of rotating disk vs. the speed of the server's processor," says Janukowicz. "With SSD, the latency issue goes away. Where you might have used two servers [each running at a low level of utilization], you would need only one." With SSD, companies could reduce the number of servers they need in certain situations.



New storage tier
The emergence of SSD as a viable enterprise storage option leads analysts like IDC's Janukowicz to speak of a new storage tier, Tier 0. Tier 0 would sit above primary enterprise storage (Tier 1) and be used for those apps requiring the fastest possible performance and the lowest latency. It's very early in the evolution of enterprise SSD; few products for mainstream corporate storage are shipping and few of the early adopters are willing to speak publicly. Still, some uses for Tier 0 are becoming apparent.

"Right now, SSD is too expensive for corporate data, but it looks like it will be great for meta data, which is read frequently," says Mike Karp, senior analyst at Enterprise Management Associates, Boulder, CO. Similarly, it would be good for database indexes.

Flash also looks promising for media, where applications are written once but read many times. Another recommended use puts flash in the server for the boot OS, which would be much faster. Yet another alternative to SSD is to use memory as a caching appliance (see "SSD performance alternative," below).


SSD performance alternative
If you want high performance and don't care about data storage (persistence), you can look at a DRAM caching appliance, such as the one provided by Gear6 in Mountain View, CA. The Gear6 appliance sits next to primary storage and promises to deliver 250,000 IOPS for $400,000 ($1.6/IOPS) for hundreds of servers in parallel, reducing the need to buy extra hard disk drive spindles but providing no actual storage. By contrast, Texas Memory Systems Inc.'s RamSan-500, which provides data storage and caching, costs $3/IOPS (100,000 IOPS for $300,000).

Flash SSD is showing up in blade servers. This summer IBM Corp. announced it was adding a 4GB USB flash drive to its BladeCenter HS21 in an effort to remove components, such as a hard disk, that are more likely to fail and result in downtime. By using SSD, IBM reports a reduction in power consumption by as much as 18 watts per blade. In addition to conserving power, the flash drives boost performance with an average seek time of 0.1 milliseconds vs. 15 milliseconds for a conventional hard disk.



Disk futures
The storage industry is moving into a period of uncertainty. After averaging 100% annual increases in hard disk areal density, the rate has dropped to 40% annually, says Janukowicz. The HDD industry is now looking beyond perpendicular disk designs to heat-assisted and bit-patterned approaches. These nascent technologies promise to boost HDD areal density to the multiterabyte range; however, they involve new processes, designs and tooling that will change the cost dynamics of the industry.

Meanwhile, the SSD industry continues to ride Moore's Law to 100% a year improvements for flash, notes Janukowicz. The result is a steadily narrowing capacity gap that will reduce the cost differential between HDD and SSD over time. Yet IDC doesn't foresee a crossover point where SSD becomes cheaper than HDD on a cost-per-gigabyte basis.

For storage managers facing critical storage performance problems and where maximum IOPS counts, the emergence of SSD as a feasible enterprise option is encouraging. For everyone else, the HDD-SSD issue remains an interesting sideshow that bears watching.



This was first published in November 2007

Dig deeper on Solid-State Storage






Forgot Password?


Your password has been sent to: