Flash-based solid-state drives nearly impossible to erase


Researchers from the University of California at San Diego delivered a paper at the FAST-11 Conference in San Jose, Calif., last week that shows it's almost impossible to reliably erase data from a solid state drive.

The tome, "Reliably Erasing Data from Flash-Based Solid State Drives" (PDF), goes through all of the known techniques for erasing data and comes up short in every case. The study's method is straightforward: They put repeating data on an SSD or USB drive, tried using various erasing techniques, took the SSD or USB drive apart, and pulled raw data off the chips. If any of the original data remained, erasing didn't work.

The culprit? SSD's so-called Flash Translation Layer, a firmware interface that makes an SSD appear to the PC like a big fat, uh, FAT device. Operating systems want to work with file allocation tables and clusters. SSDs have to deal with the vagaries of Flash media, which are quite different from rotating magnetic layers. For example, SSD blocks have to be erased before they can be written, and erasing takes a lot of time. FTL figures out how to erase unused blocks of memory when the SSD isn't doing anything else. SSD devices wear out faster if the same blocks are written and rewritten, so FTL balances the write load across all of the available memory.

You might imagine with all of these delayed erases running around and blocks of data being intentionally scattered to remote corners, there's some potential for error. Ends up, there's more than just a potential.

Here's what they found.

Erasing an entire SSD hard drive is difficult

When the researchers used the drive's built in "Erase Unit" command -- that's a legacy ATA-era command and it doesn't work on USB drives -- of the twelve SSDs tested, only four were completely erased. (One of the drives is automatically encrypted, so they couldn't tell if it was erased properly or not.) One of the drives reported it had been erased when, in fact, all of its data was immediately accessible. #FAIL.

Overwriting the whole disk fared a little bit better. With a hard drive, overwriting routines typically write all zeroes or all ones or some random character -- and then repeats the process a specific number of times. Out of eight drives tested using a similar method, one was completely erased after a single pass and two were completely erased after overwriting the data twice. One of the drives had 1 percent of the original data still on it after 20 writes. But half of the drives took more than 58 hours to perform a single pass. The researchers gave up.

Then the researchers tried degaussing, but it didn't work at all.

Encrypting drives passed muster, because deleting the key from the Key Storage Area makes the data useless. But the authors are less than enthusiastic about the approach because there's no way to verify that the manufacturer's key deleting mechanism completely obfuscates the contents of the Key Storage Area.

Erasing a single file is a horse of a different color

Single-file eradication generally relies on overwriting all of the storage space used by the file. As you might imagine, with the FTL running interference, it's going to be very hard to pick up all the pieces of a file.

When the researchers tried the usual approach -- 35 different variations on the file-overwriting theme, including several military spec algorithms -- they found single-file erasure just didn't work: "All single-ﬁle overwrite sanitization protocols failed: between 4 percent and 75 percent of the ﬁles' contents remained on the SATA SSDs. USB drives performed no better: between 0.57 percent and 84.9 percent of the data remained."

Bottom line: There's no way to reliably erase an entire SSD or a single file on an SSD or USB drive, although using an encrypted drive comes close. Some specific SSDs have full-drive erasing routines that work, but there's no way for you to know which ones.

The authors of the study conclude by presenting a new set of extensions to a baseline FTL that will make SSDs erasable.

Perhaps some day we'll see the recommendations applied to an SSD device. In the meantime, the only sure way to erase the data on an SSD or USB drive requires a very large hammer.

This story, "Flash-based solid-state drives nearly impossible to erase," was originally published at InfoWorld.com. Get the first word on what the important tech news really means with the InfoWorld Tech Watch blog. For the latest business technology news, follow InfoWorld.com on Twitter.