RAID, flash and erasure coding: What works best with solid state?

RAID levels

RAID systems protect data in two main ways – either by mirroring a physical disk in its entirety, or by storing parity data on one or more disks that is used to rebuild a failed drive.

System designers that build RAID arrays need to balance performance, resiliency and the capacity overhead of either the mirror or the parity data. At its simplest, RAID 1, or mirroring, simultaneously copies all data to two or three drives. The capacity overhead – the amount of additional storage needed for the mirrors – is therefore either 100% or 200% of the usable storage.

Other RAID levels aim to reduce the storage overhead and therefore the cost without an undue performance penalty. RAID 1 provides the best performance and, critically, the fastest rebuild times with no need to recreate data using parity bits.

RAID 4 uses block-level striping, and stores parity data on a single drive. RAID 5 uses striping with distributed parity without the need for a dedicated parity disk. RAID 6 uses dual distributed parity for additional data protection. RAID 10 combines mirroring and striping.

RAID levels 2 and 3 are rarely used today in enterprise systems. A number of suppliers now have proprietary RAID systems, with the goal of reducing the capacity overhead still further than RAID 5 or RAID 6.

Is RAID relevant for flash storage?

In performance terms alone, RAID is not an obvious choice for flash-based storage. Single media performance easily beats RAID in almost all common enterprise scenarios.

“SAS SSDs using NAND flash offer an order of magnitude lower latency and one to two orders of magnitude higher throughput, depending on whether we’re talking about random or sequential reads,” says Eric Burgener, research vice-president in IDC’s enterprise infrastructure practice.

But in terms of cost, flash media is still significantly more expensive than spinning disk.

Leaving aside flash media’s generally lower capacity per drive, which necessitates the use of larger arrays, SSDs are seven to eight times more expensive than equivalent spinning disks.

Although this does not rule out RAID 1 or RAID 10 for flash-based arrays, it will limit it to applications with little tolerance for data loss, and a need for short restore times.

“RAID 1 provides the best performance if you’re using it locally – you only have two writes, or three if you’re running three mirrors,” says Burgener. “It provides the least impact in recovery mode, as you only have to read from one device and don’t have to perform any computation to ‘reconstruct’ the data. But it’s the most expensive.”

RAID 5 and 6: A practical option for flash

Those high costs prompted researchers to develop RAID 4, 5 and 6. The technologies are now mature because storage system suppliers cut their teeth when spinning disks were smaller and costlier than they are today. Much of that work translates well to the flash storage world.

“This high cost of redundancy is one of the reasons RAID levels like RAID 4, 5 or 6 or erasure coding approaches were developed,” says Burgener. “They offer a much lower capacity overhead, but they tend to have more of an impact in recovery mode, as data must be read from multiple devices and computation must be performed on that data using one or more parity stripes.”

However, these disadvantages are overcome, to an extent, by flash storage’s higher performance, especially in reads. The downside remains that a significant percentage of expensive flash storage is set aside for the parity data. In a RAID 6 system, with a standard layout of 4+2 (four data bits and two parity bits), the overhead is one-third of the storage.

For mainstream systems, and where data storage needs are relatively modest – up to around 20TB – RAID 6 is likely to suffice. Most IT architects calculate that the greater resilience of RAID 6 is worth the cost increase or capacity penalty over RAID 5. RAID 10 is also an option, typically for the most latency-sensitive applications and for high availability.

Data protection for all-flash arrays and the cloud

Even so, suppliers are developing new forms of data protection that are better suited to flash-based storage, by cutting the capacity overhead. These systems also exploit the smaller average drive size for flash, to distribute parity data across more volumes.

Vendors of all-flash arrays have largely offered RAID 6 to date, but a number of companies have challenged this with new technologies.

Kaminario, for example, uses a 22+2 format for its all-flash arrays. This cuts the overhead, but only works with arrays with 24 or more drives.

VAST Data combines Intel Optane SSDs and QLC SSDs and uses a very economical 150+4 design, which works out at an overhead of around 2%. But the minimum system capacity is 1PB.

Erasure coding

Adoption of erasure coding has been driven by the large cloud platforms, but it is becoming more common in hybrid and on-premise object storage. Erasure coding is likely to be more common, not least because it gives enterprises more control over the level of protection they want, as well as physical redundancy.

The downside of erasure coding has largely been performance, although with all-flash storage performance, that gap has narrowed.

Data decay: Different architectures

Flash systems do, however, have some key technical differences that affect the design of RAID schemes.

Flash storage deteriorates the more it is used, and that is more the case with writes than reads. Newer but cheaper flash media, such as quad-level cell (QLC), has a shorter write life than previous generations of NAND flash. But enterprise applications can be write-intensive, so hardware designers have to create techniques to minimise the writes and extend the system’s working life.

This is not something that happens with spinning disk, so IT departments cannot simply “lift and shift” workloads from conventional to flash arrays and expect the same levels of protection.

“When designing data protection algorithms for flash-based data, you have some new inputs – much lower latency and higher throughput, but much lower endurance,” says Burgener. “So schemes that minimise writes are more interesting.”