A peek into the future of storage

The flash era

As storage suppliers continue to refine their storage and data management software, including hyper-converged infrastructure software at the edge, they are also dealing with more data stored on flash-based storage systems.

“Flash is here to stay for the foreseeable future,” says Matt Swinbourne, chief technology officer for cloud architecture at NetApp Asia-Pacific.

“With the existing split in the market between performance and capacity flash, and with quad-level cells (QLC) currently entering the enterprise market, there is a rise of increasingly cost-effective storage platforms.

“At the performance end of the spectrum, the rise of persistent memory solutions continues, as does non-volatile memory express [NVMe]. We continue to see this market accelerate in two directions – one towards performance, the other towards economy.”

Meanwhile, flash storage suppliers will continue to innovate to deliver higher flash densities that enable users to start using all-flash arrays to optimise performance and capacity, says IDC’s Arora.

Flash suppliers are also working to improve the longevity of flash media by increasing flash endurance, which is defined by the number of times a system can write or erase data from flash media. Suppliers building all-flash arrays are writing firmware that can intelligently manage wear-levelling and intelligent block management. 

“Small random wears reduce the life expectancy of flash media more than larger sequential writes,” says Arora. “The former results in accelerated flash wear. The software enables the system vendor to ensure that data is written and erased uniformly across the flash media.”

Hard drives and tape will remain

Arora expects hard disk drives to remain in use for an extended period, especially in hybrid flash-disk arrays to optimise capacity. As the cost of hard disk drives remains lower than flash, they are ideal for secondary workloads or as a backup and archival platform that may not require fast access and retrieval.

“Enterprises still have a need for archives, secondary and tertiary copies of data, test and development environments, and so on,” says Paul Haverfield, chief technologist and presales manager for hybrid IT at Hewlett Packard Enterprise (HPE) in Asia-Pacific.

He adds that there is a large “sunk cost” in hard disk manufacturing as well as ongoing research and development, driving manufacturers to use this to their advantage for as long as possible and until the market speaks.

Lenovo’s Karnick expects that tipping point to come when the cost per terabyte of flash closes in on that of hard disks, which could eventually be phased out. He notes that manufacturers have already stopped making 15,000rpm (revolutions per minute) hard disk drives, and 10,000rpm drives will follow shortly.

“Although 7,200rpm drives are holding out for now, the reality is that the physics of spinning drives will, one day, be their undoing,” he says. “It will be interesting to see this future unfold and storage as we know it will be re-imagined.”

But what about tape? NetApp’s Swinbourne warns about the danger of predicting the end of a storage technology, noting that enterprises still have large investments in tape archives that require a significantly positive business case to change. 

That said, some industries are seeing the advantages of recalling data stored on tapes to convert it into an online or nearline solution, such as object storage, to create data lakes on which to build machine learning models or other analytics solutions.

“It is a long and expensive process, but companies that are able to build business cases are seeing massive upsides, such as avoiding unplanned downtime, optimising maintenance costs or having better medical insight for a research institution,” says Swinbourne. “But these companies would be the exception.”

What’s next?

For now, there is no clear successor or follow-on technology for the next solid-state storage technology, says HPE’s Haverfield.

“Beyond flash media, there are probably greater shifts in preparation around interfaces and protocols to enable us to achieve maximum performance out of the flash we have today,” he says. “We anticipate protocols like NVMe and interconnects like NVMe over Fabrics to be the next major disruption in storage architectures.”

Haverfield also expects a follow-on to NAND flash in the form of a next-generation solid-state storage technology – although exactly which one and when is not yet clear. “The solid-state storage industry is now developing storage class memory that delivers the persistence of NAND flash at DRAM speeds and write cycles, overcoming two of the main downsides to NAND,” he says.

Helium drives is another storage technology to watch. As they generate minimal friction, they could, in theory, solve one of the most pertinent pain points of every storage supplier in the world.

“But they’re also very expensive and are likely to be replaced by flash at some point down the line,” says Lenovo’s Karnick, although he believes helium drives will co-exist with flash storage in the interim while the large gap in cost per terabyte remains.

Matt Oostveen, chief technology officer at Pure Storage in Asia-Pacific and Japan, singles out the use of DNA to store very small packets of information – albeit at a speed 170 million times slower than it takes to write data onto flash storage.

“We’re also challenged by our ability to read data from this technology,” says Oostveen. “If you were to have your bitcoin information stored on a DNA liquid that you could wear on your neck – and it’s important, maybe during the apocalypse – who’s going to read that? You’ll probably need to take it to a laboratory, get it read and the response time could be a day or two before you get your information.

“So, there are lots of challenges that await us on the DNA front. While it’s fascinating and I do love the long-term nature of DNA storage, I’m not quite clear what it would look like.”