At Taiwan’s Linkou Chang Gung Memorial Hospital near Taoyuan Airport, heaps of data is being used in genomic analysis, with insights distributed to its 29 specialty centres that treat in excess of 10,000 local and international patients a year.
The hospital also shares this data with other hospitals and local medical providers in Taiwan. Thanks to the use of speedy all-flash storage arrays and the non-volatile memory express (NVMe) communications protocol, other healthcare providers can quickly leverage this data in their workflows to improve research and patient outcomes.
First released in 2011, NVMe speeds up the way microprocessors and solid-state drives (SSDs) communicate. It replaces the existing small computer system interface (SCSI) protocol and reduces the inefficient bottleneck typically associated with these older protocols, allowing for massively parallel, low-latency data paths in all-flash arrays.
This, in turn, offers enterprises significant performance advantages that support high-performance applications, as well as real-time processing within datacentres and at the edge, says Mark Jobbins, vice-president of technical services at Pure Storage Asia-Pacific and Japan.
In this buyer’s guide, we examine the benefits of NVMe, why you should care about it and what to look out for in the technology.
Massive parallelism
Traditional existing SCSI protocols place a message in a queue, a stack of commands which a device will execute. However, regardless of how advanced or expansive the network using the SCSI protocol is, every command is actioned, one at a time.
“NVMe replaces this inefficient bottleneck within the back end of all-flash arrays with massive parallelism – in fact, up to 64,000 queues and lockless connections that can provide each microprocessor core with dedicated queue access to each SSD,” says Jobbins.
This multi-parallel advantage, along with performance density, will help enterprises better tap advances in multicore microprocessors, super-dense SSDs, new memory technologies and high-speed interconnects, paving the way for adoption of emerging technologies such as artificial intelligence and automation.
Businesses can also leverage the speeds associated with NVMe to back up data in the event of a data failure. This allows companies to restore and protect data within the same compliance windows, despite constantly accumulating more data.
According to Kashish Karnick, product manager for storage at Lenovo Asia-Pacific, these technical advantages ultimately translate to the following business benefits:
Faster transaction processing: When transaction processing applications are subject to increased performance (those apps that can be measured as $/transaction), the faster process translates into more revenue for the customer.
Faster decisions: With decreasing response times, organisations can now attend to more queries in the same time frame than before. Decision-makers can cut their response time with quicker analysis.
Gaining another competitive edge: Faster software will result in customers having to spend less time waiting for their queries to be attended to. This faster app engagement leads to a better user experience for the customer.
Matthew Hurford, NetApp Asia-Pacific’s director of strategy and technology in the CTO office, says businesses will also see a decrease in total cost of ownership (TCO) due to licensing or exploiting existing shared fabric infrastructure in the enterprise.
“By modernising their architectures with NVMe, enterprises can accelerate not only workloads and analytics, but business outcomes through data-driven digital transformations,” he adds.
Adoption trends in APAC
Despite its benefits, NVMe is still in its early phases of adoption in Asia-Pacific, says Rajiv Ranjan, research manager at IDC Asia-Pacific’s storage team, noting that storage suppliers that have implemented NVMe mostly in high-end all-flash arrays are starting to do so for mid-range arrays as well.
“Some NVMe products may claim full NVMe compliance but may not do so. Users also need to check if they integrate well with their existing storage systems”
Rajiv Ranjan, IDC
“Limited awareness and higher cost are two key reasons for its limited adoption,” says Ranjan. “Since NVMe delivers very high performance and low latency, it is currently implemented in mission-critical or Tier 1 business applications across large enterprises.”
Lenovo’s Karnick says for the most part, more mature enterprises are taking their time to experiment with proofs-of-concepts and test the performance benefits of NVMe – at least when it comes to external storage.
“That said, NVMe adoption in Asia-Pacific for onboard storage has gained significant momentum – and given the non-disruptive deployment nature of NVMe solutions, it’s likely we’ll see increasingly rapid implementation across the region. It will all come down to the applications that can make the most use of this technology,” he adds.
Key applications
In the long run, applications that will benefit most likely from NVMe-based storage are those that use block-based storage, says NetApp’s Hurford.
“Performance-hungry – especially with respect to latency – and throughput-based applications will also benefit most from NVMe-based storage. Examples of these applications would be those that utilise NoSQL databases such as MongoDB, Apache Cassandra, Oracle and VMware.”
According to Huawei, online transaction processing (OLTP) applications, which are extremely sensitive to latency, will benefit the most from NVMe-based storage. These include payment processing and airline reservation systems where queries need to be completed within seconds and payments processed with near-instant verification.
“Good storage responsiveness can improve performance by supporting data processing workloads, so transactions are completed fast,” says Pure Storage’s Jobbins.
NVMe-based storage can also help the financial services industry in a number of ways – such as with real-time data analytics for risk management, fraud detection, transaction monitoring, as well as high-speed trading.
“With transactions and trades becoming increasingly automated, high-speed data processing has become the most important factor for accurate decision-making,” Jobbins says. “Investing in high-performance NVMe storage that’s matched with high-speed networks and servers can help traders shave milliseconds off their financial trades, which could translate to millions of dollars in profit.”
NVMe considerations
Enterprises often make the mistake of believing that all NVMe SSDs are equal and choosing suppliers based on cost. As NVMe SSDs differ in endurance, input/output consistency and quality of service, cheaper NVMe SSDs could mean lower endurance or performance.
“Doing enough research on the type and specification of NVMe SSDs is a must,” says IDC’s Ranjan. “With more storage arrays supporting NVMe SSDs, enterprises need to check what sort of management software is included, what level of NVMe support they are getting and how that will impact their workload performance.”
Deploying an NVMe storage product that isn’t fully compliant with NVMe standards could pose problems down the road. Ranjan advises enterprises to check if NVMe products comply with industry specifications and deliver the expected performance, endurance levels and cost savings.
“Some NVMe products may claim full NVMe compliance but may not do so. Users also need to check if they integrate well with their existing storage systems,” he says.
Before investing in NVMe, organisations should also perform a cost-benefit analysis. They need to understand the performance requirements of key workloads and whether it is justifiable to invest in a new technology like NVMe to meet those requirements.
“Create application workload profiles, analyse those profiles and use them with a workload simulation platform to evaluate the performance gains and determine if the cost is justified. If the cost appears to be too high, users should consider traditional all flash-arrays,” says Ranjan.
Read more about storage in APAC
New Zealand-based email hosting provider SMX needed to scale up its infrastructure to be able to handle 400,000 mailboxes, and found object storage from Scality fitted the bill.
Conventional all-flash arrays come with controllers that may not harness the full benefits of NVMe, and hence be unable to improve performance as expected. Instead, enterprises should opt for all-flash arrays that provide end-to-end NVMe connectivity.
This enables hosts to use the native NVMe protocol to talk directly to the NVMe SSDs and send native NVMe commands over Ethernet and FC networks that pass through the array’s controllers.
By 2021, IDC expects over 50% of primary storage revenue to be generated by NVMe-based storage platforms, with NVMe-oF in use on most of those systems.
Pure Storage’s Jobbins says with NVME-oF, datacentres that run applications and traditionally rely on direct-attached storage will see further consolidation. “Given faster network speed and broader bandwidth, these apps will be able to run on shared networked storage,” he says.
Read more on Storage fabric, switches and networks
