Cloud flash storage: SSD options from AWS, Azure and GCP
We look at the flash storage options available from the big three cloud providers – Amazon Web Services, Microsoft Azure and Google Cloud Platform – that can help narrow the gap between on-site and cloud workloads
Flash storage is increasingly important to the enterprise market, and analysts expect solid-state drives (SSDs) to largely replace spinning disks for enterprise systems, endpoints and consumer devices – possibly within this decade.
Meanwhile, some predict spinning disk will be used mostly by “superscalers” – the likes of Google and Facebook – and the cloud providers.
The latter – centred on the big three Amazon Web Services (AWS), Microsoft Azure and Google Cloud Platform (GCP) – are investing heavily in solid-state storage as well.
Buyers of the longest-established cloud storage applications, such as backup, recovery and archiving, are largely agnostic about the underlying hardware they use. Key metrics there are cost per gigabyte per month, availability and, for applications that hold sensitive data, geography.
For the cloud providers, scale matters, and drives demand for ever larger hard drives, with 50TB on supplier roadmaps.
But other cloud applications demand performance that hard drives cannot offer. The answer lies in flash storage.
With cloud suppliers chasing the market for higher-performance applications, including production enterprise systems, flash storage is increasingly important for high-performance object storage.
Object storage, though, is not alone. The big three cloud suppliers now offer flash technology for block and file too, for compute workloads and virtual machines (VMs).
AWS, for example, introduced SSD as the default option for its Elastic Block Store (EBS) general-purpose volumes back in 2014. The cloud provider targeted SSD storage at “small to medium-sized databases, development and test environments, and boot volumes”. The other large cloud suppliers have similar offerings.
The aim is to narrow the gap between storage performance in the cloud and in the datacentre.
Moving to flash storage in the cloud improves the performance of cloud-based compute instances, and that’s one reason the suppliers offer premium tiers for storage-intensive applications such as databases.
Flash storage bolsters the performance of virtual machines, and should make it easier to transfer workloads between on-premise systems and the cloud, with reduced – or at least predictable – performance penalties.
As ever, the capabilities depend on the architecture IT teams choose, and their choice of cloud platform.
The three largest cloud suppliers – AWS, Microsoft Azure and GCP – all have a range of flash storage options that mix cost, capacity and performance.
Amazon Web Services
AWS provides SSD storage across its block storage EBS, GP2 and IO1 volumes, as well as file storage FSx Windows and FSx Lustre.
EBS: EBS solid-state drives are available as general-purpose (GP2) and provisioned IOPS SSD (IO1). For GP2, capacity ranges from 1GB to 16GB, maximum input/output operations per second (IOPS) per volume is 16,000 and maximum throughput is 2,375MBps. For IO1, capacity is 4GB to 16GB and IOPS is 64,000.
FSx: FSx For Windows is an SMB file share, based on Windows Server, although it can connect to Windows, Linux or MacOS clients. Available as SSD or HDD, AWS claims “sub-millisecond” latency for the SSD variants. Each file system supports up to 64TB of data, and users can specify throughput from 8MBps to 2,048MBps. FSx for Lustre supports three tiers of persistent storage: Persistent-50, Persistent-100 and Persistent-200, with a baseline throughput of 250, 500 and 750MBps per terabyte of storage, respectively. This is based around 10,000 IOPS. AWS claims FSx Lustre operates up to petabyte scale.
Microsoft Azure
Azure provides Azure Managed Disks as its block-level storage option for Azure VMs. As with AWS, there are solid-state and magnetic options. Microsoft also offers NetApp files, premium files and storage accounts.
Azure Managed Disks: The top tier of storage is for input/output (I/O)-intensive workloads, including SQL and Oracle databases and SAP Hana. Maximum disk size is 65,536GB, IOPS of 160,000 and throughput at 2,000MBps. Premium SSD, recommended for production workloads, has a 32,767GB maximum disk size, 900MBps maximum throughput and 20,000 max IOPS. For web servers, test and development, and lightly used applications, Azure suggests its standard SSD, with specs of 32,767GB, 750MBps throughput and 6,000 IOPS.
Azure Premium Files: This is a high-performance, SSD-based file share. Users specify storage using a baseline GB/IOPS/throughput ratio, although Microsoft points out that this is also determined by the file system. The base tier is 100GB, baseline IOPS of 100 and burst up to 300 IOPS. This rises in steps to 102,400GB, and 100,000 IOPS.
Azure NetApp Files: This account type also provides Ultra, Premium and Standard SSDs, compatible with any POSIX-compliant workload.
Azure Storage Accounts: Storage accounts support blob, file, queue, table and disk. General-purpose V2 and V1 support standard and premium storage tiers, BlockBlob Storage and FileStorage are premium only, and BlobStorage is standard only.
Google Cloud Platform
Google’s cloud provides its premium Local SSD storage for virtual machine instances and Persistent Disk for less demanding workloads.
Local SSD: Local SSDs are directly attached to the VM server used by Google’s Compute Engine. Capacity is either 6TB or 9TB, made of up to 24 local SSD partitions. Google’s Local SSD is its premium storage tier, with persistent disk the lower-cost, lower-performance option. Performance ranges to 2,400,000 IOPS read and 1,200,000 IOPS write with read throughput up to 9,360MBps and write throughput of 4,680MBps (for 9TB instances). Local SSD comes in two versions: SCSI and non-volatile memory express (NVMe), with the highest performance available on NVMe. Google recommends Local SSD for flash-optimised databases, “hot caching” layers for analytics applications, and application scratch disks.
Persistent Disk: Google’s Persistent Disk platform supports flash and conventional hard disk volumes, up to 64TB. Instances can span up to 257TB, far higher than Local SSD. On costs, provisioned hard drive space is US$0.04 per gigabyte per month and SSD provisioned space is US$0.17 per gigabyte per month. Read IOPS ranges from 15,000 to 100,000 per instance and 15,000 to 30,000 for writes. Google recommends its persistent disk, block storage for virtual machines and databases.
Price, performance and future-proofing
Cloud providers look set to continue to offer flash and spinning disk storage for the foreseeable future.
The lower costs of hard disk storage continue to make sense in a large-scale cloud environment, and the cloud infrastructure providers are able to deal with the disadvantages of 20TB-plus drives, such as long rebuild times.
But flash-based offerings will continue to grow, to support production-grade VMs, file-based storage and more demanding applications such as analytics and artificial intelligence (AI).
The exact mix of flash options, or flash and hard disk drive (HDD) options, will depend on the workload, pricing and the cloud provider’s storage tiers. Specifying flash storage on-premise and in the cloud will maximise performance and compatibility, and make it easier to tier storage to the cloud – an option that more storage and application suppliers are working towards.
Read more about cloud storage
- Cloud storage 101: We run the rule over file, block and object storage from the big three public cloud providers: AWS, Azure and GCP. We look at what’s on offer and the use cases they are aimed at.
- Cloud storage on-site hardware: We look at the big three cloud suppliers’ on-site offerings: AWS’s Outposts, Gateway and Snow hardware; Azure’s Stack and Arc appliances; and Google Cloud’s software-defined Anthos.
