markrubens - Fotolia
Who needs hybrid cloud object storage?
Hybrid cloud object storage products are an emerging category. But what use cases are driving their emergence? And which suppliers lead the way?
Each time a consumer saves a document to Dropbox they are using cloud object storage. Dropbox was first built on top of Amazon Web Service’s (AWS) S3 storage architecture.
Meanwhile, Facebook users upload some 300 million pictures each day to the company’s bespoke object storage systems that can handle volumes beyond the reach of conventional, hierarchical storage architectures.
But if Facebook led the way in the mid-2000s with its Haystack architecture, a much wider range of businesses is now looking at object storage to handle ever-growing volumes of unstructured data.
Object storage – where data and metadata are stored together along with a unique identifier – brings the potential to store much greater volumes of information, as well as large files, efficiently.
And, because object storage is not reliant on a hierarchical file system and file paths, it is better able to distribute data across a range of storage devices and even physical locations.
“Object storage means you don’t have to worry about where data is stored, and you can distribute it widely,” says Gigaom principal analyst Jon Collins.
Hybrid cloud is go-anywhere storage
Businesses can, and do, use object storage in their datacentres and as local storage, to handle larger volumes of larger files.
Organisations in the media, research and development, scientific research, medicine and government use object storage, especially for media and imaging data. Here, the benefits are mostly around availability and resilience, and of course scalability.
Openstack Swift, for example, is used to store genetic research data, as well as data files for mass-market online games.
And object storage is also being pushed forward by its use in the cloud.
Read more about hybrid cloud object storage
- A single environment across on-premise and cloud environments is possible with a new class of product that builds file systems and object stores with hybrid cloud.
- This year’s storage news so far has provided a firm impression of the increasing prominence of the cloud, and in particular of attempts to harness the public cloud and private datacentre in hybrid operations.
Amazon’s cloud storage architectures, including S3 and Glacier, are object-based storage systems.
AWS’s S3 architecture is so widely used that it has become a de facto standard for large-scale, cloud-based data stores. Cloud-based application suppliers increasingly write their software so it works with S3, and S3 is being used to build services such as cloud-based backup and recovery. Nor is S3, and object storage, limited to public clouds – private clouds use it too.
But to see object storage as either an architecture for the datacentre, or a building block for cloud applications, is to miss much of its potential.
As object storage is hardware-independent, and therefore location-independent, as well as scalable – it can cover both use cases. Vitally, it allows data to move between on-premise and cloud-based storage, and makes it easier to share computing workloads between the two, so is potentially ideal for hybrid cloud.
Hybrid cloud dreams
One of the challenges for developers of early cloud computing was data storage.
The conventional, hierarchical systems used for local storage become increasingly inefficient at scale. File system paths also force developers to work with fixed links between applications and storage.
One of the benefits of the cloud is being able to aggregate resources, and scale capacity up or down on demand. This works well enough with compute resources, but until developers adopted object-based storage, it was a challenge for data.
With object storage, it’s possible to design architectures that are hardware- and location-agnostic. In turn, this allows businesses to run the same workloads locally, in private and in public clouds.
With hybrid cloud object storage, there is a greater chance of fine-tuning the architecture to meet performance and budget requirements, as well as other constraints such as data location and security.
Businesses can, for example, use local resources for processing-intensive tasks, but tap into the cloud for lower-cost longer-term storage or for additional compute capacity.
Movie studios, for example, use cloud computing for rendering, especially overnight when cloud charges for compute instances are generally lowest.
But for that to work in practice, the cloud-based compute instance needs to be able to read and write to local, or private cloud, storage. Object storage can do this, because it uses a single namespace for data.
Furthermore, hybrid object storage isn’t restricted to scenarios where organisations want data to span on-premise, private and public clouds. Multi-cloud systems are increasingly common.
Object storage supplier Scality calculates that most enterprises already use four or more clouds. Object storage allows application designers to put data and workloads on whichever cloud system offers the best features, performance and cost – and to move them again, if the need arises.
To do this, the object storage system needs to keep the data in the application’s native format, but that is possible with today’s technology. Bloomberg Media, for example, uses hybrid and multi-cloud resources to store its vast quantities of video and other files.
As Enterprise Strategy Group senior analyst Scott Sinclair puts it: “With hybrid cloud object storage you can extend on-premise object storage into the public cloud, as essentially a single massive storage pool.”
Ready to deploy?
Organisations looking at hybrid cloud object storage are likely to be those that already store vast amounts of information, or that expect their data storage needs to grow rapidly.
The industries at the forefront of the technology’s adoption have largely been those in social media, gaming, video and film production, and scientific research because they handle large volumes of large files. Usually, they already have substantial experience of object-based storage, in-house, in the cloud, or both.
But for any organisation looking to move to hybrid cloud object storage, the starting point should be the data. Suppliers increasingly offer powerful services and tools that can move data across all physical storage categories, and orchestrate them in a way to maximise efficiency and performance. Only the business, though, can decide if a data store suits their own resilience, security and privacy criteria.
Hybrid cloud object storage suppliers
Caringo
Texas-based Caringo offers Swarm ‘scale-out hybrid storage’. Swarm operates as software, hybrid cloud (on Microsoft’s Azure platform) and as integrated hardware as Swarm servers.
DataDirect Networks (DDN)
DataDirect Networks provides WOS, an object storage architecture that supports S3, Swift and REST application programming interfaces. The company provides hardware and software-only systems for use with industry-standard equipment. The company is 15 years old and has a strong presence in financial services and manufacturing, as well as cloud services.
Dell EMC
Dell EMC is the enterprise and storage arm of Dell Technologies. Dell’s takeover of EMC in 2015 was the industry’s largest at the time. Dell EMC offers software-defined object storage under its Elastic Cloud Storage brand, but also offers appliance and software-only object storage systems. It claims a significantly lower total cost of ownership (TCO) than standard public cloud.
Hitachi GST
HGST, now part of Western Digital, brands its object storage system as ActiveScale. This is provided as a turnkey solution, and is focused on large capacity and low TCO operations. HGST systems come with an S3 connector, so they can be combined with cloud-based storage for a hybrid system.
HPE
HPE has a number of object storage systems under its Scaleable Object Storage label. This includes a partnership with Scality to offer its RING hybrid cloud storage architecture, as a software-based system. HPE supports connectors to AWS and Microsoft Azure.
IBM
IBM offers a cloud object storage architecture via RESTful APIs. This allows applications to connect directly to object storage systems, and IBM cloud services, as well as operating across disparate datacentres. There are also S3 connections, as well as specialist services for collaboration, file sharing and archiving.
NetApp
NetApp’s StorageGRID is its ‘web-scale’ object storage offering, available as an appliance-based system or as software. It can also deploy on VMware, OpenStack and Docker.
Scality
Scality RING is a software-defined object storage system that works on industry-standard hardware, and includes S3-compatible APIs. The technology supports S3 for cloud, NFS for local volumes, and a connector for Azure Blob Storage. Scality has also developed Zenko, an open source multi-cloud controller, for businesses that want to span storage across cloud providers.
