Unified file and object storage: The best of both worlds?

File and object: Kissing cousins

File storage has been the bedrock of computing systems from the outset, with files ordered into a directory structure accessible directly by the operating system and applications. File storage provides good read-write performance, but becomes hard to manage at scale.

Object storage, meanwhile, uses rich metadata to organise objects. Object storage systems work independently of location and hardware, and can scale without limit. As a consequence, object is the main building block for cloud storage and is finding its way into large-scale on-premises data projects in the enterprise too.

“The difference is between accessing unstructured data over the stateless internet in the form of objects, versus accessing structured – file – data over Ethernet,” says Alex McDonald, EMEA chair of the Storage Network Industry Association.

But as McDonald notes, the underlying physical storage does not change according to the access protocols. It still comprises bytes on a drive. So, if vendors can expose a pool of storage to multiple access protocols, enterprises will gain flexibility in how they store data and how they design applications.

Much of the vendor work in the space aims at combining the benefits of file and object and overcoming the limitations of each.

“File systems are a key part of all computing systems, and remain the most common way that applications and operating systems store data,” says Steven Hill, senior analyst for applied infrastructure and storage technologies, at 451 Research.

“File provides a simple, tree-based directory model. However, the filename, directory tree and the few limited attributes offered by file systems don’t really provide much insight into the data’s contents. Object storage provides a different abstraction model for data that supports a broad range of customisable metadata.”

Hill notes that this powerful metadata provides: “Granular automation of security, protection and lifecycle management of unstructured data.” But object storage is not directly accessible to conventional applications.

Enterprises either need to re-factor applications to work with object storage, or use a gateway that translates between file and object, and which creates a potential bottleneck.

File and object: More than the sum of the parts

Combining file and object protocols into one storage system offers the benefits of efficiency, scale and hardware independence that come from object storage with the operating system, application and device compatibility of file access.

Could unified file and object storage usher in a new age of flexible, scalable and high performance universal storage?

“There’s really nothing that goes against the idea,” says SNIA’s McDonald. “In my opinion blocks, files and objects have been a fairly artificial, if useful, way of thinking about streams and accessing data. We’re getting more accomplished at moving away from how it’s stored – blocks on disk for instance – towards what we might want to do with the data.”

Enterprises, though, will make their own decisions about the speed of adoption. Systems that combine file and object storage are likely to be more expensive and complex than either cloud storage or simple NAS architectures alone. 

Unified file and object storage is nowhere near becoming a universal storage format, however.

Although it is possible to run object, file and block on one system the performance penalties are too great, especially for applications such as databases and enterprise systems that need high-speed block-level access.

Instead, the market is likely to settle around file and object for scalability and flexible data access, and file and block at the array level for performance. 

Use cases for unified file and object

For now, vendors that combine file and object storage target specific use cases.

Pure Storage, for example, aims its unified file and object storage at artificial intelligence and machine learning, as well as data analysis and high-performance computing. The technology could also be used where data needs to move between formats, perhaps using object to support cloud-native applications.

IBM, with its Spectrum Scale system, sees unified file and object storage as a time-saving device. It removes the need to port data from an object store to a file system so analytics applications can read it. The same data can be read as an object, or a file.

Evolution not revolution

However, industry watchers expect unified file and object to be less of a market segment than an evolution of existing technology. Ideally, vendors will hide the complexity of how data is stored from the end user, with intelligent technology making the decisions about how and where it resides.

“I don’t know if it’s a market segment as much as an evolutionary process,” says 451’s Hill. “A majority of the data in the cloud today resides on object storage, so now it’s about the need to better leverage the capabilities of both platforms to provide global data management, automate data lifecycles and insure data security that spans all forms of enterprise storage.”

SNIA’s McDonald agrees, and sees the cloud driving the move to unified storage. “Cloud providers now have file and object offerings that are, apart from the protocol used to access the data, built on the same stuff and very often allow file and object access to the same data,” he says.

The question is how quickly, and how effectively, the technology will expand to cover the rest of an enterprise’s storage needs.