canjoena - stock.adobe.com
Taming the data beast with software-defined storage
Australian enterprises are turning to software-defined storage to improve data management and speed up testing and development
As people and things get more connected and mobile, organisations are facing a flood of new data and new data types, both structured and unstructured.
For that data to power business growth, organisations need to do more than just store it. They need to manage it, extract value from it and keep it protected and compliant.
In its 2018 storage prediction, Gartner notes that “the cost of an internet connection declines while its speed increases, more diverse kinds and quantities of data will be shipped, shared and stored in corporate and consumer systems. This will engender new and enlarged dimensions of collision, convergence, dislocation and transformation.
“Most of these new dimensions in corporate and personal computing will demand more storage capacity and faster, more cost-effective and more-efficient storage infrastructures.”
Software-defined storage (SDS), which promises greater transparency and control over data – where it is stored, how it is accessed, what applications it is made available to – is an increasingly important tool in the enterprise technology armoury.
The promise is that businesses can direct data to different tiers of storage or to applications, on premise, in the cloud or at the edge of a network, without loss of control or visibility.
By the end of this year, Gartner has forecast that about a third of all enterprises will have some form of SDS platform installed.
Early adopters in Australia
Some are underway already. Queensland-based Langs Building Supplies was the second worldwide to sign up for Rubrik’s data management solution, cementing Australia’s reputation as an early adopter of SDS, according to Rubrik co-founder and CEO Bipul Sinha.
On a recent trip to Australia, Sinha explained that for most organisations, the pain point that led them to explore SDS was managing backup and disaster recovery, and that is where Rubrik has carved out a global niche. But he explained that Rubrik has been designed to allow enterprises to dial up the data they want for any application.
“Once you modernise backup and recovery, whole new possibilities emerge,” said Sinha, as it allows data to be accessed for compliance purposes, to help with security, or be served up for analytics or testing and development.
Intelligent data management
Intelligent data management is particularly important for cloud computing users. “Companies provision compute – but you have to provision data as well and you want it secure.” he said, adding that SDS was a way to automate and orchestrate the data.
“The real value is that the business user no longer has to call the IT department – they can do self-service. And a developer who wants any of the production data for test and development can provision that with the click of a button,” said Sinha. “The goal is to deliver any data to any user at the right time.”
Australia’s Sanitarium is another Rubrik user, benefiting from its backup and archival facilities, as well as expediting test and development workflows.
According to Alastair Stuart, Sanitarium’s infrastructure services manager, the company’s database administrators had to build servers from the ground up to test enterprise resource planning (ERP) software upgrades previously, but could now do so within five minutes.
Theo Hourmouzis is the ANZ managing director of Cohesity which has a SDS management platform that directs data to a customer’s own storage, an HPE platform or into the cloud.
Better data management
While managing data volumes is a growing challenge for all enterprises (many Gartner clients report 80% annual increases), the usual trigger for a move to SDS is the data protection challenge and the control that an SDS platform can deliver. “It is about shining a light on dark data when it sits in secondary storage and being able to extract value from that data,” said Hourmouzis.
And data sitting in the dark offers little enterprise value. Joydeep Das, senior director of product for Cloudera, says a key challenge for companies juggling lots of data is that it is often used only intermittently.
“There is a lot of investment in storage. The issue that most customers face is that it is static and often not fully utilised – for example analytics may run only for two hours a day, but it is there idle for six or eight hours,” he said.
He says that more intelligent data management can dynamically move workloads to wherever makes most economic sense, cleaning and curating it along the way. Companies will also need to understand the “gravity” of data, as well as the costs and delays involved with moving it around over a network.
“Where we process data we compress it,” he says, reducing the communications load as data is shipped from one location to another. “It’s always a good idea to have data not move around too much,” said Das.
VMware has long focused on developing software-defined computing, storage and networking. Its ANZ product marketing manager Brad Engstrom says that by abstracting everything into software, an enterprise can get much greater flexibility without sacrificing control.
A software-defined approach makes it possible to ensure consistent policy – for example, requiring data to always be encrypted wherever it is stored and however it is used – and consistent performance regardless of whether data is stored, even in OpenStack or in containers.
Read more about storage in Australia
- Even as Aussie firms such as Bauer Media and Redbubble are lapping up object storage, others are facing visibility and analytics issues with the technology.
- On-premise and cloud-based flash arrays that offer big improvements over spinning disks are making a splash in Australia.
- Airtasker is using Amazon Web Services to ramp up its storage capabilities to meet the needs of an expanding user base.
- The storage of big data is proving to be major headache for organisations in Australia and New Zealand.
While Engstrom acknowledges that economics continues to play a significant role in driving enterprises down the software-defined route, the additional flexibility and opportunity to future-proof a business are also highly prized.
NetApp technical sales leader and field chief technology officer Matt Hurford stresses that the enterprise drivers for adopting SDS are primarily around agility. Businesses have made too many decisions in the past where they have been trapped by architectures and can’t move as fast as they would like.
“This means they are able to scale quickly, leverage the cloud and provide pace to the business,” he said adding that moving to SDS meant that tasks which once took eight hours can now be compressed to eight seconds.
It does demand a workforce rethink as well, though. “I talk to a lot of storage administrators and tell them they need to be data administrators, thinking about the business logic of where data and workloads should be deployed,” said Das.
