Cloud Storage Revisited: What has the last decade taught us?
It is clear that cloud storage has become an important component of modern IT infrastructure. Nearly every IT professional is faced with rapidly growing data volumes, and in many instances they are relying on this data as a foundation for rapid decision-making and to help direct innovation. However, as the dependence on cloud storage grows, so do the challenges.
The initial marketing promise of cloud storage, namely almost unbounded, low cost capacity, accessible from anywhere, has slowly given way in the light of experience to a far more complex reality. This means that many IT professionals are now faced with issues that established cloud storage solutions were not designed to address effectively.
Learning from experience
In recent years we have heard some challenges mentioned time and again in the research we have carried out. And close to the top of the list is cost unpredictability. For example, beyond basic storage costs, complex pricing models often include less visible charges for data retrieval, egress, and API calls. These can significantly impact budgets, especially when dealing with large datasets or frequent access as new uses of the data are established.
Security considerations also crop up frequently. While cloud providers offer robust security measures, the centralised nature of traditional solutions can present risks, especially if good practice hasn’t yet been established. Given the sensitive nature of some data, it is important to consider if there might be adverse implications when storing all data with a single provider, especially if multiple legal or financial jurisdictions are involved.
Another factor to bear in mind here is the challenge of managing security coherently across different storage repositories and service providers. This is a particular issue where different storage repositories may use solutions proprietary to the provider, storage stack or that are reliant on specific applications. In all of these scenarios, building in security at the data level can be important, especially to protect against potential ‘back door’ unauthorised attempts to directly access the storage repository?
It is also important to consider concerns about data mobility, or rather stickiness of data in many cloud services today. The ability to move and manage data flexibly across different environments is becoming more significant for innovation, cost optimisation, and to meet changing regulatory requirements or the concerns of shareholders and customers. However, several factors can complicate cloud data mobility.
Location and Lock-in
Changing providers can be expensive and time-consuming due to contractual vendor lock-in or the substantial costs associated with large-scale data egress, never mind potential compatibility issues. There may also be occasions where data sovereignty regulations require precise control over data location and access. Another factor easy to overlook is where performance may vary appreciably depending on geographic location, potentially impacting application responsiveness and user experience, especially important for geographically distributed or international teams.
These interrelated challenges underscore the need for cloud storage solutions that offer greater agility in data management and deployment, balancing flexibility, compliance, and performance optimisation. Any lack of flexibility can limit the ability to take advantage of better services, superior performance or more cost-effective offerings from other providers.
So, how can we leverage the benefits of cloud storage while addressing these complex requirements? The answer may lie in next-generation cloud storage solutions that offer a more comprehensive approach to data management.
Time to reappraise
Given that most cloud and on-premise storage solutions have evolved in comparative isolation, what capabilities are desirable in a modern cloud, and I am using the term in its broadest sense, storage management platform? Here are some features that are likely to be desirable.
- Multi-cloud Architecture: Solutions that allow data distribution across multiple clouds and cloud providers can help reduce dependency on any single vendor and enable cost optimisation by utilising the strengths of different providers.
- Advanced Security: Assess features beyond basic encryption, such as erasure coding and distributed storage. These can enhance security and resilience by reducing the risk of a single point of failure compromising the entire dataset
- Intelligent Data Lifecycle Management: Consider solutions that offer automated tiering and customisable retention policies to help manage data more efficiently across its lifecycle, balancing accessibility with cost-effectiveness.
- Compliance, Location and Data Sovereignty: Data protection regulations means solutions with granular control over data placement and access can help ensure compliance without sacrificing performance or incurring unnecessary costs.
- Cost Optimisation: Some next-gen solutions incorporate FinOps capabilities using intelligent algorithms that can give insight into complex cloud spend and suggest ways to optimise costs in complex, multi-cloud and on-site environments.
- Integration with Existing Systems: Modern solutions should support a wide range of standard interfaces and protocols such as S3 API, NFS, SMB etc, as well as common file systems to enable simple integration with existing tools and workflows.
- Flexible Deployment: It is important that solutions are able to adapt to specific organisational needs and future data storage requirements, whether on-premises, private cloud, public cloud, or a hybrid approach.
At Freeform Dynamics, we’ve been developing this list for some time, indeed it’s been gradually evolving over a period of nearly two decades. So what prompted us to put this out in its latest form? Well this article was stimulated by a conversation with a company called Vawlt, that seemed to prioritise many of the same things we do – not just a convenient subset that reflects its proposition. As we had a good ‘meeting of minds’ we thought it was only right to give the company a shout out, though to be fair we are seeing most good providers moving in a similar direction.
So what does this mean for customers?
The future is bright, but you need to stay on your toes
Looking forwards it is clear that cloud and on-prem storage solutions will continue to evolve, reinforcing the need for more flexible, secure and cost-effective storage architectures. This progression introduces both challenges and opportunities.
Navigating this changing landscape is a challenge faced by all IT professionals. Many find it useful to regularly reassess their cloud storage needs and costs, keeping them aligned with their overall IT strategy. After all, the cloud market is constantly evolving, and yesterday’s optimal solution might not be the best fit for today. But staying up to date about emerging solutions and technologies can be time consuming.
The concept of multi-cloud strategies has become widely accepted, offering potential for enhanced flexibility, improved performance and optimised costs. As cloud storage has evolved from being a utility into a strategic asset, the demands placed upon it have grown. Data security and compliance remain critical, increasing the need for advanced data management features. The ability to automate processes and gain insights is becoming essential for efficient, cost-effective cloud and on-prem storage management.
In this evolving landscape, cloud storage decisions already carry broader business implications. The right solution could enhance productivity, drive innovation, and open new opportunities, while a less suitable option may lead to increased costs, security risks, and missed chances. While traditional solutions have served the industry well, the time has come to explore more advanced options. The cloud storage of tomorrow, and even today, is not just about capacity, it’s about enabling the organisation to do more with its data, securely and efficiently.