Cloud storage cost challenges and how to tackle them

Cloud flexibility, but costs fall for upfront commitment

The cloud offers almost unlimited capacity, and the flexibility for organisations to pay for the resources they need. These are key benefits to the cloud. Firms are not constrained by the need to build datacentres, and they don’t – or shouldn’t – pay for capacity they do not need.

But, as with any other rental agreement, there is a price for flexibility. And cloud providers reward customers who can commit up front, or commit for a longer period of time, with discounts. This makes it harder for IT departments to predict pricing, and some experts argue, overturns the pay-as-you-go model of cloud.

“The economic model of the cloud is essentially that the more you can commit to use, the better the deal will be,” says Stephen Edwards, a digital expert at PA Consulting. “It is not really that different to a mobile phone contract. If you commit to a higher [volume], they will give you a better discount.”

This, says Patrick Smith, chief technology officer (CTO) for Europe, Middle East and Africa (EMEA) at storage provider Pure Storage, is pushing buyers to signing up to larger contracts. They are potentially over-buying, or moving too quickly to the cloud. “One thing with those [contracts] is organisations have to adopt at a certain rate,” he says. “If you don’t adopt fast enough the discount diminishes.” If organisations fail to “onboard” to the cloud quickly enough, they face additional costs.

Managing cloud lock-in and purchasing

Added to this is a tendency for the larger cloud providers to promote proprietary technologies that lock buyers in. If it is harder for firms to move between suppliers, they are less likely to be able to consolidate their requirements with a single supplier or use the threat of switching to drive down costs.

This is reinforced by the different architectures and approaches used by cloud providers. Suppliers have their own views of best practice that make it harder to move between clouds and to create a setup that works equally well across multiple clouds.

The situation is made worse by the ease of cloud purchasing – because almost anyone can spin up cloud resources, it is hard for IT departments to maintain control.

“The key challenge in cloud cost management is visibility into the consumption and use of cloud services. For many organisations, adoption of cloud services happens in a decentralised way,” says Nick Heudecker, a director at Cribl, a company that helps organisations sift through their IT data.

“Another aspect of visibility is simply understanding the bill at the end of the month from your chosen cloud service providers,” he says. “The decentralised nature of cloud consumption in most organisations makes deciphering bills difficult, if not impossible.”

Then there is the issue of the underlying pricing itself. Cloud pricing is far from transparent.

According to Aran Khanna, CEO of Archera, a cloud pricing specialist, Amazon’s pricing data alone is a 2GB JSON file. This makes comparisons difficult. “You can’t just load that up and then put in the Azure pricing sheet,” he says. “This is why when you look at large enterprises’ cloud management teams, they have three or four data scientists sitting there. It is no longer an Excel job.”

Big challenges to right-size and forecast

Perhaps the biggest challenges facing firms using the cloud is accurately forecasting demand, and then ensuring what is bought is actually being used.

“From a cloud perspective, the real challenge is cost predictability,” says Paul Walker, EMEA technical director at iManage, which provides tools for knowledge workers on top of Azure.

“The key questions that CIOs get asked when trying to build a business case for moving to the cloud is how much will it cost us annually, over two years and so forth? Also, does a scalable solution mean that our costs will fluctuate month on month and year on year?”

A key source of overspending is buying more capacity than the project or business needs. Forecasting helps here, although it is an inexact science as part of the point of the cloud is to be able to add capacity quickly.

“Can you make [usage] more predictable? You can do planning based on what you expect to use on the platform, but the issue is it’s just too easy to add capacity,” warns Oscar Arean, head of operations at cloud service provider Databarracks.

“If someone in the past needed additional compute, they’d ask the IT department and might decide it is not worth the cost,” he added. With the cloud, unless there are clear purchasing controls, it is all too easy for developers and others to add compute, memory or even storage.

Making sure to scale back down

Waste also occurs through keeping capacity running once a task has finished, or through failing to use the scalability features of the cloud platform. “One of the big buckets we look at is waste,” says Aran Khanna at Archera. “You have to shut stuff down or resize it.”

This means finding redundant test-and-dev servers, and taking them offline. Firms should look to scale down systems out-of-hours or during less busy periods. And extending this to storage, firms can move less frequently accessed data to cheaper storage tiers or dedicated archiving products. Business units face hardware limits with on-premise technology. With the cloud, it is too easy to add capacity without considering the cost.

“This needs active management to flex services up and down according to business requirements,” says Terry Storrar, managing director at Leaseweb UK.

Organisations also need to update their architectural approaches to suit the cloud. Not all firms are ready for cloud-native technology such as Kubernetes.

Nonetheless, the worst option is to recreate existing infrastructure in the cloud. Unlike local hardware, cloud systems do not need to be built for usage peaks as they can scale quickly. Virtualisation is less useful than it is on-premise, and it can be costly when virtual machines are using resources all the time. The cloud provides easier and cheaper backup and business continuity systems, even across geographies.

Experts suggest that potential savings of 30-40% are possible through better planning and management, optimising architecture for the cloud, and better purchasing.

“One of the advantages of cloud architecture is that solutions can be very easily procured,” says iManage’s Paul Walker. “The challenge is to ensure a level of adoption that delivers the return on investment.”