Massive hyperscaler GenAI spend raises questions on costs

The latest Gartner worldwide IT spending forecast shows that the analyst firm expects a 7.5% increase in spending compared to 2023. Total spending on IT is set to reach $5.26tn in 2024, driven by an increase in datacentre investments powered by demand for generative AI (GenAI).

Gartner has forecast that spending on datacentre systems is expected to increase 24% in 2024, up from the previous quarter’s forecast of 10% growth.

John-David Lovelock, distinguished vice-president analyst at Gartner, said: “The compute power needs of GenAI are being felt across the datacentre, and spending in that segment reflects this ravenous demand.”

According to Lovelock, it has taken the hyperscalers between 10 to 15 years of accumulative spending to match the amount of money that they spent just in 2022. He said that $67bn was spent on servers in 2022 and this figure is set to double by 2025 to $134bn. By 2028, Gartner expects spending on servers to triple again, which means that hyperscalers are set to spend up to $500bn on AI servers by then.

Lovelock pointed out that there is a massive gap between the level of investment in GenAI infrastructure and the ability of the software industry to monetise GenAI: “By the time we get to 2028, the hyperscalers are spending $180bn a year on servers to run GenAI. Yet the most advanced company right now, OpenAI, has an annual run rate of $3bn.”

The situation with GenAI revenue is something CIOs need to start considering. Lovelock said that between 2009 to 2022, cloud providers convinced CIOs that public cloud services meant they either got more functionality every year or the price went down.

“They had almost 15 years to learn that cloud prices don’t spike. So, when the cloud providers started to raise their prices, some CIOs felt this was a betrayal,” he added. 

He warned that cloud investment costs are spread into the future, which means that it is very hard for cloud spending to go down, and cloud providers rely on new contracts coming in to payback for the upfront investments they continue to make.

“It is completely predictable from a market mechanics perspective that this could happen,” he said. “But no one expected it to happen to them.”

Such price increases, Lovelock said, have meant that many CIOs are seeing their entire annual IT budget increase disappear. 

GenAI is fast becoming integrated into the feature-set of enterprise software. Yet, while some software spending increases are attributable to GenAI, according to Lovelock, in a typical software company, GenAI most closely resembles a tax. “Revenue gains from the sale of GenAI add-ons or tokens flow back to their AI model provider partner,” he said.

In spte of the demand for AI, Gartner’s forecast suggests IT services spending is set to grow 7.1% in 2024, down from 9.7% within the last forecast, due in part to slower spending across subsegments that include consulting and business process services.  

“The change fatigue in CIOs that we saw at the start of the year has now abated, and the contract backlogs going back to the third quarter of 2023 are being cleared. We expect to see a larger rush towards the end of the year to make up for the slow start,” said Lovelock.