Do IT leaders need server upgrades?

Artificial intelligence (AI) has become the new Moore’s Law. Semiconductor manufacturers may strive to double processing power every two years, but that is not what their largest customers actually need.

In fact Google, Meta and Microsoft are extending the life of their servers in a bid to reduce costs operating cloud infrastructure. Meta’s chief financial officer, Susan Li sums this up in the company’s latest quarterly earnings when she revealed the company could see little performance gain from new server chips. Instead, the hyperscalers are focused on the next growth opportunity: AI.

Nvidia has set the benchmark for AI chips, pivoting its business to provide highly parallel graphics processing units (GPUs) and a programming environment to power machine learning and AI inference. Rival AMD is gearing up to unveil a new AI product roadmap in December. Google has its TPU, a custom-designed AI accelerator, which it claims is faster than Nvidia GPUs. And now Microsoft has developed its own AI accelerator chip, Maia, which the company says has been designed specifically for the Azure hardware stack.

The big loser in this race to win AI workloads is Intel. The company has posted dire quarterly earnings due, in part to its biggest customers, the hyperscalers, scaling back on their x86 server upgrades.

The PC market is also in decline. The latest data from IDC shows that PC shipments continued to fall during the third quarter of 2023 as global volumes declined 7.6% year over year with 68.2m PCs shipped. Linn Huang, research vice president, devices & displays at IDC believes generative AI could be a watershed moment for the PC industry.

On the server chip front, Intel is now ramping up its AI plans. It announced work on a new supercomputer, Aurora, with Argonne National Laboratory and industry partners to create what it describes as “state-of-the-art foundational AI models for science”. Aurora uses the Intel Max Series GPU architecture.

Companies like HPE are also joining in on the AI party. Nvidia is working with HPE’s Cray supercomputer division on a family of AI-optimised supercomputers based on its Grace Hopper GH200 Superchips.

But beyond specialist HPC hardware for AI, server shipments are expected to be relatively modest. This, as Meta’s Li suggests, server hardware is good enough to run existing enterprise workloads and the product mix from the major server manufacturers is not able to meet the AI requirements enterprise customers are looking for.

Clearly, the market will shift as server manufacturers try to become more relevant in the age of AI workloads as AI acceleration hardware is likely to be top of an IT leader’s shopping list.