Green code - TurinTech: Code optimisation for sustainable AI

The true cost of inefficient code

Despite economic hardships, stubbornly high energy prices and ESG priorities, businesses can often overlook the importance of optimising their AI code when looking to streamline operations and reduce costs. This can mean inefficient code emerges from a lack of priority or awareness.

During my time working in the finance industry, I saw firsthand just how time-consuming and resource-intensive the manual process of developing and optimising efficient code was. It involves iteratively refining and testing code to achieve optimal performance, a process that can take days, even for experienced engineers. This challenge grows with large codebases, where the lack of a comprehensive overview hinders effective optimisation.

While there is no easy price estimation, the costs of deploying a basic AI system can reach $50,000, largely attributed to hardware, software and data. But it is not just deployed AI systems that skew the balance sheet: for every success published about AI’s evolution, there have been a host of unpublicised failings in trials and testing.

Each failed test and training process carries a subsequent environmental cost for the planet. The training of OpenAI’s GPT-3 consumed 1,287 MWh, resulting in emissions of over 550 tons of carbon dioxide equivalent. This is comparable to one person making 550 round trips between New York and San Francisco.

Furthermore, inefficient code severely impacts both profitability and customer experience. With AI-driven software now incurring cloud costs up to 80% of revenues, optimising code is crucial for financial health. This inefficiency risks profits and damages user experience, particularly where speed is vital, such as in trading and autonomous driving. Consumer apps also suffer, affecting satisfaction and loyalty.

The rise of LLMs

A report released last month by the Centre for the Governance of AI revealed that, “Over the past thirteen years, the amount of compute used to train leading AI systems has increased by a factor of 350 million” – and it is doubling every six months. While surging computing is delivering impressive AI capabilities, this rapid growth has the simultaneous drawback of crippling costs, both financially and environmentally.

It has been estimated that ChatGPT bills an eye-watering $700,000 a day to operate – a highly unsustainable model – and submitting just one GPT query will consume 15 times more energy than a Google search query. If we are to integrate LLMs into search engines, for example, this could lead to a five-fold increase in computing power and higher carbon emissions. These numbers underline the pressing need for energy-efficient coding, especially as this era of generative AI expeditiously evolves. The problem is models are growing much faster than their hardware capabilities and overlooking inefficient code is exacerbating this unbalanced growth. However, when it comes to taking ownership and addressing this unsustainable development, it is not clear how much effort the big players are putting into the issue behind the scenes.

The cloud tech world’s big three – AWS, Azure and GCP – are gradually adapting to the situation and offering optimised compute and memory services. Yet, in comparison to the surge in data centre and virtual machine centre growth to run large AI applications, these offerings do not present a sufficiently sustainable alternative. For smaller players looking to integrate and run AI applications, then, they need to find alternative ways of making their models as efficient as possible.

Code optimisation fosters sustainability

Neglecting code efficiency is like loading delivery vans only half full with parcels; it delivers less performance but consumes more energy and produces more carbon emissions.

Code optimisation ensures code runs at peak performance on the target hardware. It’s not just about making the code work; it’s about making the code work efficiently, reducing latency and improving the overall user experience.

The latest code optimisation platforms leverage advanced optimisation techniques and pre-trained LLMs to automatically scan through entire codebases and identify lines of inefficient code. They then recommend and benchmark optimal code changes to enhance performance and allow developers to compare the improvements made with the original code. This accelerates the execution time of applications and significantly reduces compute usage and cost. It’s code performing the same tasks, just with much greater efficiency.

Kanthan: Optimising efficient code involves iteratively refining & testing to achieve optimal performance.

Optimising code to improve model performance can trigger a 46% reduction in memory and energy consumption on average and deliver cost-savings close to $2 million a year for AI model production and deployment. Crucially, this has the potential to create 256kg of CO2 emission savings a year alongside hitting sustainability targets much quicker.

Code optimisation in practice

The technology and financial sectors are the two that stand to benefit the greatest from code optimisation, as there is continuous integration taking place.

Datacentres are on the front line of AI’s sustainability conundrum and code optimisation is often overlooked as the golden ticket to reducing their energy consumption. But the effects of businesses optimising their code can deliver massive carbon savings: it reduces the computational load on servers and optimises data processing, storage and running software. Typical server wattage roughly stands at around 1,800 kWh per server per year. Taking the 46% energy consumption reduction as a benchmark, datacentres could reduce their emissions from 668 CO-eq in kgs/server/year to 360 CO2-eq in kgs/server/year.

In the finance world, from everyday banking transactions and the vast amount of data transferred between systems via APIs to high-frequency trading, the energy strain of maintaining software performance is enormous. Through adopting code optimisation practices, finance firms could, for example, adjust the target return rate of a trading model to maximise profitability while considering factors such as model prediction speed, electricity consumption and carbon emissions.

The golden ticket for sustainable AI

For companies using or integrating AI, everything is now centred on the addressing amount of compute power used and consequently its cost and energy consumption. Many companies rely on open-source models and so there is a big onus on the big providers to offer more code optimisation services. However, there are effective ways for companies to improve AI efficiency by using code optimisation tools to better model performance and align it with green metrics.

By embedding code optimisation into their processes, they have an immediate business impact. Efficient AI enables them to make accurate predictions quicker, analyse data faster and, most importantly, reduce costs and emissions. And if you can optimise your code so it runs quicker, you can get your product into production much faster.

The AI industry needs wide-ranging, holistic strategies to make sure it’s positively contributing to sustainability and not taking two steps back for every gain. Green energy, practical policies and hardware transformations are essential. Focusing on code optimisation can be the golden ticket for delivering immediate sustainable solutions for AI. And our planet can’t afford to hang around for other options. 

TurinTech tweets at @turintechai