AI is renewing interest in high-performance computing

The growing adoption of artificial intelligence (AI) is driving more organisations in Asia to turn to high-performance computing (HPC), according to a senior Lenovo executive.

Speaking to Computer Weekly on the sidelines of the recent supercomputing conference in Singapore, Bhushan Desam, Lenovo’s global AI business leader, said HPC is moving from the hands of researchers to the broader enterprise market as more organisations require a lot more computing power to crunch large datasets quickly in AI applications.

“HPC is going to accelerate AI, but not just in the HPC community,” he said, noting that HPC capabilities such as InfiniBand high-speed networking and GPU (graphics processing unit)-based processing power will help enterprises make sense of data in hours, not weeks.

Desam said industries such as healthcare and financial institutions that are dipping their toes into AI will benefit from HPC, which is good at breaking down mammoth tasks into chunks for machines to process in parallel.

“Healthcare organisations are using HPC to crunch millions of images in image-based diagnoses,” he said. “In manufacturing, HPC is used to not only run engineering simulations, but also to predict when something will go down so they can minimise downtime and improve operational efficiency.”

Sensing the market opportunity in AI and HPC, Lenovo said in 2017 that it would earmark $1.2b over four years for AI research and development. It has already built up a team of about 100 data scientists who have been working on AI projects at research centres in China, Germany and the US.

With AI, Desam said demand and innovation in HPC is taking place in both the Western and Asian markets. “AI has turned the tables around and levelled the playing field,” he said. “In the past, most HPC innovations were driven by the West, but now we see a lot of enthusiasm and interest in AI in Asia, in areas such as computer vision and robotics in Japan’s manufacturing industry.”

In approaching the AI and HPC market, Desam said Lenovo is not taking a hardware-centric strategy, dovetailing with the company’s aim to move away from shipping boxes.

Instead, Desam said Lenovo is taking a more consultative stance, helping enterprises to identify the benefits of AI and HPC, through proof-of-concept (PoC) projects to test different use cases with the help of partners such as research institutions.

On the software side, Lenovo is also supplying tools to help enterprises manage their HPC workloads, including orchestrating workloads across multiple nodes on the same infrastructure and speeding up machine learning using pre-trained datasets, said Desam.

Desam’s work comes at a time when interest is growing in tapping cloud services to run HPC workloads, including AI.

For some years now, major cloud suppliers such as Amazon Web Services and Microsoft have been offering services aimed at governments and research institutions with HPC requirements. According to some estimates, the global cloud HPC market is expected to reach $10.8bn by 2020, up from $4.4bn in 2015.

Although Desam is not dismissing the benefits of the cloud, he pointed out that most cloud-based HPC workloads related to AI are experimental projects by enterprises that are just starting out in AI.

“But once you’re past prototyping, you need to move data, which is mostly stored on-premise, to the cloud,” he said. “It’s not economical to move the data from the datacentre to the cloud. There are also industries, such as healthcare, that cannot store their data in the cloud.”

That said, Desam noted that enterprises can train their neural networks with data residing in on-premise datacentres, and deploy them in cloud-based applications for employees to use across the globe.

According to Hyperion Research, the global HPC server-based AI market will grow at a compound annual growth rate of 29.5% and surpass $1.26bn by 2021, up more than three-fold from $346m in 2016.