In-memory for continuous (& simpler) deep learning Part 1

Ivanov writes….

Companies increasingly deploy machine learning (ML) to improve business processes and enable real-time, automated decision-making. When doing so, they often face application speed and scalability challenges.

In the first of this two-part series, we look at how in-memory computing (IMC) can provide the necessary speed and scale for real-time applications. In Part 2, we will explore the role of IMC in ML and deep learning (DL)

To take advantage of machine learning to improve business processes and enable real-time, automated decision-making, companies must be able to guarantee real-time application performance at scale. They may also need to frequently update their machine learning models.

Finally, they must accomplish all this while minimising infrastructure costs.

Consider these examples:

• Credit card companies are under pressure from card issuing banks to shift from nightly to hourly (or even more frequent) updates of their ML-powered fraud detection models to enable faster detection of new fraud vectors.
• E-commerce sites want to improve the relevance of their recommendation engines by more frequently updating their machine learning models based on the latest user website interaction data.
• Enterprises across multiple industries want to improve information security and detect new threats faster by more frequently updating their model of ‘normal’ network activity.

For companies deploying deep learning technologies, another key consideration is minimising the cost and complexity of their DL infrastructure.

An in-memory computing (IMC) platform can help companies create continuous learning applications and more effectively manage infrastructure costs.

An IMC platform can:

• Eliminate the time delay introduced by time-consuming extract, transform and load (ETL) processes and reduce the time required to analyse data.
• Distribute processing across a cluster of computers to provide the speed and scalability for processing, analysing and acting on massive amounts of data in real-time.
• Run ML training algorithms within the platform on the operational data and retrain the ML model at any time without impacting system performance.
• Feed the operational dataset directly into a deep learning training application without the need for separate analytical storage infrastructure.

Application speed & scalability

An in-memory computing platform, deployed on a cluster of commodity servers, pools the available CPUs and RAM and then distributes data and compute across the cluster. The IMC cluster can be deployed on-premises, in a public or private cloud, or on a hybrid environment. When used as an in-memory data grid (IMDG) inserted between an existing application and disk-based database, no rip-and-replace of the database is required.

The IMDG synchronises with the data in the database and then processes all the reads and writes. The application sends new transactions to the IMDG, which writes them to the underlying database, ensuring data consistency and availability. With all data in memory, the IMC platform applies massively parallel processing (MPP) across the distributed cluster to enable up to a 1000- times increase in application performance.

Since the in-memory computing platform uses a distributed architecture, increasing the compute power and RAM of the cluster is as simple as adding new nodes. The platform automatically detects additional nodes and redistributes data to ensure optimal use of the cluster CPU and RAM.

In the next article in this series, we will look at how an IMDG can enable organisations to overcome the performance and scalability challenges of continuous machine learning.

Ivanov: Deep in thought about his in-memory memories.