CW Developer Network

Modern development - Splice Machine: A new strategy for application-first storage

Adrian Bridgwater
Adrian Bridgwater

This Computer Weekly Developer Network series is devoted to examining the leading trends that go towards defining the shape of modern software application development. 

As we have initially discussed here, with so many new platform-level changes now playing out across the technology landscape, how should we think about the cloud-native, open-compliant, mobile-first, Agile-enriched, AI-fuelled, bot-filled world of coding and how do these forces now come together to create the new world of modern programming?

This contribution comes from Monte Zweben, CEO of Splice Machine — the company is known for its scale-out database built with Machine Learning (ML) capabilities. The company says that Splice Machine is a scalable SQL database that enables companies to modernise their legacy and custom applications to be agile, data-rich and intelligent – all without re-writes.

Zweben writes as follows…

The world has shifted from on-premises big data storage architectures to inexpensive cloud data architectures. As a result, many articles have stated that the public clouds have killed Hadoop. But that is not really the case. We believe it was complexity that killed Hadoop.

Hadoop hurdles

There are two major challenges to Hadoop:

  • Operational Complexity — The DevOps burden of keeping a massively distributed system based on commodity hardware alive, performant and secure.
  • Development Complexity — The development burden of duct-taping many disparate compute and storage components together to form a functional solution without suffering from latency due to data movement

The public cloud nailed operational complexity challenges. This was a major blow to the Hadoop distribution companies like Cloudera, Hortonworks and MapR that were late to the cloud. AWS, Azure and GCP nearly eliminate the operational complexity of operating the core components of the Hadoop ecosystem.

Zweben: Don’t get caught in the data swamp.

Additionally, other cloud providers even provided more analytical SQL capabilities such as AWS Redshift, Google BigQuery and Snowflake. This really simplified analytical processing operationally with the ability to run analytical workloads and then shut down the resources, saving money.

However, we believe cheap storage is not really going to deliver on business outcomes.

It is easy to replace your big data architecture with a modern cloud storage architecture. But now what? It’s the same old data swamp. Lot’s of data in, not much business value out.

Forward strategy: application modernisation

By focusing on the modernisation of applications with data and intelligence, you will end up with apps that can leverage data to make decisions in the moment that result in superior business outcomes. 

Here are five ingredients to a successful application modernization strategy:

  1. Pick an application to modernise: Rather than focusing your efforts on centralising the data, first, pick an application that you would like to modernise. Once you have identified the application that can deliver a competitive advantage to your organisation, then you can focus on sourcing the data required to power that application and whether that data can be made available from the data lake.
  2. Use scale-out SQL for your application modernisation: SQL has been the workhorse of workloads in the enterprise for a number of years and there are hundreds of developers, business analysts and IT personnel in your organisation who are fully conversant in SQL You need your storage to be actionable. That requires SQL.
  3. Adopt an ACID platform: ACID compliance is the mechanism through which transactions maintain integrity in the database and allows users to perform actions such as commit and rollback. It is a critical functionality to power operational applications as it ensures that the database does not make changes visible to others until a commit has been issued. Files in cloud storage don’t provide ACID transactions.
  4. Consolidate Analytics Engines: According to a recent Gartner blog, historically, there were good reasons to separate your IT infrastructure into operational (OLTP) and analytical (OLAP) components, but that is no longer the case. ETL kills our SLA’s with latency. This ETL is no longer required and you can run analytics on the operational platform, often using the operational storage.
  5. Embed Native Machine Learning: One of the primary reasons for modernising your application is to inject AI and ML into it so that it can learn from experience, dynamically adapt to changes… and make in-the-moment decisions. In order to make your application intelligent, it is critical that you select a platform that has machine learning built-in at the database level so that updated data is always available to the models to experiment, train and execute.

This is a fundamentally different approach than storage modernisation on the cloud. This approach delivers tangible business value to the line of business faster through the application that can now leverage the data.