Snowflake brings AI to data for Australian businesses

Data governance

The Snowflake Horizon governance system helps organisations control access to content and protect sensitive or personally identifiable information, said Ramaswamy, while making it easier for employees to find the data, data products and models they need to do a better job.

Supermarket giant Coles used to operate an on-premises data platform, but problems with security, scalability and cost led to a migration to the cloud that is near completion, said Coles’ head of data, Saurav Sachdev.

With executive sponsorship of security and governance issues, standards were in place and “everything we do follows that.” Consequently, governance is not seen as a roadblock, he explained.

Coles’ data team is working on improving its efficiency, for example through increased automation, to allow more time to be spent on innovation such as applying AI. Personalisation is another hot topic, with the goal of providing a seamless online and in-store experience, but “there’s a long way to go,” Sachdev conceded.

Judo Bank is another local success story for Snowflake. Its general manager of data, Isaac David, said despite being granted its licence as recently as mid-2018, he had a difficult conversation with the bank’s chief operating officer because the original SQL Server-based system had reached its limits.

After considering various tools including Databricks, Judo settled on Snowflake. Despite involving around 1,400 workloads, the migration to Snowflake only took two months, helped in part by the in-house creation of tools to automate code changes.

The migration put Judo “in a really good situation,” David said. There is less fluctuation in processing times, the platform requires less ongoing management, freeing up around 5% of the team’s time, and provides secure data sharing and collaboration.

Governance has also improved and running costs were reduced from A$8,500 to A$315 a month for the same workloads. The volume of data under management has since tripled, and the data team is continuing to work on gaining insights to help customer relationships from the expanding data.

Snowflake caters to all users, according to the company’s director of product, Jeff Hollan. From data pipelines to data products, the platform supports batch and streaming ingestion, analytics for various data types, and Snowflake Copilot for enhanced productivity.

Commonwealth Bank’s (CBA) chief data and analytics officer, Andrew McMullan, lauded Snowflake’s versatility for model creation and execution, including models from external platforms. Applications range from real-time fraud detection to personalised customer offers. The bank even uses Snowflake to detect and block abusive messages in online payment descriptions, sharing these algorithms with other financial institutions.

All of the bank’s AI models are monitored in real-time via Snowflake, with alerts going to the right people if a model’s performance drifts beyond specified limits, with the option of using a “kill switch” in situations where performance drops below a certain threshold, which could be in terms of the accuracy of the model’s predictions, or the degree of customer engagement with targeted communications.

McMullan spoke highly of Snowflake’s Data Exchange capability, which enables organisations to grant others access to selected data sets. Importantly, this is done without having to copy the data. One way Commonwealth Bank uses this is to consume data from the Bureau of Meteorology, fire services and others, in order to alert customers of impending natural disasters and let them know what the bank can do to assist, whether that is repayment holidays for borrowers or arranging temporary accommodation for insurance policyholders.

“This is one of the best features of Snowflake,” he said, pointing out that it provides a way to allow others to use an organisation’s data in computations without being able to see the raw data. This, for instance, is how the bank works with various partners such as loyalty schemes to deliver better targeted offers to shared customers.

He also appreciates the way Snowflake Marketplace allows organisations to offer their Snowflake-native apps to other users. In 2023, CBA introduced NameCheck, a mechanism that reveals the name of an account when a customer creates a new payee based on the bank state branch (BSB) and account numbers, and also helps identify fraudulent or suspicious accounts. Later that year, it began offering NameCheck to other financial institutions via an application programming interface (API) and is now looking to make it available in Snowflake Marketplace.

McMullen said the bank is talking to Snowflake about a potential project to move the data from old applications into Snowflake, where it would be accessed via a large language model. That would be easier than migrating to new applications, as well as being cheaper and less complex than continuing to run the original apps.

More generally, CBA wants to build new features faster than anyone else, and a modern platform such as Snowflake makes that possible, he said.

For generative AI (GenAI) projects, Snowflake Cortex is a fully managed service within the Snowflake platform that provides access to top-tier foundational models, and a no-code playground for getting started with GenAI, said Snowflake’s Hollan. Taking a foundational model and training it with organisation-specific data results in a small, economical model that gives the required results, he added.

Vivek Luthra, Accenture’s senior managing director and Asia-Pacific data and AI lead, sees AI as a transformative opportunity, with financial services leading adoption in the region. He emphasised the need for scalable AI solutions, measurable business value, and the importance of decoupling foundational models from enterprise architecture so new and more effective models can be more easily adopted.

Over the next year or two, organisations should contextualise interactions based on their own data, tune models with their own data, investigate how they will enable autonomous processes at scale and how to put AI resources at the edge, Luthra suggested.

Bunnings’ Elliott has a similar view: a successful AI strategy “requires a strong, carefully thought through data strategy.” The retailer’s investment in its enterprise data platform and its integration with cloud applications has set up Bunnings to handle AI initiatives at pace, she said.