Data and artificial intelligence (AI) company Databricks has unveiled DBRX, a general-purpose large language model (LLM) that it claims can outperform other open source models.
The company said DBRX outperforms existing open source LLMs such as Llama 2 70B and Mixtral-8x7B on industry benchmarks including language understanding, programming, maths and logic.
“DBRX democratises the training and tuning of custom, high-performing LLMs for every enterprise so they no longer need to rely on a small handful of closed models,” the company said.
Ali Ghodsi, co-founder and CEO of Databricks, said DBRX enables enterprises to build “customised reasoning capabilities based on their own data”. Because DBRX beats GPT-3.5 on most benchmarks, he said it should accelerate the trend Databricks is seeing across its customers – of organisations replacing proprietary models with open source models.
DBRX outperforms GPT-3.5 across language understanding (MMLU), programming (HumanEval) and maths (GSM8K), Databricks said.
DBRX was developed by Mosaic AI and trained on Nvidia DGX Cloud. Databricks optimised DBRX for efficiency with a mixture-of-experts (MoE) architecture, built on the MegaBlocks open source project. The resulting model is up to twice as compute-efficient as other available leading LLMs, the company said.
DBRX is available on GitHub and Hugging Face for research and commercial use. On the Databricks Platform, enterprises can interact with DBRX and build custom DBRX models on their own unique data. DBRX is also available on Amazon Web Services (AWS) and Google Cloud, as well as directly on Microsoft Azure through Azure Databricks. DBRX is also expected to be available through the Nvidia Catalog API and supported on the Nvidia NIM inference microservice.
While the model is open source, Databricks also offers services around it to help enterprises build and deploy production-quality generative AI (GenAI) applications.
“This is going to be by far the best open source model out there – it surpasses GPT-3.5 in quality and it is completely open source”
Naveen Rao, Databricks
“This is going to be by far the best open source model out there – it surpasses GPT-3.5 in quality and it is completely open source, and what’s more, we have innovated on the compute architecture of this model,” said Naveen Rao, vice-president of GenAI at Databricks.
Rao said the mixture-of-experts architecture used in the model is similar to having 16 models in one.
“When you query the model and say, ‘generate this output’, it takes a subset – four of them – to create the response. This is useful because you spread knowledge out amongst the different experts and you have this learned routing which figures out ‘these experts are the ones to query for this response’,” he said.
“We can get the speed and latency of a small model with the capabilities of a much larger model. This is something that, because of its computing architecture, is extremely fast. It’s fully open source, [so] companies can take this model, they can build upon it, fine-tune the model and they own the model weights – that’s a crucial piece here. They get the best economics for the quality,” he told Computer Weekly.
Being open source should allow customers to feel more comfortable about sharing their data because they have more control over the model than they would with a closed source model.
“We believe in a world where companies can build IP [intellectual property] for their purposes and wield that IP how they wish. Being able to fine-tune a model and have it served behind some firewall that you can never get access to is not IP creation. That’s actually IP creation for the model provider,” said Rao.
Rao added that regulated industries are reluctant to use their most important and sensitive data to train proprietary models, in part because they do not have control.
Making the model open source gives enterprise customers an incentive to use it across a variety of use cases, he added. “This whole idea of portability is very important, and it’s very hard to do it if it’s not open source,” he said.
If customers are able to take the model elsewhere, that gives Databricks the incentive to add value to its customers while giving them the flexibility they need, he said.
Included in Databricks’ announcements were comments from customers, including Zoom, which said it looked forward to “evaluating DBRX’s potential to make training and serving custom generative AI models faster and more cost-effective for our core use cases”.
Mike O’Rourke, head of AI and data services at Nasdaq, said: “The combination of strong model performance and favourable serving economics is the kind of innovation we are looking for as we grow our use of generative AI at Nasdaq.”
It could be that, after a period of domination by a small number of firms, the market for enterprise GenAI is beginning to change.
Databricks is one of a number of companies, large and small, including Meta (Llama 2) Google (Gemma), xAI (Grok), Mistral AI, Hugging Face and more, offering various open source GenAI options.
According to venture capital (VC) firm Andreessen Horowitz, closed source GenAI tools accounted for 80% to 90% of the market last year, with the majority of share going to OpenAI. But its research has found that half of the enterprise executives it spoke to now prefer open source models.
“In 2024 and onwards, enterprises expect a significant shift of usage towards open source, with some expressly targeting a 50/50 split – up from the 80% closed/20% open split in 2023,” the VC firm said.
It said that while enterprises are still interested in customising models, with the rise of high-quality open source models, most are opting to use retrieval-augmented generation (RAG) or fine-tune an open source model.
While the real impact of GenAI is still unclear, a recent study found that AI could help automate a vast range of the work done by civil servants across hundreds of government services. Another survey found that 80% of business leaders had invested in some form of AI in 2023, but said the biggest barriers to preparing workforces for AI included a lack of organisational expertise, employee scepticism and a lack of regulation.
