frenta - stock.adobe.com
Ali Ghodsi, Databricks CEO: Scandi-Californian hybrid is an ideal
Ali Ghodsi, CEO of Databricks, reflects on the business culture of Silicon Valley, his Iranian-Swedish background, and where big data analytics is going during these early days of machine learning
Ali Ghodsi, co-founder and CEO of Databricks, remembers as a child seeing Tehran turning dark as the lights went out and Iraqi aircraft bombed the city during the Iran-Iraq war of 1980-1988. One night he believed his family’s house had been bombed. It hadn’t, but neighbouring houses had been.
“I wouldn’t say that sort of experience makes you more resilient, but it does prepare you for anything happening,” he says. “One day, everything can be great; the next, everything can have collapsed. It helps you stay calm in a crisis.”
Ghodsi moved to Sweden with his family and was educated there, from kindergarten to postdoc. And while he says the famously social-democratic culture of Sweden has left an imprint on his thinking, he prefers the risk-taking culture of Silicon Valley. “I always felt Sweden was built for the benefit of the big companies, like Ericsson, Volvo and ABB,” he says. “Not so much for the little guy wanting to launch his own company.”
When liberals in San Francisco or London speak about Sweden’s egalitarian and liberal reputation, arguably a more civilised country than the capitalist norm, Ghodsi says they are thinking of the Sweden of the 1960s and 1970s: “The Sweden of the 1990s wasn’t like that. And the Sweden of the 1930s was different again.”
Although business technology executives of the Iranian diaspora are less numerous than their Indian counterparts, they are emerging as a force. Ghodsi is the CEO of Databricks, a data-focused enterprise software company that first came to prominence as the inventor and commercial distributor of the Apache Spark processing platform that largely ousted MapReduce from the Hadoop family of data-storage technologies.
The company emerged from a project at the University of California, Berkeley, and one of Ghodsi’s co-founders is Romanian-Canadian Matei Zaharia, chief technologist at the company and a professor at Stanford.
Ghodsi continues: “The United States was created by refugees over the past 300 years or so. In Silicon Valley, you do find a lot of people who had to leave their home countries. And as a result, the country as a whole is like a startup, compared to where I grew up in Sweden. And so, in the Valley, you find a lot of high-risk-taking people.”
“In Silicon Valley, you do find a lot of people who had to leave their home countries”
Ali Ghodsi, Databricks
The children of Iranians who emigrated by force of the Islamic revolution of 1979 are now showing up in Silicon Valley, he says, just as previous waves of emigrants did – from the Hungarians in the mid-1950s (think of Andy Grove, CEO of Intel) through continuing waves from India, Russia, Europe, and elsewhere.
“You do find a lot of excellent research being done by Iranians, especially from electrical engineering, which is a good training for machine learning and AI [artificial intelligence] because it’s all about statistics and probabilities, and so on,” says Ghodsi. “And then in Silicon Valley, there is a small contingent of entrepreneurs and executives, like the CEO of Uber [Dara Khosrowshahi] or Omid [Kordestani], one of the key guys behind Google. And there’s more and more that you see.”
He sees a parallel with Afghanistan today. “There are lots of refugees leaving,” he says. “There’s going to be people that stick around. People don’t want to leave their home country. But then some of them will find that untenable in four or five years, so they’ll eventually have to flee as well.”
A United States of extremes
Ghodsi takes the view that the US exhibits extremes in ways that are not true of Sweden. “Sweden is a country where you find less extremes in either direction,” he says. “In the US, you will find the best universities, you can you find the best healthcare on the planet. But you can also find people who get zero healthcare, they get absolutely nothing. They’re on the streets and they’re on their own. In Sweden, you won’t find the world’s absolutely best healthcare, but there is a safety net. And in the US, the safety nets are not great. That’s unfortunate – it’s not good.
“In Sweden, for a refugee family, it was fantastic for us. We were given the average that every person should have. I was able to go to college and get a PhD [from the KTH Royal Institute of Technology, in computer science] and those kinds of things. It’s harder to accomplish that in the US.”
Ghodsi adds: “I would like to have a system that is more of a hybrid. For there is no denying that a lot of world culture comes from around here, it was innovated here, and in a short radius of here, in San Francisco. And a lot of that was created in the last 10 to 20 years. If you go to a coffee shop pretty much anywhere on the planet, you’ll find people on laptops, probably from Apple, and they are on Facebook or Twitter.
“It’s all spread from this small area. And we’re not speaking of things that were invented 100 years-plus ago, like cars from General Motors. These are recent developments, and it is great that that this area incentivises that kind of innovation.”
Ghodsi concedes that there are downsides to this turbo-charged technological development, because regulation can’t keep up with the speed and scale of adoption globally. But he takes the view that the big beasts of the Valley are people who have genuinely wanted to make the world a better place. “And I’ve had the privilege to meet many of them,” he says.
Machine learning is the rational core of AI
In vogue at the present time is artificial intelligence, and there is an established view among commentators such as McKinsey that we are in the early days of AI – or, as Ghodsi maintains, of machine learning. Artificial general intelligence – where machines could reason, have self-awareness, decide that humans are bad for the planet and exterminate us all – he sees as irrelevant.
“Very few people are working on it [artificial general intelligence],” he says. “99.999% of the activity is on machine learning, whether that is for natural language processing or machine vision, or whatever. That other branch of AI is largely dead. Essentially, people gave up on it. People tried it in the ’60s, ’70s and ’80s. But they didn’t have much success, and that stopped.
“Most of the brilliant minds in the world are working on machine learning, or a good chunk of them. Just like in early 20th century, when Einstein published his seminal four papers in 1905, a huge amount of people went into working in physics.”
Read more about Databricks
- Spark processing engine more at home in cloud, Databricks CEO says.
- Databricks unveils data sharing protocol as part of data lakehouse.
- Databricks taps groundswell in lakehouse adoption.
But what does it mean to say AI (of the feasible kind) is in its early days – or early innings, as Americans are wont to say, using a baseball metaphor?
For Ghodsi, it is mainly about scale. He gives the example of Rolls-Royce as a Databricks customer that has created an “intelligent engine platform” using the supplier’s data lakehouse technology, which enables predictive maintenance for each of its engines. “Let’s say there’s an engine that’s been flying to Qatar a lot, so there’s sand getting in the engine and, even though it’s only been three months, it absolutely needs maintenance right away, but another engine, even though it’s been in use for six or seven years, it doesn’t need any maintenance at all,” he says.
“But that’s just one company. Imagine doing that with every piece of equipment that’s being built.
“Where AI – properly, machine learning – really excels is playing games, and people do get freaked out by that when they see it. But it is also a limitation. There are no nuances there. It’s very different from normal life, which is all about nuance, context, seeing patterns among things that are seemingly unrelated, and where the goal is unclear.”
Data lakehouse as a new category?
As a company, founded in 2013 now with 2,300 employees, Databricks has been steadily open-sourcing technologies developed in-house, such as Apache Spark, Delta Lake, MLflow and Koalas.
Recently, it has hitched its star to the concept of a “data lakehouse”, a portmanteau term that joins together the data warehouse and the data lake. It is said to deliver the data management and performance typically found in data warehouses with the low-cost, flexible object stores offered by data lakes.
Ghodsi defends the term. “When we mentioned it two years ago, all the reactions were around the name: ‘Do we need another water metaphor for data? This is getting silly’, and so on,” he says. “But we’d say it’s a technological breakthrough – pay attention to technology.
“Without the lakehouse, the world is divided into two different parts. There are warehouses, which are mostly about the past, and you can ask questions like ‘what was my revenue last quarter?’ whereas there’s AI on the other side, machine learning, which is all about the future. ‘Which of my customers is going to disappear?’ ‘Is this engine going to break down?’ These are much more interesting questions about the future. These two worlds have been separated into data warehousing and data lakes, which people use for machine learning. We combine it in one.”
Databricks has been raising a great deal of venture capital in 2021. Most recently, it announced a $1.6bn round of funding, associated with the data lakehouse concept, led by Counterpoint Global (Morgan Stanley), putting the company at a post-money valuation of $38bn.
Ghodsi adds that it has passed $600m in annual recurring revenue, and the company is growing by 75% year on year. And he believes the data lakehouse constitutes a new technology category. “I think the lakehouse will be the way of the future and 10 years from now you won’t see data warehouses, or rather they will be around, but just like mainframes are around,” he says. “I really think the lakehouse category is going to subsume the warehouse.”
However that prediction pans out, there are likely to be children coming to the West in the wake of the Taliban’s takeover of Afghanistan who will be the tech entrepreneurs of tomorrow. Probably in Silicon Valley – and perhaps via Sweden, Germany, France, or the UK.
