Databricks CEO on What Happens to AI Without Jensen Huang
Newcomer PodNovember 16, 202600:19:3817.99 MB

Databricks CEO on What Happens to AI Without Jensen Huang

What does AI lose without Jensen Huang? Databricks CEO Ali Ghodsi explains the stakes for NVIDIA, startups, and open source AI.

Ghodsi joins Eric Newcomer to discuss Jensen’s role in backing the AI ecosystem and why he believes NVIDIA’s strength can outlast its iconic leader. He also shares his outlook on the Anthropic IPO, why Databricks won’t stay private forever, and the pressures of rising AI costs.

Inside Databricks, Ghodsi explains how routing between AI models can lower costs and how he uses the company’s Genie AI agent to help run a 12,000-person organization. Plus, why he’s bullish on American open source models and what startup founders should understand about building billion-dollar businesses with small teams.

[00:00:00] NVIDIA will be... they'll do fine and they'll do great. And that's why you see their stock goes down like only 5%, which means the market is pricing in that they're fine. You can cut the cost down significantly, you can specialize the models. We specialize a lot of the models. I also think we're gonna see more open source models come out of the United States that are pretty good. My life has changed. I'm like 10x better CEO now than a year ago and it has nothing to do with me. We should have prepped the questions so I knew what was coming. I tried! You're like, no, come at me! Zero chance. I'm very, very happy.

[00:00:29] I mean, you know, Jensen had started to position himself as, you know, the godfather of open source. Something that you feel very strongly about, you know, acquiring companies dedicated to open source, funding them obviously, and just being their biggest champion. We've heard that here on stage. I think it was Igor River who was bringing up that point. Is this a blow to the open source AI movement?

[00:00:55] Yes. It's a blow to, I think, everything and pretty much every player in the space except direct competitors. You know, I mean, direct competitors obviously benefit. But yeah, I mean... Like AMD. You're saying chip makers, right? Yeah, I mean, if you're building a chip, I mean, I'm sure, you know, they knew him and they're sad about it as well just like the rest of us.

[00:01:15] But I mean, if you're a direct competitor and you have an iconic CEO of, you know, greatest company, no longer being the CEO, that's good for you at some level. But everybody else, it's bad for the ecosystem. It's bad for open source. It's bad for, you know, I mean, near clouds, you know, everybody. Right. Because they were kind of... They were the back kind of, you know, the backbone of the whole ecosystem.

[00:01:44] They would support all these companies and also companies that needed to sell. You know, many of them found a home at Nvidia. So yeah, I think it's all around just really, really bad. But having said that, you know, it takes two, three years to create these chips. So I think we're buying the chips today that were produced, you know, that they were working on two, three years ago. I think Nvidia will be fine the next two, three years. It's not going to be even affected. And three years from now, I don't even know. I mean, if you talk to AI people, they say, you know, you know, I don't know, they think there's 10% chance that the world doesn't exist in three years. So then, you know, what are we even talking about?

[00:02:13] So they're a great partner and I think Nvidia will be... they'll do fine and they'll do great. And that's why you see their stock goes down like only 5%, which means the market is pricing in that they're fine. Besides the hit, you know, AI valuations, what's the damage to Databricks specifically or where does it hurt you? I think Databricks is totally fine. And I think also Nvidia as a partner is fine. I don't think it really affects us directly.

[00:02:37] But I think the whole AI space, you know, I mean, they were just, you know, whenever there's any doubts around anything, any financials, Nvidia was there to say like, oh, don't worry, we'll walk back, stop that. We'll do this. We'll help with this. We'll help there. You know, it was just like a big supporter of startups and ecosystem. I mean, Jensen himself was the supporter of that whole ecosystem. So he was always there. So whenever there was like hesitation and things are in the bubble, he was there to say, oh, don't worry about it. I got their back. That's no longer there. So I think on the margin, it's going to start hurting a lot of the players.

[00:03:06] Databricks is a couple of layers removed from the chips directly. We run on top of the hyperscalers. So we're fine. You know, what's your view on the iconic CEO these days? I mean, you're an interesting case in that obviously a big personality looms large in a room, but you were not the founding CEO. You came, you were a founder, but you sort of came into it. How much do you think, you know, a company like Nvidia

[00:03:30] lives or dies based on a Jensen or these companies can rewire and find new charismatic leaders? I think the essence of the personality of the CEO continues to live in the company for long to come. So I think Jensen's legacy will continue to be there at Nvidia for decades to come. So he built an epic company and I think that's going to continues. Why do I say this? Look, Bill Gates was Microsoft, right? It's like he wasn't Microsoft. You could not disentangle the two and the two got disentangled.

[00:03:58] But still many of the competitive sort of aspects of how he thought about, you know, how they strategize, how they bundle things. All of those things kind of lived on for many, many years to come. Right. And Travis was Uber and so on. And that continued living. Right. So I think the culture of a company is a lot actually the personality of the CEO that is there for a long time. Over time, CEOs can change it. You can see with Steve Jobs and Apple and so on. So over time it changes and it depends on who the next CEO is. But by and large, I think it continues. It's like,

[00:04:29] it's all those little micro decisions that the company does every day and how they operate and how they think that has been put in places over, you know, many decades by Jensen that will continue the same way. I think you were on the record saying 2026 would be a terrible year to go public. I got so much shit for that. So many people reached out immediately after it was like, why would you say that? Don't do that. You're like ruining the party while it's going. Are you glad you're not going public right now? Yeah, I stand by what I said. I think I was correct.

[00:04:57] Any advice to Anthropic? I mean, Anthropic is an epic company. They're doing great. I'm sure they'll have an amazing IPO. The way it's priced, you know, I would be shocked if it doesn't pop like crazy. You think like above 2 trillion or? I wouldn't be surprised. I wouldn't be like, oh my God, I'm so shocked that it's like 2.5. No. All right. You know, but then it's stabilized at some point. But yeah, I think, you know, and I think also opening a fantastic partner also. I mean, what do you think Anthropic is worth today? I don't know. I'm not, you know, the markets,

[00:05:26] they go up and down and it's scarcity and it's floats and it's games and so on. But just after a while, like, you know, as they say in the short term, it's a beauty contest, but in the long run, it's going to be a scale that weighs the discounted cash flows of the future. So after a while, it'll like stabilize. You saw that with SpaceX, right? It like popped and then it went down and it was like really down and then now it's like stabilized or something that makes sense. I think same thing will happen with Anthropic and same thing hopefully with OpenAI. And what about Databricks? When does Databricks go public?

[00:05:53] That will happen too. As I've said, I will not disappoint. At some point, we will also go public. And you don't believe in this perpetually private, like, you know, strike. No, I don't believe in it. You know, I don't want to mention names, but I do have friends who believe in more perpetually private. I don't believe in the perpetually private because, you know, every year we provide liquidity opportunity to our employees. So we have to facilitate that for 12,000. Actually, it's like more like 14,000 employees because we do it also for X employees. So we kind of have to like simulate a whole marketplace more or

[00:06:22] less where we, you know, buyers and sellers and set the price. And it's a big lift for my finance department to do every year so that all these employees can get liquidity. So it's just there is something that was invented that can do that really, really well. It's the public markets. So eventually that will happen actually and we will do that. But people are don't care so much about it anymore at Databricks, the employees at least, because they already have that opportunity. Let's talk Databricks for a couple of beats here. You've taken this really sort of model agnostic

[00:06:51] approach. Can you talk about that? What's the reception from the customers? I mean, on one hand, you know, we're in this sort of, I guess, already post token maxing world where people do care about costs. On the other hand, you know, I'm running an AI conference and I don't even necessarily feel sophisticated enough to choose which version of Anthropic I need for a given task. It's a lot to ask a customer. How much are you going to make the decision for them? Like what's your view on sort of shuffling between models today? Yeah, we will release something called the Databricks

[00:07:21] AI Index where you can actually see we actually run one of the biggest routers. You know, over four quadrillion tokens go through us and you can get tokens from us for OpenAI, Anthropic, Gemini, Grok, and all the open source models. So we get a cross section of what everybody's doing and we've seen huge surge in open source usage over the, oh, since the last summer, just this summer, is a huge surge in open source. And so we released a smart router that automatically just detects if the question

[00:07:50] you asked is simple enough that it can route it to a simpler model, to an open source model, and you can get massive cost savings and we allow people to opt out. So if you kind of suspect that we're routing you and smart routing you and you feel like you got short change, you can opt out of it. And the opt out is like one percent or less. So people would want off to have because they're worried that you're getting some cost savings by routing them? No, someone might say, oh, well, wait a minute. I asked a very complicated question. This would be the bad scenario. I asked a very complicated question and I got a

[00:08:20] really dumb answer. You know, did you route me to a really cheap model that's not smart enough? That is basically non-existent. We're seeing in the data that there's like one percent, less than one percent that that happens. So basically we're able to take a lot of questions and route them through cheaper models and significantly compress the cost. So check out the Databricks AI index when it comes out. But yeah. When's that? Very soon. Okay. We're working on it. I mean, when your own employees use models,

[00:08:45] you're like, oh, we're a sophisticated tech company. We get to use the state of the art or what's your what do you hear from especially your coders in terms of their appetite for the best of the best? Yeah, I'm saying there's very little opt out. I mean, we told them like we ran A-B tests between the two groups as well. And yeah, people are totally fine. They're like, this is great. Because you have to contrast that with the cost situation. The costs are going up exponentially, but revenue is not going up exponentially for all these companies that are using these models. So something has to give. So they

[00:09:14] really want to compress that cost down. So it's very common. I mean, we see in our data over a majority of our customers have over three models. So you can just see there, right? So I think it's not super sticky and people are pretty flexible with that. What is sticky are the harnesses. You know, the harnesses are sticky. And are you building your own there? Yeah, we announced Omnigent, which is an Omni harness that uses the existing harnesses. So you can have codecs, you can have cloud code, you can have pi,

[00:09:44] you can have open code, and it just flips between these. Because we do see that otherwise the harnesses are sticky. And once people get stuck on a harness, they don't want to change. So that is very different from the models. The models are not sticky. People are like, oh, you know, anyway, it gives me a different answer. I'll take the cheaper one, the smarter one. That's changing very quickly. But the harness, once they get used to the keyboard shortcuts, their memory, how it acts, the ergonomics, they're like kind of, there's a stickiness there. But is that because they're not that expensive?

[00:10:11] That, I mean, would you spend a lot of money on a harness or it's just, okay, if you're giving me a harness as a free feature on some other service, I'm not going to change it very much? I just think over the last 40 years, software has had these amazing gross margins. Why is that? Well, it's because a lot of these software companies had this moat. What was the moat? The moat was that people, once they get used to a UI, it just, we're humans. We haven't developed super fast in the last years. Well, I used to work at Bloomberg. It's like people learn all the... Yeah, you know the

[00:10:36] Bloomberg Terminal, you learn, you know, shortcuts. Same thing here. You're an Android person, you're an iPhone person, you know Windows, you know Mac OS, you don't want to switch. And the harnesses have that. The models don't have that. It's just something you speak English to and it spits back English randomly to you and it's like different every time. So I think that, you know, there's a big difference between those two. So that's why we released Omnigent so that you can actually now multiplex between any of them. You, I think you were number three on the AI 100 we

[00:11:03] released today. What lessons would you give? You know, we have sort of a next generation, not that you're not the next generation, but a new generation of startup founders growing today. Yeah. Like what advice would you give them in building their businesses, both just growing teams? Like are you, are you, do you sort of think we should have, you know, 10 person teams driving billion dollar companies today? Or you think they should still hire top talent aggressively? Like what advice

[00:11:29] would you give to startups? I think they should strive at doing that. You know, I think for, look, when you were start, I was a startup. It's everything, everything works against you. It's just, everything is uphill. It's all just bad news all the time. It's like, you never get good news. And it's just, just like this vicious cycle. And then you see all these people that are like in the news and they're just crushing it and everything's going up, up to the right and so on. And you look at yourself and you're like, I just suck. And it's never going to work. Like, you know, and it's just

[00:11:54] bad news after. And then you have a board meeting and they yell at you. So, right? So that's, that was our life for three years or something like that. Right? It was like Databricks for three, four years. So, uh, because you're, for a while, you're dummies. You like haven't figured out. And then you're geniuses once you. Yeah. Then it's telling me, it's like, okay, these guys obviously had everything right from first moment. So, um, the way you're going to beat the big guys is to exactly bet on that secular shift where the

[00:12:21] company is completely automated and it's, it's doing things in a completely different way from the big guys. That's what you should bet on and you should go for it. No one's ever done it. No one's built a 10% company that has billions of revenue, but you all should strive to do that. And some of you will succeed in doing that. And that's what, you know, and, and you know, you need to make these bets that are asymmetric where people say, that's a bad idea. That's stupid. You cannot do that. You cannot have a 10% company that has billions of revenue, right? You need to have that. And then you need to prove

[00:12:47] them wrong. Uh, only then can you actually win because that's the big guys will then ignore you and say, yeah, it's a, it's a stupid strategy. You can't do that with AI. Uh, but you need to make that asymmetric bet and then you need to prove them all wrong. And that's how you become a great company. And I think some of you here will, you know, manage to do that. Some of you will fail then try again. Cause you know, for most of us, it wasn't our first startup either, right? It's not like I started a company and that was Databricks. It wasn't, I had another one. It didn't go so well. Uh, so try again and again and again, and eventually

[00:13:15] you're going to get there. It takes many years. Even for us, when I said those three years, we, we, we built the whole company on an open source project called Spark, which we started in 2009. 2016 was the year that we finally took off and people started saying, maybe there's a there, there 17, everybody was like, okay, yeah, definitely the business model works. Right? So that's eight years in. Are you agent-pilled? Like, are you using a lot of agents yourself to interact on your behalf? Yeah. So we launched something

[00:13:41] called Agent Bricks or agent, uh, uh, uh, you know, software that's called Genie. We launched it three, four years ago. We released a new version of Genie yesterday. So second generation of Genie. It's completely autonomous. It runs in the background. And, but the most important thing is it builds an ontology of your company's org chart, the people, what they're working on. You know, it has access to email. It has access to Slack. It has access to all your documents and all of that. And I run the whole

[00:14:07] company with this. Would you let it reach out to people on your own behalf? Yes. It's this, I run the whole company. Like my life has changed. I'm like 10x better CEO now than a year ago. And it has nothing to do with me. But what happens if somebody is like, oh, was this the real Ollie or was this the agent Ollie? First of all, it knows how I, so it has access to all my emails. So it uses my tone for everything. Right. So it's all in my tone. But you, you click approve or just, it's sending? I prefer, the way I use it is that it has a draft mode. So things end up in my draft outbox and I look at

[00:14:35] them and send them. Right. But it's all like prepared for me and it goes through all of those. Um, but that's like not the most amazing thing. The most amazing thing is like, you know, you have this org chart of the company and everything has to go up and down the org chart. We're 12,000 people. So this is a very long time to make decisions. But the way Databricks or any company actually works is like kind of horizontally, right? We're like customers give us feedback to product management, which gives it to engineering, which builds it, which, you know, gives it to marketing that puts it on websites and gives it to sales. They sell it. So really everything happens like this, but

[00:15:02] decision-making has to go up and down and it's like extremely slow. So with Genie and ontology that it has, we can start going more this way and it just speeds things up so much. Uh, so yeah, I run the company completely differently using Genie for everything. Um, you know, in every department, finance, engineering, I use it. Uh, and it's been a game changer. There's a risk of undermining your executives where it's like, I'm, I'm sort of pulling up the information about your own department and sort of knowing facts that you used to be the gatekeeper for. What, how do you see those dynamics?

[00:15:32] But they do the same thing. They're doing the same thing. I remember I needed to know at some point I was like, I need this statistic to be correct. I need to know the penetration of fortune 500. So I asked the person in sales that actually has that number and she said, Oh, sorry, I'm on a plane. I can't, I can't log into Genie. I was like, so you're just getting it from Genie. I can get it from Genie myself. So I got kind of irritated and I sent it to our CFO, Dave. And I said, Hey Dave, can you get me this fortune 500 foundation? And he sent me a screenshot of Genie. So everybody's using, so we're all doing the same thing, uh, in the company. So yeah,

[00:16:00] everybody has access to all of this and I think it's a good democratizer. So yeah, you can't hoard information and guard it and you have to be the gatekeeper of it. It's all there. Of course, this supports the access controls that we have in the company. So, um, you know, it's confidential information stays. You get to see everything. Um, pretty much everything, but not everything. Like we did the culture survey just now of 10,000 people. I don't know who answered what. Um, what is your scorecard on American open source right now?

[00:16:28] I mean, you sort of, I don't famously to me fame, like invested in open source models yourself and then sort of realized, Oh my God, I'm going to sink endless amounts of money into this. If I keep going, let's not spend it and stop. And then obviously there've been a lot of Chinese companies that have distilled models that American companies have, you know, gone to our U S inference providers to, to help them access and they access directly. Now we've seen a wave of funding partially from NVIDIA

[00:16:55] into state of the art open source U S startups. Are you optimistic about that effort or is open source really a Chinese thing that we take advantage of? Or how do you think about the actual open source models themselves? Yeah. So I think that, uh, this was obviously bad news. The Jensen, um, news was, is bad for open source, obviously. Um, but I'm an open source bull and I'm an American open source, uh, model bull. Uh, I also think we're going to have proprietary models. And I mean, Anthropoc is going to be fantastic. Open AI, these companies are going to do amazing,

[00:17:25] but open source is going to be there. Um, I think just distillation, which the original models were distilled of all the information on the internet, right? Um, distillation is just so powerful. It just works so well. It works even better when you're doing it on, you know, reasoning traces and so on. And it's nearly impossible to stop distillation, right? Because it's very hard to know. Anthropoc thinks they're going to stop people from distilling. I think fully stopping distillation is literally impossible because how can you discern what

[00:17:53] questions you're asking from a model is being going to be used for installation. Some of it is just legitimate usage and this stuff also leaks out, right? So as a result of that, I just think it's really, really effective to do reinforcement learning for specific use cases and compress the price down for, you know, use cases that are repetitive that you do again and again and again. And most of the work in the economy is in that class. Now there are things we need to do, like new drugs that we need to

[00:18:19] discover and, you know, diseases that we want to cure and, you know, financial models and so on, that where we need the latest, latest, best intelligence possible. That will remain and we will use frontier models. But as our smart router shows that I just mentioned earlier, you can cut the cost down significantly. You can specialize the models. We specialize a lot of the models. I also think we're going to see more open source models come out of the United States that are pretty good. So you're done. You're out of the model game for now. Yeah, uh, you have to change your mind or well, it's, it's a, it's a how much of your market

[00:18:49] cap do you have to spend on GPUs? Okay. Uh, to, uh, to, to be in that game. And has that come down or no? It has come down, but not enough for me to say, hey, let's do it. So either the database market cap has to go up really fast or the cost of training these things has to come down or both, uh, then we might change our minds. Yeah. All right. But, but yeah, I think I'm, I'm bullish on American open source. There will be models that are awesome that we're all going to use and we will continue using frontier proprietary models as well. Oli, great to have you. Thank you for joining us. Likewise. Thank you.