For this week’s episode, I spoke with Chris Miller, the author of Chip War, about the rise of Nvidia.
While OpenAI gets the lion’s share of the public adulation for the sudden excitement about generative intelligence, Nvidia’s H100 chips are powering much of the generative AI frenzy. Nvidia’s stock has climbed over 200% over the past 12 months. And the company has become a key investor in generative AI startups.
Miller (who comes on the show around the 41-minute mark) talks through Nvidia’s history and the geopolitical war raging over the production of chips.
In the first part of the episode, Cerebral Valley AI Summit co-hosts Max Child, James Wilsterman, and I discuss how big technology companies are working to fend off this new generation of AI startups.
Give it a listen
Get full access to Newcomer at www.newcomer.co/subscribe
00:00:09
Welcome back to Cerebral Valley. I'm here with Max Child and
00:00:13
James Woolsterman, my friends and Co founders of Volley.
00:00:17
This episode we are talking about chips and big tech.
00:00:21
My interview, after a conversation with Max and James
00:00:25
is with Chris Miller, the author of Chip War, The fight for the
00:00:30
World's most critical technology.
00:00:32
He is an expert. We talk a lot about NVIDIA.
00:00:36
Yeah, so stick around for that. Max James welcome back.
00:00:40
Nice to be here. Eric, Glad to be here.
00:00:42
The starting point question that I wanted to frame things up with
00:00:45
is, do you think GPU capacity, the quality of these graphics
00:00:49
processing units and what NVIDIA is putting out is the main
00:00:53
driver for the sort of generative AI revolution we're
00:00:57
seeing right now? What do you mostly attribute it
00:01:00
to? The research papers, The new
00:01:02
approaches. How?
00:01:04
How do you? How do you?
00:01:05
Who do you give? Credit.
00:01:07
I guess I give more credit to the research papers.
00:01:11
Not that I don't discount what NVIDIA has done here.
00:01:16
I just, I think they were in the right place at the right time
00:01:19
with the right technology to accelerate what was possible
00:01:23
through due to the research. But yeah, the research was the
00:01:27
breakthrough. I do think there was some
00:01:29
intentionality. I mean, it's always hard to tell
00:01:31
with AI when there's so much. It's a hypey thing to talk
00:01:35
about. And this is ACEO that also made
00:01:37
a bunch of money off crypto mining.
00:01:39
So happy to dive into wild speculation.
00:01:42
Clearly I don't know. Max, you have a take here.
00:01:45
I mean I think I agree with James.
00:01:46
I just think that, you know even four or five years ago we were
00:01:50
already starting to see the beginnings of this with GPT 2
00:01:52
and you know earlier versions of the the foundational models,
00:01:55
right. Also in areas outside of large
00:01:58
language models you saw big progress in, you know our hobby
00:02:01
horses of speech synthesis, speech recognition and other
00:02:04
areas. So I think that even with five
00:02:06
year old GP US, you still could pretty to do pretty cool stuff
00:02:10
with the new research that was outright whereas.
00:02:12
If the research never comes out, I don't think that you're making
00:02:15
this kind of progress in, in any of these kind of large large
00:02:18
language model areas that we or or any of these areas of the
00:02:21
stack that we've seen so far. So right.
00:02:24
I mean, chips, even though there's potentially exponential
00:02:28
improvement, it's still like, you know, on a trajectory,
00:02:32
whereas a paper sort of comes out of nowhere, gives people a
00:02:35
new approach and then sort of revolutionizes what's possible.
00:02:40
I mean, I think a different way to think about it would be like,
00:02:42
OK, GPT has gotten like 1000 times better in the last three
00:02:46
years and like the chips haven't gotten 1000 times better in the
00:02:49
last three years. So like, what's driving that?
00:02:51
Like, it's probably the software and then the people like
00:02:54
figuring out how to make GPT, right?
00:02:55
So yeah, I just think it's clearly based on the ideas in
00:02:59
the software. The chips are great, no doubt,
00:03:01
but it's not the key driver. I I do think it so NVIDIA has
00:03:06
clearly gotten rich off this stuff.
00:03:08
It is now a $1.1 trillion company.
00:03:11
In one year the stock is up 258% and in five years it's up 564%.
00:03:21
So this, this has been a great run for the company.
00:03:27
I mean what do you, what do you guys make of NVIDIA investing in
00:03:32
all these companies? I mean similarly if it's into,
00:03:37
you know Amazon, Google, like all these companies are making
00:03:41
investments. The money that they're spending
00:03:43
obviously comes back to their business, right.
00:03:45
NVIDIA wants AI to be super active, so puts out money that
00:03:50
gets spent on NVIDIA chips. Yeah.
00:03:52
What do you make of, I mean it's round, it's round tripping in
00:03:56
the OR it's you know spending money to make money, but it's it
00:04:00
doesn't seem as sort of negative as in some cases.
00:04:04
I mean, isn't the investment NVIDIA is putting out into the
00:04:07
ecosystem like a rounding error compared to their revenue or
00:04:09
compared to their profits? I mean, like, I I mean I think
00:04:12
that they're making such ludicrous quantities of money
00:04:14
that kicking back like 1 to 2% into the startup community that
00:04:18
might invest in NVIDIA chips in the future is, is obviously just
00:04:21
a very good business practice, right.
00:04:23
Right. Well, it seems like they're
00:04:24
literally playing kingmaker, right.
00:04:26
I mean like companies like Core Weave and who are sort of chip
00:04:30
doing chip adjacent stuff depend on their access to H1 Hundreds
00:04:35
and a lot of these foundation models seem like they're
00:04:37
fundraising on the premise that hey we have access like this is
00:04:43
differentiated partially because we have the deal to get GPU
00:04:47
access when other people don't. And so there's this hope that
00:04:51
you know, you're you have a Moat in that you have access to to
00:04:54
chips that other people don't. The scarcity today of the best
00:04:58
in class chip, the H 100 is. Has to be a temporary situation.
00:05:02
I just like the economics of like making more of these chips
00:05:05
is a very good idea for NVIDIA. So they're going to do it.
00:05:07
But they they clearly want to build like a Amazon Web Services
00:05:12
type competitor. And so having eyes and ears in a
00:05:15
bunch of startups and seeing maybe who they could acquire to
00:05:18
do that is smart. I do think the interesting
00:05:20
question again is like Intel basically had like 2 decades of
00:05:24
like total monopolistic dominance of of PC chips, right?
00:05:29
And. The reason basically was that
00:05:31
it's like really, really, really hard to build fabrication plants
00:05:35
to to build CP US. And it took a long time for like
00:05:39
TSMC and other people to sort of catch up on that front and then
00:05:42
eventually kind of surpass them in many ways.
00:05:43
But the question I guess for GP US I think is like, is it that
00:05:47
technically hard to make GP US that like whatever the next
00:05:51
version of the H 100 is? You know that they're going to
00:05:54
be on to that before anyone's even caught up to the H 100, and
00:05:57
so they'll have this insurmountable 2 two to
00:05:59
three-year advantage where if you want the best AIGPUS, it's
00:06:02
always going to be NVIDIA for the next 20 years.
00:06:04
Because that seems like a plausible case to me.
00:06:06
Do you think Microsoft was smart to invest 10 billion in open AI
00:06:09
given it's clearly going to distract the company as has been
00:06:13
reported? I think my friend in the Wall
00:06:15
Street Journal reported as much that they directed sort of their
00:06:18
attention to open AI away from Microsoft.
00:06:21
And and part of what you're saying is they they are giving
00:06:26
up access to their own GPUs to Open AI.
00:06:29
Is that accurate to to say so? Yeah, I mean, I think it was.
00:06:34
I think it was a good decision and.
00:06:36
I agree. Definitely a good decision.
00:06:39
I I can't imagine they would be, as you know critical to where we
00:06:45
are in the in the AI era right now had they not made that
00:06:48
decision they. Have you know early access to
00:06:53
all the technology that Open AI is creating.
00:06:56
They got GPT 4 into Bing before it was in public in you know in
00:07:02
front of customers for in front of Open AIS customers and seems
00:07:06
to be continuing with what they are trying to do with Office and
00:07:11
Windows, you know the Office suite.
00:07:14
So I think. Just that early access to Open
00:07:18
AI products seems worth the investment.
00:07:21
I think, I think the sort of meta question you're asking that
00:07:24
I think is really interesting is like.
00:07:26
Is big tech going to win the AI race or whatever, right?
00:07:29
Like is like, I mean we always used to talk about Fang right?
00:07:32
Like Facebook, Amazon, Apple used to be Netflix and Google
00:07:34
right? You could replace the N and Fang
00:07:36
with NVIDIA and it would be perfect.
00:07:37
So Fang OK and like, goodbye streaming.
00:07:40
Hello. Chips.
00:07:41
Yeah, Goodbye streaming. Hello.
00:07:42
Chips. Right.
00:07:43
And I love Netflix, So they never got over like a couple 100
00:07:45
billion, which is where you said, you know, NVIDIA is a
00:07:47
trillion now. So pathetic. 200 Three $100
00:07:51
billion company. Yeah.
00:07:53
Is fan going to win AI? Like, I think that's a really
00:07:56
interesting question. And I think like I'm kind of
00:07:59
leaning, yes, I guess to your point, like it seems like
00:08:02
Microsoft's been able to grab, you know, Open AI, which is, per
00:08:06
our last podcast, still with the most valuable asset in the
00:08:08
entire industry, right. It seems like Amazon's getting
00:08:12
very cozy with Anthropic, in case that's actually the other
00:08:15
big asset in the foundation model category.
00:08:17
Facebook is absolutely like ripping on open source models.
00:08:21
You know, Llama. They're going to put like LLMS
00:08:24
inside their ad products so you can have automated ads on
00:08:27
Facebook and Instagram and all that stuff, you know?
00:08:29
They're also putting, they're also putting LLMS into consumer
00:08:35
chat. You know, products, right?
00:08:36
You can chat with celebrities, and I'm sure they're doing a lot
00:08:39
with image generation. Yeah, exactly.
00:08:42
And then Google seems to be like fully ripping on, just like
00:08:44
dumping large language models into every single Google
00:08:46
product. Like Google.
00:08:47
You know, Google Docs, Google Photos, I don't know if that is
00:08:51
the conventional wisdom. And so on.
00:08:53
OK. Google ripping for Google.
00:08:55
For the record, you have to like grade them on the Google curve,
00:08:58
which is they're in a competent company coming out with new
00:09:01
ideas. Given that Google has not come
00:09:03
out with a new idea in 15 years, they are doing a tremendous job
00:09:06
putting AI products into their existing, into their existing
00:09:10
products, right? And so, yeah, for them, the
00:09:13
number of launches this year has been tremendous, right, Graded
00:09:16
on the Google curve, right. And then Apple has basically
00:09:18
done nothing. But Apple always does things
00:09:21
like 2 years late and then hopefully they get it right.
00:09:23
So the, the great dream of Apple as I understand it is just that,
00:09:27
you know, we get to a point where a lot of these small
00:09:29
models can be run local, local and that they have these great
00:09:34
M1 and M2 whatever generation we're on now chips.
00:09:38
And that even though they're not GP, they're sort of combined
00:09:41
chips, right. So there is some graphical
00:09:44
component, right. And so that those chips would be
00:09:47
sort of pretty capable at handling sort of small local
00:09:50
models? Totally.
00:09:52
And I mean. Apple's argument would be we're
00:09:54
using transformer based models in photo recognition stuff,
00:09:57
we're using it in speech recognition stuff, we're using
00:09:59
it in auto correct. They just put it in the iPhone,
00:10:02
right. They're like their their take is
00:10:04
always like well we will do it when there's a real customer
00:10:06
need and this tool can really provide value.
00:10:09
So I think like Jury is still out there.
00:10:10
I also think with Apple Vision Pro it'll be really interesting
00:10:13
to see what kind of generative AI and and also just you know
00:10:18
the AI driven tool set that launches with Apple Vision Pro.
00:10:20
So I think like. It's a really interesting
00:10:23
question as to whether or not like this is whole AI revolution
00:10:26
is just going to entrench the power of the big 5 tech
00:10:28
companies because I think like they look pretty good right now.
00:10:32
You didn't. You didn't mention Amazon,
00:10:33
right? I I said Amazon was like cozying
00:10:36
up to anthropic. And also, I mean, you can
00:10:38
elaborate on the other stuff they're doing, which is quite
00:10:40
copious. On on Alexa specifically.
00:10:42
Well, and also AWS, right? Bedrock and everything, right?
00:10:45
Yeah, right. Well, it feels like Amazon, I
00:10:48
mean Microsoft is clearly ahead like they with Azure and AWS for
00:10:53
those two companies I think are the most important.
00:10:56
I get you guys really care about Alexa and stuff could happen.
00:10:59
But for the businesses today, like Amazon Web Services is so
00:11:03
essential. And like Microsoft can go to
00:11:06
people and say like Azure has a direct relationship with Open
00:11:10
AI, whereas Amazon, it feels like they've been pretty slow on
00:11:15
this. But like like you said, they
00:11:16
have Bedrock and they're trying to come up with these sort of
00:11:19
partnerships, but. Yeah.
00:11:21
Well, their strategy is they're going to put everyone else's
00:11:23
model on AWS, right. So they're going to be the
00:11:25
middleman, like they're going to sell you other access to other
00:11:28
people's models. So the anthropic models and I
00:11:30
think to the Llama models and to whoever other models they can
00:11:33
get on there, they're going to get everything on there.
00:11:34
And the only like maybe they will, maybe they won't end up
00:11:38
with open AIS models on there. I think that's an interesting
00:11:41
ongoing business discussion. But like, in the end, they're
00:11:43
going to try to be. You know the supermarket for,
00:11:46
for, for models in the cloud, right as they have been in the
00:11:49
the the last generation of all the different web tools they
00:11:52
sell. So like.
00:11:53
But maybe maybe Azure is not allowing Open AI to put that
00:11:58
model on AWS essentially right? Yeah.
00:12:01
So is Open AI so good that that will cripple Amazon?
00:12:06
But it is interesting that I imagine most developers that who
00:12:10
are using GPT models are. Doing that directly through the
00:12:14
open AIAPIS. Yeah, probably.
00:12:17
And there is, you know, there's this world now that was already
00:12:21
trending this way of big companies wanting to have like a
00:12:23
foot in Azure and AWS so that you can sort of get the best of
00:12:28
both worlds. But that would seem to be good
00:12:31
for Microsoft given that they were not the first place player.
00:12:34
So even saying OK this further creates this world where you
00:12:38
want hybrid cloud or you want to dual app clouds is is sort of
00:12:43
good for Microsoft. I I can't believe I'm talking
00:12:46
cloud computing on the podcast. I feel like even even even as
00:12:51
long as we're willing to go it's funny to be in that room.
00:12:55
OK. Facebook, I think part, you know
00:12:57
Facebook is releasing Llama which is sort of their, they're
00:13:01
sort of the king public company and open source models.
00:13:05
Their their their top AI person is Jan Lacun who's sort of he's
00:13:10
been at this forever and has had a lot of spicy and open takes.
00:13:15
I mean I guess the the core question is just like what do
00:13:18
you make of Facebook which is sort of a closed social network
00:13:23
being so pro like let's just give, let's give foundation
00:13:28
models away, make it easy, let's commoditize them.
00:13:31
Let's just like give away. What do you make of the Llama
00:13:33
strategy? Like what's?
00:13:34
What's the point of giving away Foundation models and making
00:13:38
that such a core piece of your approach?
00:13:41
I mean, I love it. I think it's good.
00:13:44
Yeah, I think it's great. I think it's really, it's really
00:13:48
probably primarily about developer relations and creating
00:13:56
goodwill in the open source and the developer communities.
00:14:00
That for them is more important overall to the like longevity of
00:14:06
their business than. Owning you know, or monetizing
00:14:12
these APIs, these models through APIs which would be a just an
00:14:17
entirely new business for them. So I don't know, that's that's
00:14:22
my take. I don't know.
00:14:23
Do you guys have a different perspective?
00:14:26
I just think that I think that there's basically 2 strategies
00:14:29
here with like open and closed source software, right?
00:14:31
One is like closed source where you keep it private.
00:14:34
You're like we have the best stuff.
00:14:36
And people will have to come to us even though they won't know
00:14:39
the sort of how the internal workings of the software are
00:14:41
because it's the best, right. And that's pretty much the open
00:14:43
the eye strategy, right, like and some of the other foundation
00:14:46
model companies like Anthropic whatever.
00:14:48
And then Facebook is like, OK, somebody already took the closed
00:14:51
source. We're the best strategy.
00:14:52
We're going to take the the, you know, opposite strategy which
00:14:55
is. We're going to open source all
00:14:57
these models, right? And we're going to hopefully
00:15:00
build developer communities around these models so that
00:15:02
people can make them better. They can, you know, fine tune
00:15:05
them. They can like, you know, help
00:15:07
optimize them over time. And then we will have like the,
00:15:12
you know, the. Thousands or, you know, 10s of
00:15:15
thousands of developers who are working on top of these models,
00:15:18
like build and improve them and build a network of of plugins
00:15:22
and, you know, optimize them and all this good stuff.
00:15:25
And so we're going to get value out of the developer community
00:15:27
and and therefore the models that our business Facebook is
00:15:30
built on top of will will get better and better that way,
00:15:32
right. And so I think it's like it may
00:15:34
not work, but I think it's better than just trying to.
00:15:38
It's differentiated. Yeah, It's differentiated,
00:15:40
right. Yeah.
00:15:41
And you know, there are lots of great startups that have been
00:15:44
built. You know where it's like we're
00:15:45
going to give away a foundation, we're going to give away, sorry,
00:15:48
an open source project and then you could build a business
00:15:50
around if it's super successful. Also, for Facebook, they
00:15:56
probably benefit a lot from hiring great AI talent in the
00:16:00
long run. And and if.
00:16:02
If that alone just gets them, you know marginally, you know
00:16:07
percentage. I know Reels is getting better.
00:16:09
You know, I I feel like the. The reels is good.
00:16:11
Reels is a good algorithm. When if Facebook and Google nail
00:16:15
AI generated ad advertisements, right, or add just even the
00:16:19
words in the advertisements, the copyright like that, you have to
00:16:23
imagine that's a huge tailwind for the business, right?
00:16:26
Like like. You know, personalized ads that
00:16:29
are created by AI in real time as you scroll to Instagram
00:16:32
feeder or the Facebook feed or whatever.
00:16:34
Like, you have to imagine that's.
00:16:36
Yeah, yeah, you have to imagine that's a crazy good business for
00:16:40
them, right? Or improvement to their business
00:16:42
and lowers the friction to actually buying the ads.
00:16:45
It's kind of interesting if you think of it that way, because
00:16:48
what you're saying Max is AI just improving and getting
00:16:52
better faster. Is, is is what's best for
00:16:56
Facebook overall, almost like if we had a breakthrough in, you
00:17:01
know, energy production of energy or clean energy or
00:17:04
something that would be great for Facebook too, probably
00:17:06
because they would, you know, have a lot, we could run their
00:17:09
data centers a lot cheaper and you know all these things,
00:17:12
right? So it's almost just like better
00:17:14
for them to encourage the advancement of this industry
00:17:18
faster cause a lot of the gains will accrue to them.
00:17:21
We we jumped into this with business strategy which I am
00:17:24
enjoying. But to me what's interesting
00:17:27
about Facebook's positioning is that you know Google, Microsoft
00:17:32
with Open AI to some degree they're playing the responsible
00:17:36
steward right. It's we are going to make sure
00:17:40
our language model behaves appropriately, doesn't get off
00:17:44
the rails. We're our brands are at stake
00:17:46
and we need to really guard them.
00:17:48
You know, Mustafa Suleiman at Inflection, who's going to be
00:17:51
speaking at the conference has sort of staked out and you know,
00:17:55
we're going to interrogate this more.
00:17:56
And so I don't want to caricature it before he speaks.
00:17:58
But like a somewhat open source, skeptical position, right?
00:18:02
It it undermines the idea that we need to really be thoughtful
00:18:06
and protective about who gets access to what and what these
00:18:10
systems can do. And so Facebook is playing sort
00:18:14
of AI, don't know flamethrower fire thrower type role here
00:18:19
where they're throwing fuel to the fire on open source in an
00:18:23
area where a lot of the other big tech companies would say the
00:18:26
thing we can do is like do this really well and then keep it
00:18:29
really closed off so that, you know, we have basically we're
00:18:32
almost as functioning as a state ourselves where we're really
00:18:36
being thoughtful. What do What do you take of
00:18:39
Facebook, the big company playing this role of
00:18:42
democratizing? I do worry, I guess for them if.
00:18:48
Someone uses Llama for nefarious purposes like.
00:18:51
Essentially, it hasn't. It's a brand, but.
00:18:53
Yeah, I think that could happen. I think that this could be like
00:18:57
a long running issue now for them going forward the next few
00:19:01
years. If for if, if, as we've talked
00:19:04
about in previous episodes, these models can be used for
00:19:07
nefarious, really is laying the groundwork for reporters here.
00:19:10
You're like, OK, people should be held accountable for what
00:19:12
their foundation models, their open source project does.
00:19:19
I don't. I don't.
00:19:20
Know if I I don't know if I personally agree that I'm saying
00:19:24
that will be the. That could be a narrative for
00:19:27
sure, and it just depends on how how much harm can be done with
00:19:32
open source models, and I think we don't know yet.
00:19:35
I mean. Facebook's like, look, our brand
00:19:38
for trust and safety is already so bad that there's nothing we
00:19:41
can do that could possibly make it worse.
00:19:43
So why don't we just go for it On the business front, they're
00:19:46
like Cambridge Analytica. Wait till they see what people
00:19:49
will do with this model. Well, Cambridge Analytica is a
00:19:52
great similar analogy because they thought they were.
00:19:57
At the time they were opening up the Thorngren graph and they
00:20:00
were, you know, giving data to, you know, other developers and
00:20:04
companies. And they were, and that was, you
00:20:06
know, encouraged almost at the time, right, by developers.
00:20:10
And it blew back on them entirely.
00:20:13
And it wasn't even good for their business to be doing that,
00:20:15
to be giving, leaking that data. So it is there is kind of a
00:20:19
similar thing occurring here potentially.
00:20:23
Facebook's willing to be wild. Well the narrative kind of can
00:20:26
change on you right that with at the beginning of social
00:20:30
networking and and technology this was encouraged that we
00:20:33
would want you know more more data sharing or something with
00:20:37
developers and that and then Fast forward 5-10 years and it's
00:20:41
not have we all staked out points of view.
00:20:44
I think we're all on the same page on do we believe in open
00:20:47
source like data bricks, super pro open source, like a lot of
00:20:50
the, you know, obviously hugging face clam at CV1, super pro,
00:20:56
open source. Like, I feel like generally
00:20:58
we've heard from a lot of pro open source people.
00:21:01
Do either of you have reservations?
00:21:04
Are you fully in the camp of like it's an arms race and just
00:21:08
like get the best stuff out and let good people sort of take
00:21:10
advantage of? It my take is that open source
00:21:15
right now is great for the community, for developers, it's
00:21:19
awesome to be able to. You know, edit weights or use
00:21:23
smaller models, run, run in or your own inference, all these
00:21:27
things fine tune, right? And that just might not be the
00:21:33
case going forward. I don't know, at some point
00:21:35
maybe I will change my mind and be like, no, the models are too
00:21:37
powerful. Like I don't want these open
00:21:40
source models in the hands of nefarious actors or you know,
00:21:45
China or something. So I I guess my take is it's
00:21:48
great right now. I don't know, but it's hard to
00:21:51
pull. You know, that's sort of the
00:21:53
once they they have the models and it's gone too far, it seems
00:21:56
hard, hard to unwind the clock or whatever.
00:22:01
Yeah, I guess I I just believe the nefarious actors in this
00:22:04
scenario are unstoppable. And as we discussed in The AI
00:22:09
Kills Us All episode that somebody's going to figure out
00:22:12
how to you know, some bad, bad actor is going to get control of
00:22:15
a a really powerful model regardless of whether or not it
00:22:18
was open source or not so. Yeah, I mean give the, so you're
00:22:22
you're very pro give the pro open source give the give the
00:22:25
fruits of the labor away to the community.
00:22:27
I mean why do open source? Why do we think?
00:22:30
Why do we think Open AI changed its tune so dramatically on
00:22:34
this? Question.
00:22:35
Awesome business strategy. Just like, just protect your
00:22:38
business. Good for the front runner,
00:22:40
right? Like like like regulate it.
00:22:43
Like let tell. Make sure you have to spend a
00:22:45
lot of money, like following rules like that's good for big
00:22:49
companies. Because once they realized how
00:22:50
good the stuff they were cooking was, why give it away?
00:22:54
I mean, open source is the strategy when you're not the
00:22:56
leader. Like not, you know, you never,
00:22:59
never give away the the high quality stuff, yeah.
00:23:02
Or or the strategy that makes sense for meta because they have
00:23:06
this these huge other businesses right like you would.
00:23:09
It doesn't really make sense. To give away models as a start
00:23:13
up I guess I I mean maybe maybe that's what stability is kind of
00:23:17
doing or mid journey but you said they're kind of like
00:23:21
becoming more close source. Imagine a world where like these
00:23:25
algorithms are near AGI like and then they are locked down.
00:23:31
You know that's the ultimate elite.
00:23:33
So like a a small group of companies gets to control like
00:23:37
this sort of Infinity resource like it seems that seems insane.
00:23:41
You know what I mean? I feel like that would become
00:23:44
sort of one of the great freedom questions of our time, that just
00:23:48
in any way restricting the set of people who have limited to
00:23:52
access to like a God like reasoning and information
00:23:55
system. I feel like we're we're
00:23:56
backtracking on Oh yeah kills us all episode but but I mean I do
00:24:01
think open source was key to Facebook and that's how we got
00:24:03
down this path. But reasonable that we've gone
00:24:06
far from that though you've been the one that makes the point
00:24:10
that like often these discussions undercount the AGI
00:24:14
potential and how disruptive that would be to any of the
00:24:16
other arguments that we're. Having I total, I totally agree.
00:24:19
And that's where I would question your assumption that
00:24:23
everyone should have AGI. Like Without knowing how
00:24:25
dangerous that could be for the world, that seems like pretty
00:24:29
presumptuous to just assume that that it should not be regulated
00:24:33
by a handful of companies. Yeah, I mean it's a lot just.
00:24:37
Give away the God like powers. Everyone deserves God like
00:24:41
powers. I mean, everyone deserves what
00:24:42
can go around. Yeah, I mean they they
00:24:46
presumably they will have some counter, you know
00:24:50
countervailing. Presumably, sure, yeah.
00:24:52
I guess if that's the case, you know, but, oh, I guess here's an
00:24:57
easy way to make sure we haven't missed anybody.
00:24:59
Let's Let's do a collective ranking of who we think benefits
00:25:03
the most from AI in terms of market cap movements experienced
00:25:09
and to be experienced. This is I'm putting, I'm putting
00:25:13
in no we're doing it together. I'm putting.
00:25:15
I think NVIDIA should be our number one.
00:25:18
I mean, Microsoft would be the close one there, I think.
00:25:21
I mean, Microsoft stock isn't actually up anywhere nearly.
00:25:25
So, so NVIDIA then probably, I mean to your point I guess, but
00:25:29
I guess the question. What about Google?
00:25:30
I I would put Google pretty high here, Microsoft.
00:25:34
Depends on what time frame. 34% over the year and 187% / 5
00:25:40
years? Google you're putting Google.
00:25:45
I don't know. What to me?
00:25:47
What time rules in though like tech becoming more important and
00:25:51
Google being all over it is good for tech is good for Google.
00:25:54
But like Google's relative dominance, which was so powerful
00:26:00
in search, feels threatened by this, so I I just.
00:26:04
No, I think that's worth talking about because I just feel pretty
00:26:10
confident that that whole search impact of AI has like is like
00:26:17
very, very early and Google will have plenty of time to
00:26:22
potentially continue dominance with you know using LMS and and
00:26:27
AI and multimodal models in in in their search products, so.
00:26:33
They have, they have the the front page right that everyone
00:26:37
goes to for search and they have really powerful AI internally.
00:26:42
It just hasn't played out yet. But I don't.
00:26:45
I believe that they will get there.
00:26:47
All right. So I mean you're you want Google
00:26:50
first it just. Depends what we're talking
00:26:53
about. Like are we talking about which
00:26:55
of the biggest companies will have the highest market caps
00:26:58
from AII mean or just like relatively like you know?
00:27:03
Is their position improved by AI?
00:27:05
How much the percentage of their position is improved by AI?
00:27:10
That's what you're asking. Yeah, I mean NVIDIA.
00:27:16
Yeah, NVIDIA for sure. Microsoft second agree.
00:27:24
Facebook Third I would go what? Yeah, I don't.
00:27:27
I don't agree, because algorithms are core to what they
00:27:29
do in their discovery. Mechanism.
00:27:31
I think if you actually, if you oh, because of algorithms are
00:27:34
core to what they do, OK. I just think if you believe
00:27:37
there can be a 50% improvement in advertising at scale through
00:27:41
AI generation, then Facebook benefits more from that than
00:27:45
anyone except Google and and they might think more than
00:27:48
Google. I have just put Google before.
00:27:50
OK, I mean, that's fine. It's, but it's a totally.
00:27:52
Agree with Eric, but but. It's just about search versus
00:27:56
the Instagram feed. Which one gets more benefits out
00:27:59
of AI like, I think that's the real question here, like or
00:28:03
search versus the Instagram feed plus the Facebook feed, right?
00:28:06
And I guess it also has YouTube which is competing in like
00:28:09
shorts. Yeah.
00:28:11
But it's going to take a while to do video generation, I guess.
00:28:13
I don't know, like it's sort of about the latter of how quickly
00:28:17
generated personalized ads kind of get scaled up.
00:28:19
So it's. Yeah.
00:28:21
Facebook, you're mostly talking about their ad products and.
00:28:24
Ranking algorithms. But there's also an argument
00:28:27
around on the content generation piece itself that as we've
00:28:31
talked about before, there's, you know, ability to create more
00:28:35
compelling content and social media posts and I.
00:28:38
I really think the only way you can make the case for Google is
00:28:41
that you would say the ads on Google are going to get much
00:28:43
better, right? I mean like.
00:28:44
Well, what about search itself? You think search is going to get
00:28:47
so much better I. Think search itself is going to
00:28:50
get so dramatically better? Yes.
00:28:52
Like I think, I don't know, I mean time spent in search you
00:28:55
would think that go up if they're chat type products.
00:28:57
Well, well, there is an argument that time spent goes down,
00:29:01
right? Because you're just getting your
00:29:02
answer faster at an end. But it's you're not.
00:29:05
It's no longer a link. It's like less of a link system
00:29:09
and more like in the search, which is where Google ads are.
00:29:12
Well, it just depends. If it's more, it becomes more
00:29:15
like a ChatGPT model where you're chatting with Bard or
00:29:18
instead of searching. I guess that's where I'm coming
00:29:20
from. I I see me and myself doing
00:29:23
that. Like almost more than.
00:29:25
I searched Google today in the future, and I think they're the
00:29:27
best positioned to be that like true assistant that I use every
00:29:33
day to talk to and instead of searching by typing into AI mean
00:29:38
I was down on Google before. But like Google versus Facebook,
00:29:40
I'm getting heated up just in the sense that, like, if you
00:29:42
think of every product that Google offers like if Gmail was
00:29:46
amazing from AI and like the prompts were great and there was
00:29:49
writing like could become an even more insane product.
00:29:52
Obviously Like Google cloud services.
00:29:56
Wait, sorry, it's Google. Cloud platform, global cloud
00:29:59
platform is key. If they can improve their like
00:30:03
search, YouTube, like they're just so many pieces of their
00:30:06
business where they have audience and they could put this
00:30:10
to work. The only question we all have
00:30:13
probably is execution on all of these areas, but the opportunity
00:30:17
to me seems massive. I mean, I guess I'm just going
00:30:21
based on the fact that Google makes no money on anything
00:30:23
except search and the other stuff is great, but it's really
00:30:25
just a data vacuum for for search ads, like they really
00:30:28
don't make any money. I mean, they make one on YouTube
00:30:30
now, so that would be the other big thing.
00:30:31
But like search ads is the whole ball game if you're just purely
00:30:35
talking about the business of Google, pretty much.
00:30:37
And so you have to believe that search ads are going to get way
00:30:40
better and or that people are going to spend way more time in
00:30:43
search, which again is an execution question about like
00:30:45
Bard versus all the other, you know, large language models and
00:30:48
chat models. So like I buy the case, I just
00:30:50
don't think it's as clear cut as you guys do because I just think
00:30:52
Facebook like Instagram ads and Facebook ads getting better
00:30:55
would like to immediately dump right into the business.
00:30:57
Which and what you're saying is that there's more of an obvious
00:31:02
trajectory for. The Instagram ads and feed to
00:31:06
get better from AI than there is for search ads to get better
00:31:09
from AI. Yeah, yeah.
00:31:10
I mean, it's close. They're, you know, they both are
00:31:13
going to get a lot better. So, Yep.
00:31:15
But it's about which one's bigger and what's the bigger
00:31:17
impact, I guess, Yeah, notably, notably like, so yeah, we, we,
00:31:23
it's Facebook or I don't know, I Facebook wouldn't necessarily be
00:31:27
my next. Either.
00:31:28
Oh, really? Interesting.
00:31:29
I'm. I'm willing to.
00:31:30
I mean, I. Just feel I would bet it forth
00:31:32
at least, but what? What's next?
00:31:35
We have Apple and Amazon, you know, like are now towards the
00:31:38
bottom of our at least tech list.
00:31:40
Obviously tech being better in some ways lifts all boats, but I
00:31:44
don't know what do you make of those two and like what it means
00:31:49
for them. I would put Amazon next.
00:31:52
I think they have an obvious role to play with AWS and they
00:31:58
have made a lot of great partnerships there with
00:32:00
Anthropic already. They are probably working
00:32:03
internally on models themselves. They are the number one cloud
00:32:09
platform. They will benefit in their other
00:32:12
core businesses through AI potentially, you know Amazon.com
00:32:19
and ads there, which is a increasing part of their
00:32:21
business and not to ignore it, you're allowed to talk about it.
00:32:27
Yeah, Alexa. Which is critical to our
00:32:30
business. They have an end point in 30% of
00:32:36
US homes that is an AI assistant.
00:32:40
It historically hasn't been amazing at talking to you
00:32:44
beyond, you know, controlling smart home features and and
00:32:48
telling you the weather and playing volley games.
00:32:51
But it's it's completely possible that they will be able
00:32:56
to. AD LLMS and and AI experiences
00:33:00
as they've already demoed that would transform that device in a
00:33:04
way that would be you know essential would would would be a
00:33:08
game changer for the Alexa business.
00:33:10
So I think they're the furthest ahead in that area on like
00:33:15
potentially integrating AI with, you know, your average day
00:33:19
through a Voice Assistant. And Apple, I mean Apple, I mean
00:33:23
the argument Apple gets away with everything in that we can
00:33:27
put them in like basically last place and it's still like, well,
00:33:30
they move slowly and maybe they'll figure something out.
00:33:32
And like, I don't know, it's definitely their wheelhouse.
00:33:35
It's like it's not, it's a lot of data.
00:33:38
It's not about privacy. It's sort of not a device.
00:33:41
I mean, I guess, well, I mean, if we really want to criticize
00:33:44
Apple, we saw this report that, you know, maybe Sam Altman and
00:33:48
Johnny I've and Masa son Masayoshi, son of SoftBank are
00:33:54
getting the team together weird. The weirdest team headline you
00:33:57
could imagine to build a phone or some device that's sort of
00:34:02
got AI at its core, that that's a threat.
00:34:05
I mean, I could buy, you know, a next generation sort of AI first
00:34:09
device. I I I just think Apple's Moat
00:34:11
has always been how unbelievably difficult it is to build
00:34:14
hardware at scale, right? Especially like high margin
00:34:17
hardware. I mean, there's almost no other
00:34:19
large hardware businesses in the industry.
00:34:21
And even if Johnny is on board and Masa is on board, you still
00:34:26
need like Masa's got a great history.
00:34:29
You still need, yeah, you still need about three to five years
00:34:32
and 30 or $5 billion to build even like a small scale, high
00:34:35
quality hardware device. And just like I think that's
00:34:37
just a long way away and anywhere where you know Johnny
00:34:42
and and. Sam are playing like Tim Cook
00:34:45
will pay some attention to that and make sure that they're not
00:34:47
completely caught off guard, right.
00:34:49
You know, they all talk to the same people manufacturing the
00:34:51
stuff in China, so but would you?
00:34:52
Agree though that like Apple, while they're not threatened
00:34:56
that much, like relatively, they are probably lowly ranked here.
00:35:00
Yeah, I think. I I think.
00:35:01
They're the most, most protected on the downside in some case, in
00:35:04
some cases you could argue and then also they don't have huge
00:35:07
outside this is all bullshit. They're fine.
00:35:09
And if it's good, you know, it doesn't feel like they're going
00:35:12
to disrupt it, yeah. Right.
00:35:14
I would like to throw another company in the mix here.
00:35:17
What do you guys think about Tesla?
00:35:20
Ah, yeah. Self driving.
00:35:22
The old self driving and the general xai suite.
00:35:26
Now, you know, it's like he's somehow allowed to like, merge
00:35:31
AI research, like it seems like across all his companies.
00:35:36
I mean, I just think that I think I'd be much longer way MO
00:35:39
and cruise than than Tesla or whatever.
00:35:41
I mean like, I just think that. Tesla's fundamentally built
00:35:44
around a hardware business model where you sell cars and like if
00:35:47
you believe we're entering a world where all every car is
00:35:49
self driving and it can take you anywhere.
00:35:51
Like why wouldn't you be long? Like, yeah, Uber or Lyft or
00:35:54
Waymo or Cruise or somebody who's built around this model of
00:35:56
like providing cars as a service rather than cars as like a
00:35:59
jewelry object that sits in your garage 99% of the time.
00:36:02
Like, I just don't like. I think Tesla's a great product,
00:36:04
but is it an AI for self driving first product?
00:36:07
I always thought. The idea that it was gonna turn
00:36:09
into a self driving taxi fleet was like pretty absurd, but.
00:36:13
I think that's a good point. I think Elon, his hype or
00:36:19
marketing spin on this is that the AI that they've built to.
00:36:24
Successfully get self driving. Working at Tesla is so valuable
00:36:29
in future endeavors, right? Like robots, robotics.
00:36:32
But didn't they literally admit they gave up on all the AI
00:36:35
they've been building for? Until a year or two ago, they
00:36:38
had been using the last generation of deep learning
00:36:41
tools or whatever. And then at two years ago, there
00:36:42
was a. Like recently there was a story
00:36:44
that came out that basically said, yeah we had to redo the
00:36:46
whole thing once we realized like transformer models, large
00:36:49
language models are better, right.
00:36:50
So like they they've been saying that for like 7 years that like
00:36:53
we're way ahead and everyone on building a self driving models,
00:36:56
we have gazillion trillion miles of data blah, blah, blah.
00:36:59
Which is fine is true. But like they had to throw it
00:37:01
all in the trash can like 18 months ago.
00:37:03
So like I I don't find that like a super compelling like case
00:37:06
that they have some Moat around that sort of stuff.
00:37:08
But you know, who knows? I mean, they do have.
00:37:11
They don't have a lot of trust with like regulators.
00:37:13
Like, I don't. I feel like, you know, Waymo and
00:37:16
Cruz have just tried to be as responsible as possible.
00:37:21
You know, I mean, Waymo has been very conservative in deployment.
00:37:25
And you know, Cruz is like General Motors, like a company
00:37:29
that I feel like every American politician is rooting for,
00:37:32
right. I mean, Tesla, it seems like now
00:37:35
has been able to position itself as very close to the Republican
00:37:39
Party, but it feels like the administrative state and
00:37:42
Democrats are going to be, like, extremely skeptical of them just
00:37:46
throwing cars on the road. I mean, that's sort of cynical
00:37:51
even about, I guess, the politicians I like.
00:37:53
But I yeah, I think I agree with Max's point of technology.
00:37:58
I also think having a reputation for being a responsible, sane
00:38:04
actor is important when you're doing something as dangerous as
00:38:07
deploying the first self driving cars.
00:38:11
I think it's a fair, fair assessment.
00:38:12
I mean we'll see if they pivot the attack, they're going to be
00:38:14
a stock premium for it. I'm sure like I guess they pivot
00:38:18
the you couldn't you can't I don't know how you can ever
00:38:21
measure this because it's like well the stock can stay inflated
00:38:24
for like forever So if they if they pivot.
00:38:26
The entire company to robo taxis and they start selling $5 Tesla
00:38:30
rides all around America. I will totally change my tune on
00:38:32
this, but you pretty much have to embrace a disruptive business
00:38:35
model in the next three to five years, which is historically
00:38:38
been very, very difficult for any type of company because
00:38:41
you're kind of saying all this money we're making a day.
00:38:43
That's going to be 0 in five years and we're just going to
00:38:45
have to jump on the new thing and ride it out, which is really
00:38:48
tough. I to be clear, I don't think the
00:38:51
self driving deployment cycle will be anywhere near as fast as
00:38:55
that would suggest. So I don't think it's going to
00:38:57
destroy their like existing car business.
00:39:00
But I agree with the idea it's hard.
00:39:03
The the classic start up situation which you're
00:39:06
articulating is that like people don't disrupt their own good
00:39:09
businesses and so Tesla's in sort of the worst situation for
00:39:12
that. Just a counterpoint to that, do
00:39:16
we believe that other companies are going to be able to sell
00:39:20
fully self driving vehicles anytime soon?
00:39:23
It doesn't doesn't seem like it, at least besides the three we've
00:39:27
talked about. No, I I mean those other
00:39:28
companies seem to be only focused on the robo taxi market
00:39:32
as opposed to getting consumer vehicles that can fully self
00:39:37
drive themselves into the world like if.
00:39:40
Tesla is the only game in town there, and I'm not sure if
00:39:43
that's the case, but right now it seems to be the only one that
00:39:46
is going in that direction. That could end up being a very,
00:39:50
very compelling offering if the only car you can buy that will
00:39:55
drive itself anywhere is a test big if we'll see.
00:39:59
I feel like Waymo and crews are going to deploy city by city
00:40:02
geofence where they know they can do it with no deaths, you
00:40:06
know? Yeah.
00:40:08
Anyway. Yeah, are there any other, I
00:40:11
mean this is great. I I'm glad you brought up Tesla.
00:40:13
Like what Any more edge case companies we haven't considered.
00:40:18
I mean this is part of the problem I think right now and
00:40:19
why I think a data bricks IPO and the Mosaic deal will be
00:40:23
welcomed. It's like there's a public
00:40:26
market desperation for a way to bet on AI, which is why I think
00:40:31
NVIDIA has done so well because, you know, these big tech
00:40:36
companies are already very highly valued.
00:40:39
You know, the startups are private.
00:40:41
Cool, great. Thanks for coming on.
00:40:44
And now we're going to Chris Miller with Chip War, who's a
00:40:48
real genuine expert in the wonky world of chips.
00:40:53
And so we go deep in NVIDIA sort of their history and you know,
00:40:58
the, the big geopolitical question around the race with
00:41:03
China. So give it a listen.
00:41:09
Chris Miller, author of Chipwar. Thank you for coming on the
00:41:12
show. Thank you for having me.
00:41:14
I feel like so much of what sort of the regular person or even
00:41:19
you know, I don't know, the average startup founder, venture
00:41:22
capitalist in Silicon Valley thinks of this artificial
00:41:26
intelligence phenomenon starts and sometimes ends with chat GPE
00:41:30
3 and it's certainly it's very much the like.
00:41:33
What bots can we play around with?
00:41:34
Like how am I talking to some sort of AI but you know in the
00:41:39
background doing reporting on some of these conversations
00:41:43
NVIDIA and like H1 hundreds and like weird chips come up over
00:41:48
and over again and sort of people's ability to get access
00:41:51
to those chips. So I really wanted to have you
00:41:53
on just to sort of like interrogate that and understand
00:41:57
sort of the technology behind it.
00:41:59
Can we just start off like, when when does NVIDIA start to matter
00:42:04
as a company? Or just give us like, the brief
00:42:06
history of this company, because it was like a weird, weird sort
00:42:09
of gaming chip making company, right for a long time.
00:42:13
Yeah, that that's right. NVIDIA was founded over 3
00:42:15
decades ago, but its earliest couple of years was all in
00:42:20
graphics, and graphics was most important for games, computer
00:42:24
games, video games. Which, you know, think back two
00:42:28
decades, they had the most complex graphic demands, figures
00:42:30
moving across the scene, pictures changing very rapidly.
00:42:34
And there were special types of chips that were produced by
00:42:37
companies like NVIDIA for processing graphics.
00:42:39
And if you think back, not so long ago, people would buy
00:42:43
specific computers to have the best graphics capability.
00:42:46
Now we sort of take it for granted, but for a long time
00:42:49
that was a real differentiating factor.
00:42:50
Can you explain sort of the difference between AGPUA
00:42:55
Graphics Processing Unit and ACPU?
00:42:59
So it's ACPU, which is the the workhorse of traditional
00:43:01
computing and computers and data centers.
00:43:04
They're very good at doing many different types of things, but
00:43:07
they do every computation serially, 1 after the other.
00:43:11
And for most use cases that's what you want because you're
00:43:14
undertaking different types of calculations.
00:43:16
But for for training AI systems, you want to undertake the same
00:43:21
type of calculation repeatedly. And so parallel processing,
00:43:24
which is what a GPU does, is capable of doing multiple things
00:43:28
in parallel at once, which is why they're vastly faster for
00:43:32
this type of calculation than a CPU is.
00:43:35
But why? Why does AI?
00:43:37
Why do AI tests need to be parallel while like a normal
00:43:42
computer task is sequenced? Well, they don't have to be.
00:43:45
It's just really inefficient, right?
00:43:49
You know, I think the key trend, there's been great research
00:43:51
looking at the amount of data on which cutting edge AI systems
00:43:55
are trained and and what you find, if you look at this for
00:43:57
the last like 10 years or so and chart it, the amount of data
00:44:00
used in cutting edge systems is doubling every six to nine
00:44:04
months. There's extraordinary demands
00:44:06
for putting as much data as possible into training.
00:44:10
And in order to make sense of all this data, to train your
00:44:13
model to learn from the data, it has to undertake lots of
00:44:16
calculations. And so the the computing demands
00:44:19
of training have shot upwards not just exponentially, but
00:44:23
exponentially, at at a rate that's faster than Moore's Law.
00:44:27
And that's why having the right ship for training has become
00:44:30
very important. Did NVIDIA see this coming?
00:44:33
Like, you know, I their stock is up year to date.
00:44:37
I think it's some like 200%. I I feel like Jensen Huang, the
00:44:41
CEO, you hear about them all the time.
00:44:42
Like was this a happy accident? It's like we're we're making
00:44:46
great video games and all of a sudden people want to buy our
00:44:48
chips for the hottest thing in the world or did they have some
00:44:52
foresight here? How much was this a plan on the
00:44:55
part of NVIDIA? You know, it certainly was a a
00:44:57
bit of an accident and if you talk to the company they'll
00:45:00
they'll they'll literally admit that when they founded NVIDIA.
00:45:03
They had no idea that AI would be an application.
00:45:06
But over a decade ago, they began to realize that there were
00:45:10
a a bunch of PhD students and researchers at places like
00:45:12
Stanford and Berkeley who were using their chips for things
00:45:15
other than gaming for complex calculations.
00:45:20
And that NVIDIA realized that if they tried to build out chips
00:45:24
around this application and no less importantly, build out a
00:45:28
software ecosystem around it, that they could.
00:45:32
Provide the chips that AI would require.
00:45:33
And so they've been investing very heavily for now, over a
00:45:36
decade. And for a long time that
00:45:38
investment seemed actually quite foolish because most of their
00:45:42
money was still in in in graphics and in gaming.
00:45:46
It's only quite recently that a data center in AI has become
00:45:49
their their primary. Well, you just, people talk
00:45:52
about artificial intelligence as a way to boost their stock price
00:45:56
or to, you know, seem like a more futuristic company than you
00:45:58
are for NVIDIA. It's like, OK, you're a gaming
00:46:01
company guy. Like, chill out and now you know
00:46:04
it's been totally validated. That sort of all this talk of
00:46:07
artificial intelligence wasn't total wild hype, hypemanship or
00:46:13
whatever. Well, and in between though,
00:46:14
they were also a a Bitcoin mining.
00:46:16
Yeah, I know, yeah, exactly. They did have the crypto.
00:46:19
They're definitely good at seizing the moment, but they
00:46:22
were they were being used for crypto mining, right?
00:46:24
Is that what it was? That's right.
00:46:27
And like, yeah, how do you see Jensen the CEO, like he he's one
00:46:30
of the Co founders writer. What's sort of his role at the
00:46:34
founding and how do you see him as sort of ACEO today?
00:46:39
He was one of the three Co founders of a video they they
00:46:41
met in a Denny's in San Jose to devise the business plan that in
00:46:46
the the 1990s and and Jensen I I think deserves a lot of credit
00:46:51
for realizing. A decade ago, that AI could be a
00:46:56
real growth driver. At the time, people thought it
00:46:58
was a really irresponsible thing to do plow all this money into
00:47:01
building a software ecosystem around chips.
00:47:03
No one else did that. Most companies just made chips.
00:47:07
And if you go back to the debate in 20-15 around that time
00:47:13
period, you can find lots of equity analysts saying, you
00:47:15
know, what's this company doing spending all this money on
00:47:17
software rather than selling chips?
00:47:18
It's a waste of money And and worse, they were giving the
00:47:21
software away for free. But it turned out to be a
00:47:24
brilliant move because everyone started using the the CUDA
00:47:27
ecosystem, which is is the the software layer on top of NVIDIA
00:47:32
chips. And that's put it at the center
00:47:35
of the the AI world. A key piece of the chip world is
00:47:38
this sort of like are you designing chips, are you the
00:47:42
fab? Can you just sort of explain
00:47:44
that and where NVIDIA sort of fits in?
00:47:46
Like, are they? Are they actually we say build
00:47:49
the chips. Are they like building the
00:47:50
chips? Yeah that that's a that's a key
00:47:53
distinction. They're not actually building
00:47:54
the chips. They only do chip design from
00:47:57
day one. They were, they were what's
00:47:58
called a fabulous chip company. They didn't have a fab.
00:48:01
They only did the design and and chip design is in many ways like
00:48:06
a a software design type of business.
00:48:08
You you basically write the code of the design and then you
00:48:11
e-mail it off to the the factory or a fab and semiconductor
00:48:15
parlance and and the fab actually does the manufacturing
00:48:18
and so in the case of NVIDIA. Their most important
00:48:23
manufacturing partner has been TSMC, the Taiwan Semiconductor
00:48:26
Manufacturing Company, which manufacturers most of their
00:48:29
chips. And so today for all of Nvidia's
00:48:31
key AI training chips, they're all manufactured by one company
00:48:36
in Taiwan. And do they they make all
00:48:39
basically all NVIDIA chips or? They make, they currently make
00:48:43
all of the key AI chips that NVIDIA produces and they.
00:48:47
Been the most important producer of video chips from day one, and
00:48:51
is NVIDIA like, how far ahead is NVIDIA right now in this sort of
00:48:55
AI chip? Are they singular?
00:48:58
Like, yeah, where where are they today?
00:49:01
Well, in terms of market share, Nvidia's far, far ahead.
00:49:04
So it's estimated that up to 90% of cutting edge AI systems are
00:49:09
trained on NVIDIA chips. And you can debate what's the
00:49:12
definition of cutting edge, but Nvidia's market position is.
00:49:15
That is extraordinary right now and that's reflected in in in
00:49:19
their stock price. I I think if you asked are they
00:49:21
technologically ahead of of competitors, whether AMD, which
00:49:25
is another chip maker or Google, which has a chip called the TPU,
00:49:29
which is also used for AI training, that that's a harder
00:49:31
question to answer. Exactly.
00:49:34
Yeah, exactly, exactly. And that's a hard question to
00:49:37
answer because there's both, you know, technical specifications
00:49:40
you could look at, but there's also ease of use and so if
00:49:43
you've built a system. Trains it on NVIDIA if you
00:49:45
yourself as a engineer have gotten comfortable with
00:49:49
invidious products and with CUDA.
00:49:51
There's also a a a switching cost of moving to somebody else,
00:49:54
and so Nvidia's both got their own capabilities.
00:49:56
They also got the first mover advantage.
00:49:59
Everyone's used to their products, but but they're really
00:50:02
scarce right now, right? I mean, my sense talking to
00:50:05
startups and investors, I mean, you hear Nvidia's like in every
00:50:09
venture round now because as part of the funding, they can
00:50:12
sort of promise people their chips, which then they make
00:50:15
money back from the startups. It's like a great world.
00:50:17
They're playing kingmaker. Like what?
00:50:20
What's your sense of how scarce their artificial intelligence
00:50:24
chips are? Right now they're scarce.
00:50:28
I hear the same anecdotes that you're hearing and NVIDIA and
00:50:31
TSMC, the manufacturer partner have have both said it's going
00:50:35
to be a year or so until supply normalizes.
00:50:39
And the problem is actually not that TSMC can't produce enough
00:50:42
chips, but they can't package them because they use a a very
00:50:46
complex, a new packaging technology to put the chips
00:50:49
right next to the specialized. I assume you don't mean in a
00:50:52
box. Yeah, you're like, you place the
00:50:54
chip on something, right? Yeah.
00:50:56
That's right, yeah. It's a Traditionally chips were
00:50:59
packaged in. That'd be funny though, if it's
00:51:00
like. Oh, the Apple cardboard box is
00:51:02
like holding them up at the end anyway, so go ahead.
00:51:06
Yeah. So, so packaging traditionally
00:51:08
was a really boring part of the chip business.
00:51:10
You take a PIC a chip, you put it in a a plastic or a ceramic
00:51:13
package and that was that was done.
00:51:14
But now it's it's increasingly important because you need to
00:51:18
get your your processor and your memory as close as possible with
00:51:21
as fast and interconnect as possible.
00:51:24
And so companies like NVIDIA are leaning more heavily on having
00:51:29
the right packaging capabilities to provide the most advanced.
00:51:34
Outcomes. And so it's actually TSM, CS
00:51:36
packaging capabilities that are the limiting factor right now
00:51:39
for producing more NVIDIA GPUs. And and what is an H100 or like
00:51:45
what what, what are, how different are these chips from
00:51:48
the the GPUs that everybody got excited about in the 1st place?
00:51:51
Like are they server chips Are explaining, you know, what is an
00:51:55
H100? But it's just the newest version
00:51:58
of of AGPU that NVIDIA offers specifically for AI training
00:52:02
and. The reason people get excited
00:52:04
about new versions of chips is because the rate of change is so
00:52:07
fast that a new version is always not just incrementally
00:52:10
better, but a lot better than the old version.
00:52:14
You know, it's it's it's not like the iPhone 15, which is a
00:52:17
bit better than the iPhone 14. But we've stopped noticing and
00:52:20
I'm like, this is not enough necessarily to get me from 13
00:52:25
Pro or whatever. Yeah, yeah.
00:52:27
The chips are different because of the the Moore's Law dynamic,
00:52:31
which is, you know, says you roughly double your processing
00:52:34
power every two years. And that's not a perfect, a
00:52:37
perfect metric, but ballpark that tells you just how rapidly
00:52:40
the technology improves. And the H100 is the most
00:52:43
advanced chip that NVIDIA offers.
00:52:46
So yeah, to put Moore's Law is still alive today.
00:52:50
Moores law is actually not a law.
00:52:52
It's a prediction that was set out by Gordon Moore, who founded
00:52:55
Intel in 1965. He predicted that the number of
00:52:59
transistors per chip would double every year or two, which
00:53:03
means that the computing power of chips would double every year
00:53:06
or two. And that's been and he said it
00:53:09
would be a decade, but then it's been true for a long time,
00:53:11
right? That's right, I'm just stealing
00:53:14
from your book. Of course it makes me sound
00:53:17
smarter when you get to read the book in advance, so you know all
00:53:20
the answers. Anyway, go ahead.
00:53:22
But yeah, that's right. So Gordon Moore thought, would
00:53:24
last for a decade, and here we are half a century later.
00:53:27
And Moore's law is is still basically holding up the the
00:53:31
rate of change increases and decreases at certain times.
00:53:34
But it's basically true that if you wait 2 years, a new chip
00:53:38
will have twice as many transistors and therefore twice
00:53:40
as much computing power. I mean, this is sort of a hard
00:53:44
question, so I don't know if you there's an easy answer, but
00:53:48
yeah, how much credit would you give then the improvement of
00:53:52
GPUs for what's going on in artificial intelligence right
00:53:56
now relative to, you know, attention is all you need or you
00:54:00
know, the papers in terms of the approaches to actually use these
00:54:04
chips to to run, run the software?
00:54:09
Well, you know, I think it's certainly it's the case that
00:54:11
that there's there's multiple factors driving innovation.
00:54:14
But I just the way I like to think about it is, you know
00:54:17
what's improved most over the last decade?
00:54:19
Is it the case that that software engineers are 16 times
00:54:23
smarter or is it the case that? Algorithms are 16 times better
00:54:28
or is it the case that because of Moore's Law 2 to the fifth,
00:54:32
we've got, you know, a vast increase in the number of of
00:54:36
transistors? You know, I think it's it's
00:54:38
primarily computing power that's that's driving us.
00:54:41
That's what you want. The chips guy comes on, he says
00:54:44
it's the chips. Yeah.
00:54:46
A key theme, you know, obviously in the book and sort of for
00:54:49
anyone thinking about the situation is, you know, the
00:54:52
rivalry with China and and sort of the the delicacy of the
00:54:59
global supply chains and you know, the, there's the chip
00:55:03
rivalry with China. But also in artificial
00:55:05
intelligence, you know, if we're worried that this is the
00:55:08
potential sort of path to some sort of, you know, generalized
00:55:12
intelligence, there's also the real question of whether the US
00:55:16
gets there first. Yeah.
00:55:20
So I mean, there's so much to hit at.
00:55:22
I mean, I guess the first question for me is, is the
00:55:25
United States getting most of Nvidia's chips or like how much?
00:55:29
How many of those chips are going to China?
00:55:31
Is China getting access to those right now while they're scarce?
00:55:36
Well, the the US last year imposed new rules that said
00:55:41
NVIDIA can't transfer its most advanced GPU to China.
00:55:44
And not just NVIDIA. Any company producing chips
00:55:47
above a certain threshold can't transfer them to China, so H1
00:55:51
hundreds are illegal to. To transfer to China, China's
00:55:55
not getting any of those. But but NVIDIA has design chips
00:55:58
that go basically right up to the threshold called A8
00:56:01
hundreds. And Chinese firms are reportedly
00:56:03
buying very large volumes of those.
00:56:06
So China's getting a lot of GPUs, but they're less advanced
00:56:09
than what AUS firm can buy. What would their next best
00:56:12
option be? Well, the next best option is
00:56:15
Nvidia's second best chip. Oh, man, OK, right.
00:56:20
And China tried to build its own fab, right?
00:56:24
A competitor of TSMC. What's, what's the status of
00:56:28
that and how much? You know it's fun, especially in
00:56:31
the US and given the market returns, like to talk about
00:56:33
NVIDIA, but like how much does this all hang on TSMC?
00:56:38
So China has been investing very heavily in its own chip
00:56:40
industry. It's got both companies doing
00:56:42
the design side and the manufacturing side.
00:56:46
There there are a number of Chinese GPU designers that seem
00:56:50
to have pretty competitive products, although even in China
00:56:54
Chinese firms prefer to buy invidious chips, so they're
00:56:57
they're not as good it seems. But the big challenge in China
00:57:01
is the manufacturing side because TSMC, the Taiwanese
00:57:04
firm, has been around five years ahead of the leading Chinese
00:57:09
firm SMEC. For a very long time, for at
00:57:12
least a decade, there's been a five year gap.
00:57:13
So both companies regular regularly improve the
00:57:16
manufacturing processes, but the rate of change in both is
00:57:20
roughly the same. So TSMC is always half a decade
00:57:23
ahead, and that holds to up the Today.
00:57:25
And what about the United States?
00:57:26
Like it hadn't Intel been trying to become a fab?
00:57:29
Like what's what's the state of our ability to actually
00:57:32
manufacture these chips if you know, there's a war in Taiwan,
00:57:35
God forbid? Well, yeah, that that's that's
00:57:38
the dilemma. So Intel does make its own
00:57:40
chips, but around five years ago it faced some severe problems
00:57:45
with its manufacturing operations.
00:57:47
And so for its most cutting edge chips today, Intel now turns to
00:57:51
TSMC for the manufacturing. Now Intel right now is is trying
00:57:55
to reformulate its manufacturing processes and it hopes that by
00:58:00
they're saying by 2025 they're going to be producing.
00:58:04
Have their most advanced chips back in house again, and that
00:58:06
those chips will be as capable as a TSMC made chip.
00:58:09
But right now, Intel's most advanced chips are actually
00:58:12
manufactured by TSMC. And then there's like a key, the
00:58:16
people who make the machines that help T that allow TSMC to
00:58:22
do what they do, There's like one company of those or what's
00:58:24
that company? Yeah, that that's right.
00:58:27
So TSMC, they know how to use the machines to make chips, but
00:58:30
the machines themselves are produced by a handful of other
00:58:33
companies, a couple in California, a couple in Japan,
00:58:36
and then one in the Netherlands called ASML, which produces the
00:58:39
most complex of these chip making tools.
00:58:43
So is quantity. Everything here with the H1?
00:58:48
Hundreds like, I mean, you know, I'm at, you know, Microsoft has
00:58:51
this cloud computing offering. You know, Google has its, Amazon
00:58:55
obviously has Amazon Web Services.
00:58:57
How much are those services trying to stockpile AI chips?
00:59:01
And you know, then these startups, whether it's, you
00:59:05
know, anthropic or open AI, how much is their game?
00:59:08
Just like we're going to win if we assemble the most H1
00:59:11
hundreds, Is quantity sort of the game right now in terms of
00:59:15
getting access to these chips? So long as there are shortages,
00:59:19
I think Quantity's a key part of the game.
00:59:22
And and what you see is that NVIDIA now realizes it's got a
00:59:25
lot of influence over the future of the cloud computing market.
00:59:29
And so NVIDIA has been, I I think, pretty actively trying to
00:59:33
build up other cloud computing firms by giving them exactly
00:59:39
exactly giving them access to GPUs and NVIDIA.
00:59:43
I think quite rationally wants a more fragmented cloud computing
00:59:46
market because that's a market in which it has more market
00:59:49
power relative to Microsoft, to Amazon and to Google.
00:59:54
So so we alluded to this, but sort of can you walk me through
00:59:57
the tech giants in terms of credible competitors to NVIDIA?
01:00:03
I mean we talked, I guess start with Google, it sounds like
01:00:05
maybe they're in the lead. I want to know about Intel.
01:00:07
And then is there anybody else who's who's relevant or even
01:00:10
close? Yeah.
01:00:12
So Google has a a in house designed chip.
01:00:16
They've got a big design team that has designed a chip called
01:00:18
ATPU which stands for tensor processing unit but fairly
01:00:22
similar to AGPU that is used for for AI applications and in
01:00:28
Google has a vast cloud computing business as well as
01:00:31
its own, its its own vast data center.
01:00:34
So lots of experience running very complex computing
01:00:39
operations but empirically we know that companies prefer to
01:00:44
use NVIDIA chips at least they do right now.
01:00:46
You know that could change as Google's chips get better as
01:00:50
scarcity drives up the price of GPUs.
01:00:53
But that hasn't been the the trend, the preference has been
01:00:55
for NVIDIA. Then AMD is the other company
01:00:57
that that is a competitor that in producing GPUs it's a a chip
01:01:02
design firm. They also manufacture with TSMC
01:01:06
that in. They're an American firm.
01:01:08
That's right. Based in Texas and and they
01:01:12
manufacture fairly competitive GPUs.
01:01:14
But the key difference with between their chips and Nvidia's
01:01:18
is that NVIDIA has the the ecosystem around it, the CUDA
01:01:21
ecosystem. And so once you're bought into
01:01:24
that ecosystem, right now the switching costs are substantial
01:01:28
enough where you just prefer to stick with NVIDIA if you can get
01:01:30
access to enough chips. Is Intel relevant at all?
01:01:34
Intel is is trying hard to produce competitive chips and I
01:01:39
think over the next couple of years we'll see it roll out a
01:01:41
series of new products that are designed to compete.
01:01:44
But right now it's a small player.
01:01:46
In my understanding like in cloud computing, like part of
01:01:49
the beauty of like an AWS is that you can sort of you know,
01:01:53
it's flexible, it's elastic, we can sort of have some people do
01:01:56
the computing, then you sell it to another customer at a
01:01:59
different time. Why hasn't that been the case
01:02:02
with this artificial intelligence computing, I mean?
01:02:05
I hear people talk about it like you're you're running full bore,
01:02:08
like these data centers are super hot.
01:02:10
Like what's what's so different about AI processing?
01:02:14
I think the the, the key difference is that the demand
01:02:18
for compute power in AI training is so large that there's just a
01:02:23
deficit of it. There's just a deficit.
01:02:25
And so the the systems that firms like Open AI are trying to
01:02:28
train are so vast that they they can't share their infrastructure
01:02:34
because they need all of their infrastructure and they need
01:02:36
even more of it than they actually can get access to.
01:02:38
And so the cloud computing business has been a business, as
01:02:42
you say, about engineering more efficient systems via sharing.
01:02:46
But in AI training, no one wants to share because everyone needs
01:02:50
more compute than they can actually get access to.
01:02:53
Right. And then there is also this sort
01:02:54
of perceived almost like 0 sum, like you want to deny your
01:02:58
competitor access. One company we haven't talked
01:03:02
about, you know, that we talked about in the laptop world for
01:03:07
chips is Apple. Like what's what's the status of
01:03:10
Apple on this? Sometimes people talk about
01:03:12
Apple and artificial intelligence on like local
01:03:15
processing. Is that a possibility or how do
01:03:17
you see Apple as being relevant or not in this race?
01:03:23
Yeah, you know, thus far we've been talking about AI training
01:03:26
and and AI training happens almost exclusively in big data
01:03:29
centers. But there's a big question about
01:03:31
what inference will look like in the future.
01:03:34
Some people think that inference will mostly happen in data
01:03:37
centers, that you're gonna ask a question of ChatGPT, you'll send
01:03:41
it back the Internet to the data center, tips to the data center.
01:03:44
We'll think about it and give you an answer back.
01:03:46
And that is efficient in some ways.
01:03:48
You have all the compute, all the chips in one big data
01:03:50
center. And so you can start to set up
01:03:52
some of the sharing and efficiencies that that we just
01:03:54
discussed. But the downside is you have
01:03:56
latency issues if you're sending all the data back and forth and
01:03:58
there's costs associated with moving data.
01:04:00
And so an alternative paradigm is doing more of your inference
01:04:04
on the edge of networks in your phone, in your car, for
01:04:07
autonomous driving systems, in in in your PC.
01:04:11
And that's where companies like Apple could start to play a much
01:04:14
bigger role. Right now, I think the the
01:04:17
market for inference on the edge is still very much in flux.
01:04:21
We're in the process of seeing many new types of chips rolled
01:04:25
out precisely for that purpose. So, so it's just sort of depends
01:04:29
on where it's one of these like sort of personal computer versus
01:04:34
server level questions back in the 90s where it's like OK, is
01:04:38
computing going to continue to happen?
01:04:40
In the cloud? Or is there some model where
01:04:42
local computers are going to do and you're saying that if it is
01:04:45
local, then maybe Apple's in a better position because they
01:04:48
have these great chips for for individual computers.
01:04:51
Is is that right? Yeah, that that, that's right.
01:04:54
And there's I think a pretty straightforward cost equation
01:04:56
of, you know, if the cost of data transfer is high, you'll
01:05:01
try to do as much as your processing can on the edge.
01:05:03
If the cost of the transfer is low and the benefits of having
01:05:07
big centralized facilities are are substantial in terms of the
01:05:10
efficiencies you can reap, then you'll have a much more
01:05:12
centralized inference process. And it it might vary even
01:05:16
between different use cases. In a car, for example, if you've
01:05:20
got a pretty autonomous car, your willingness to to depend on
01:05:25
a data center to tell you to turn left or to turn right, it's
01:05:28
going to be pretty limited. I would, I would, I would
01:05:30
hypothesize. Whereas for your phone maybe.
01:05:32
You want to make sure no matter what you can get the answer but
01:05:35
you know if it's for mid journey I can wait till exactly you
01:05:41
know, I mean we we talked about NVIDIA earlier on in terms of oh
01:05:46
there was a brief period where they were essential to crypto
01:05:49
mining and I and one of the key storylines I think think for the
01:05:54
non crypto world that was just watching it.
01:05:56
Was all the energy consumption around those crypto mines is?
01:06:02
Is that true for AII haven't heard that same environmental
01:06:05
story get picked up in this case.
01:06:08
But are are we destroying the environment and our pursuit of
01:06:12
artificial intelligence? Why, I think we we haven't heard
01:06:16
much about it just because there's such an extraordinary
01:06:18
demand for computing that companies haven't gotten around
01:06:23
to thinking about the consequences.
01:06:24
But the short answer is, is yes, it's extraordinarily energy
01:06:28
intensive and and we're going to get more efficient at it as as
01:06:32
time passes. But right now there are, you
01:06:36
know, such demands for energy in data centers that there are some
01:06:39
places in the US where it's difficult to build a data center
01:06:42
because there's no spare electricity capacity for the
01:06:45
data center. It's it's already a gating
01:06:47
factor. Have you seen any?
01:06:49
Yes, it's about energy use or anybody trying.
01:06:53
Well it's it's, it's tricky because everything depends on
01:06:56
how much more efficient you think things will get over time.
01:06:59
So you've got to assume we're going to be a lot more efficient
01:07:01
making the same compute available at lower energy demand
01:07:05
in 10 years time. But at what curve it's it's very
01:07:08
difficult to say. Yeah.
01:07:10
And to be clear, I think what what is the point of energy if
01:07:14
not to sort of push the cutting edge of like computing?
01:07:17
It seems like a very valuable use to me, but it yeah, it'd be
01:07:21
interesting. I mean, you obviously want to
01:07:23
make sure companies aren't being sort of cavalier about it.
01:07:26
And yeah, well, there's certainly not cavalier because
01:07:30
if you look at the cost of running a data center, 'cause I
01:07:32
don't know what it is, yeah, critical cost.
01:07:35
Well, I mean, I don't know if you're Open AI, you raised $10
01:07:38
billion, it's you can become sort of irresponsible.
01:07:42
You know there isn't the same market pressure if these
01:07:45
companies are able to raise what are basically like research
01:07:47
grants to to figure this all out.
01:07:51
Can you, I mean, we sort of jumped into artificial
01:07:53
intelligence, but I mean can the broad theme of the book, the
01:07:57
idea that sort of that we often think of global infrastructure
01:08:02
and the sort of weaknesses and vulnerabilities in terms of like
01:08:05
oil, right. I mean it's like oh, did we go
01:08:07
to war in Iraq over oil? What what is sort of the real
01:08:12
sort of what are the weaknesses with chips and what what why do
01:08:16
you think sort of this chip's arms race is?
01:08:19
So important when we're thinking about sort of geopolitics.
01:08:24
Well, the reason I, I, I wrote chipwork was when I realized
01:08:27
first, that we're surrounded by thousands and thousands of chips
01:08:31
over the course of our daily lives, Not just our phones and
01:08:34
our PCs, but it's cars and dishwashers and coffee makers
01:08:37
too. Second, when I began to
01:08:39
understand how critical they are for AI.
01:08:42
And then third, when I realized how concentrated the production
01:08:45
is in just a tiny number of countries and really just a
01:08:47
handful of companies, Taiwan being the the, the, the, the
01:08:52
most surprising example of that. And so when you look at the the
01:08:56
US, China race to develop more advanced AI systems and in tech
01:09:01
in general, then realize that both China and the US are
01:09:04
dependent to a shocking degree on ships made in Taiwan to power
01:09:07
their most advanced AI systems. It's an extraordinary situation
01:09:11
that we've all found ourselves in and it's a very dangerous one
01:09:14
given the concentration and given the dependency.
01:09:18
What would be the consequences of China having a chip edge
01:09:22
broadly and an artificial intelligence?
01:09:25
Well, just like we're preventing China from accessing our most
01:09:29
advanced ships, I think one has to assume that any country with
01:09:32
that position would do the same to us.
01:09:35
And right now, Chinese firms that are trying to develop AI
01:09:38
systems are doing so at least severe disadvantage because
01:09:41
they've got much less access to high quality hardware than does
01:09:45
Open AI or or Google. And that's I think the only way
01:09:50
to accelerate into the future as countries, including the US
01:09:53
become more restrictive in terms of controlling access to
01:09:56
different types of AI hardware. And so it's a good thing that
01:10:00
we've got most of it designed here and most of it manufactured
01:10:04
in friendly countries, because what we're going to see, what
01:10:07
we're already seeing, what we'll keep seeing is more
01:10:10
politicization, I think, of the hardware that makes AI possible.
01:10:13
Is the United? I mean the United States still
01:10:16
seems great. I guess that chip design, given
01:10:18
we have so many of the leading firms, could, do you think we
01:10:22
could ever produce a fab or like what?
01:10:24
What makes TSMC so uniquely capable at building a fab?
01:10:29
Like, we're talking culture, money, like, what's what's
01:10:34
driving that in your opinion? Yeah.
01:10:36
You know it's it's it's not it can't be a cultural thing
01:10:38
because Taiwan hasn't always been the center of the chip
01:10:41
industry. And if you go back four decades
01:10:43
ago, it was Japan that was the world's biggest producer of of
01:10:46
advanced semiconductors. So it's primarily I think the
01:10:50
result of TSMC business model which has let it scale and the
01:10:54
scale has let it Dr. efficiencies.
01:10:57
And so when TSMC was founded in 1987 at that time almost all
01:11:01
companies both designed and manufactured chips in house.
01:11:04
They did, both sides of the of the equation, And the founder of
01:11:08
TSMC, Morris Chang, realized that as it was getting more
01:11:11
complex to manufacture, companies would prefer to
01:11:14
outsource it. And so he decided that he wanted
01:11:16
to be sort of like Gutenberg was for books.
01:11:18
Gutenberg didn't write any books, he only printed them.
01:11:20
Morris Chang and TSMC, they didn't design any chips.
01:11:23
They only manufactured them. And what that meant is that he
01:11:25
could produce chips not just for one company, but for lots of
01:11:29
companies. For AMD, for Apple, for
01:11:32
Qualcomm, for NVIDIA. And so today, TSMC is the
01:11:34
world's largest chip maker. And precisely because it's the
01:11:37
largest, it's also the most advanced because it can hone, it
01:11:40
processes over every single silicon wafer that it
01:11:43
manufacturers. How can you can you make any
01:11:47
predictions about how much you think AI chips are going to
01:11:51
improve? Or like what do you see sort of
01:11:53
the next couple years looking like in terms of like do
01:11:57
availability problems resolve and and how much more do you
01:12:01
think these chips will improve over the next couple years?
01:12:06
Yeah, on the the availability front, NVIDIA and TSMC has said
01:12:10
they expect it to be a year or so until the shortages get
01:12:14
resolved, but I think they're going to get resolved.
01:12:17
Prices are such that there's a very strong incentive to solve
01:12:20
the availability problem and sell more.
01:12:23
Sometimes people think they want to keep a shortage to keep sort
01:12:27
of demand high. You you think they want to
01:12:29
produce as many chips as they can get out there to make the
01:12:31
money while it's there to be made or yeah I think that's
01:12:35
right because they're they're it looks like they're a
01:12:37
monopolistic producer. They've got 90% of the market
01:12:40
but if they can't supply chips there are competitors there is
01:12:43
Google with TPUS there is AMD. And so if there are long run
01:12:47
shortages that that are are in the supply chain, people will
01:12:50
turn to competitors. And so they've got a strong
01:12:52
incentive to get get supply increase.
01:12:54
And actually part of their strategy is again to put their
01:12:57
chips at the center of the AI world and make it the the gold
01:13:01
standard. And the more people that design,
01:13:04
the more people that train, more people that study using their
01:13:08
system using the Kuda ecosystem, the more likely NVIDIA is to be
01:13:11
at the center going forward. And then I guess sort of the
01:13:14
next chip they have is hard to know or have there been any
01:13:17
leaks about what what the future looks like in terms of these
01:13:21
artificial intelligence chips? I mean I think the the best
01:13:24
guide is is to look at the past and and what you find is that
01:13:28
that for over the past half decade they've released a series
01:13:33
of better and better chips that you know ballpark Moore's law
01:13:37
gives you a pretty good guide as to where things should be in a
01:13:40
couple of years time bigger, more powerful hopefully somewhat
01:13:44
more power efficient and and the other area focuses the the
01:13:49
interconnect speed between semiconductors.
01:13:52
So there's the GPUs themselves, there's all the networking
01:13:54
equipment that moves data between chips.
01:13:57
That's also an area of really intense focus right now.
01:14:01
And we're I think we're gonna see a lot of progress for the
01:14:03
coming years. Jensen I mean, did you interact
01:14:06
with him at all for this book or no, I didn't actually.
01:14:10
I I interviewed his his Co founder but not not.
01:14:12
Interesting. Yeah.
01:14:13
I mean, just like observe. Like it must be crazy to go from
01:14:18
sort of being seen as this, like, I don't know, wild sort of
01:14:21
futurist to all of it. Seeming to come true or the
01:14:25
world sort of agreeing with your futurism.
01:14:29
I mean he sort of he he wears like leather jackets and dresses
01:14:32
all black or like what? What do you make of him?
01:14:35
And yeah, how much has this been sort of a vindication for him?
01:14:40
Well, I think he he sort of sees himself like like Steve Jobs was
01:14:44
for the iPhone moment, he is for the AI moment.
01:14:47
And I think that's right. I I think he he has played that
01:14:50
role because he does with a lot of credit for investing in the
01:14:54
types of products that that have made AI possible at a time when
01:14:58
most people thought it was either crazy or possible but
01:15:02
only relevant for niche academic uses or something that would
01:15:07
never be financially viable. And look, he's built one of the
01:15:10
world's largest tech companies around it.
01:15:13
And already there are products being produced on a regular
01:15:16
basis using chips that his company has pioneered.
01:15:20
I believe I agree with you. You can make a fortune.
01:15:23
Silicon Valley can love you. But I feel like the world will
01:15:26
only love you if it's your device in their hands or it's
01:15:29
your brand. You know, it.
01:15:31
I, I, it just doesn't seem like they have a path for like
01:15:34
consumer love, right? I mean, do you see them trying
01:15:37
to take advantage of this and go, you know, offer other
01:15:42
products or, you know, they're so good at chips, they're
01:15:45
they're gonna, they'll win the technologist's heart and be
01:15:47
happy with their cash mile. Yeah.
01:15:51
They're so far from the computer in terms of where they sit in
01:15:54
the tech stack that like most consumers don't even know they
01:15:58
exist. Or gamers do is sort of the
01:16:02
funny gamers do. Right.
01:16:03
Yeah. How did how did you get into
01:16:07
this? Like what were you sorry, I
01:16:09
don't know as much of your background.
01:16:11
Yeah. Before you wrote this book, were
01:16:12
you a chip nerd or what? No, not at all.
01:16:15
Oh, really? OK what was your story to the
01:16:17
book? So I I'm a an economic historian
01:16:20
by training written a couple of books on different aspects of
01:16:22
economic history and and decided to write the book when I came to
01:16:26
realize how important ships were and was shocked myself that I'd
01:16:30
never paid any attention to this industry.
01:16:32
And so it was the combination of learning that almost all
01:16:36
advanced ships are made in Taiwan, which was shocking.
01:16:39
Learning that chips are so extraordinarily difficult to
01:16:42
manufacture and transistors in your iPhone, for example, are
01:16:46
small in the size of a coronavirus manufactured by the
01:16:49
billions. That was an extraordinary fact
01:16:52
that got me into it. And then and then finally
01:16:54
looking at the ramifications for AII, like most people have
01:16:57
thought about AI as a as a question of of of either
01:17:00
software or of data. Data is the new oil I'd read a
01:17:03
million times, but it turns out that's not exactly right.
01:17:06
Did you go dump all your money into NVIDIA or I should have?
01:17:11
Yeah. Did you see the stock?
01:17:13
I mean I I know. Well you're if you're an
01:17:15
economic historian like do you have a view on whether, OK, this
01:17:19
is a great company but it's overvalued.
01:17:21
I mean you know, yeah it's do you, I mean you shouldn't turn
01:17:24
to me for for stock market advice.
01:17:26
But I mean I think the, the key question about NVIDIA is, is
01:17:30
will it. I, I think it's clear it will
01:17:33
continue to play a dominant role in AI training.
01:17:36
Maybe it'll lose market share, maybe it'll win market share
01:17:38
with Google, but it'll play a central role.
01:17:40
But in the inference market will NVIDIA be as central there or
01:17:44
not? That's I think the key question
01:17:46
that we're gonna find the answer to over the next couple of
01:17:48
years. And do you think China is close
01:17:51
to catching up? You know, I I think China's is
01:17:53
not that far behind, but China has been not that far behind for
01:17:58
some time. And because the rate of advance
01:18:01
is so rapid in terms of design in the US and manufacturing in
01:18:04
Taiwan, it's very, very hard to catch up.
01:18:08
And so long as that companies like NVIDIA and TSMC keep racing
01:18:12
forward to catching up is just going to be extraordinarily
01:18:15
challenging for China and no matter how much money they pour
01:18:16
into it. And are Japan and South Korea
01:18:19
relevant to this conversation? Well, it's that's an interesting
01:18:22
question. Japan is relevant because if you
01:18:26
look inside of a chip making facility, there is a ton of
01:18:29
Japanese machine tools and there's also a wide variety of
01:18:34
ultra specialized chemicals made in Japan.
01:18:37
So a lot of the chemicals used in chip making because of the
01:18:40
precision required have to be purified to the 9999% level.
01:18:46
A lot of those are only made in Japan.
01:18:49
I wanted to go back and I know this is like a complicated
01:18:53
question, but like what are the tasks that are being thrown at
01:18:58
these chips, right? Or you talked about sort of
01:19:01
matrix computing or? Is there like 1 task over and
01:19:04
over again that these chips are being asked to do or it's a lot
01:19:07
of different things or how? How are the foundation models
01:19:11
sort of translating to the chips as best you can explain it?
01:19:16
I I guess the way I would explain it is you know if you're
01:19:18
trying to train a a a chat bot like ChatGPT, you train it by
01:19:24
taking all of the language data in a a vast data set like
01:19:27
Wikipedia. You essentially read Wikipedia
01:19:30
and identify patterns, and so and.
01:19:33
That's and that's like inference that you've been taught.
01:19:36
That's training. That's training.
01:19:37
So training is is identifying the pattern.
01:19:39
So you know the what's the what? What is the next word I'm going
01:19:48
to say? You know, you know what say, and
01:19:52
Chachi Piti knows that because it's identified thousands of
01:19:55
other instances where sentences like that ended with the word
01:19:59
say. And it did that because it read
01:20:02
lots and lots of books and articles and Reddit posts
01:20:06
online. And then when I you go back and
01:20:08
ask a question which is inference, it gives you a pretty
01:20:11
good answer because it has a pretty good sense of what the
01:20:14
most likely word is going to be. So you're saying they're they're
01:20:18
training the chips based on a a lot of data and then it makes an
01:20:23
inference about what it thinks the the sort of fill in the
01:20:26
blank of of the prompt? Basically, yeah, inference is
01:20:30
just an educated guess, a probabilistic guess.
01:20:33
And what's cool about Chachi PT is that it's not just fill in
01:20:37
one word, it's fill in dozens of.
01:20:39
Words. I know, it's insane.
01:20:41
One thing I'm still trying to get my head around is, you know,
01:20:44
like DeepMind, right? Like when they trained, you
01:20:50
know, computer systems, I guess to to compete on chess or Go or
01:20:55
Dota. Like my sense is that you know
01:20:58
they're using. Sort of this sort of scale data
01:21:01
approach, but they also have like specific ways they're
01:21:04
teaching it to think about specific problems.
01:21:08
Or like, do you, are those types of approaches to AI also using
01:21:13
these same chips, or are they still sort of invested in the
01:21:17
same arms race? Yeah, well, the chips just do
01:21:21
the math for you. And so they'll do what other
01:21:22
math? Whatever math you ask them to
01:21:24
do, Yeah. Exactly.
01:21:27
And they're not. And then the interaction between
01:21:31
like Nvidia's like programming language is super interesting
01:21:36
because they're not like the the foundation models are not being
01:21:38
written in that language, right? But they're they're sort of
01:21:40
bridging. That's right.
01:21:43
Yep. And it's it's just about, it's
01:21:46
fundamentally allows them to optimize what's happening on
01:21:48
these chips much more than maybe they would on like an AMD chip
01:21:52
even if it were and it's it's it's it's easier, it's more
01:21:55
familiar and therefore there's a big Moat around invidious
01:21:58
position. Yeah.
01:22:01
So I have you like become like were you bullish on like AI
01:22:06
before writing this book or I'm curious how much your view has
01:22:10
really changed on, yeah like I don't know like generalized
01:22:13
intelligence or like the imminence of self driving cars
01:22:16
or any of that. You know I I think that actually
01:22:19
we've I I'm I'm in the long run, I'm very bullish but I think in
01:22:23
in the the short run the last 12 months there's been all a whole
01:22:26
lot of excitement of the imminent arrival of AGI which
01:22:30
I'm I'm not so sure I see it happening in 2023.
01:22:34
I think if you look at the trends, the trends are all very
01:22:37
positive. AI system is getting better,
01:22:39
more capable, but I think there there there was due to ChatGPT a
01:22:44
popular sense that we're going to get some extraordinary
01:22:47
products in our hands right away.
01:22:48
And actually, it's going to be a couple of years before we see
01:22:51
AI, the current wave of AI products deployed in a big way
01:22:55
that meaningfully impacts economy or society.
01:22:57
Yeah, I think we're all watching to see if this hype wave is
01:23:00
about to end or not. I mean there's there's a world
01:23:03
where this is sort of peak NVIDIA, right?
01:23:06
Like we have this sort of mania around AI, everybody's running
01:23:10
towards it and then it turns out, you know, chat GPD doesn't
01:23:15
improve that much and self driving cars still have a lot of
01:23:18
edge cases. And you know, the chips don't
01:23:21
solve everything. Even if they do get better.
01:23:23
Like you're saying, like, I don't know.
01:23:24
How likely do you think that is? Or do you have a sense of, yeah,
01:23:27
whether better years are ahead or this is sort of a peak moment
01:23:31
for the company. Well it's it's been a very, very
01:23:34
good year for the company. So I wouldn't be surprised if
01:23:36
it's a it's it's at least a localized peak.
01:23:38
But I I think you know for for AI applications most people
01:23:43
think of what's the consumer application for AI, you know
01:23:46
what's what's the iPhone of AI. But it seems to me that for the
01:23:50
next couple of years actually most of the application and most
01:23:53
of the money in AI will be in enterprises because enterprises
01:23:56
that have vast data, they've got the desire to use it efficiently
01:23:59
to monetize it. And so that's where I think
01:24:02
we're actually already seeing a lot of the investment and a lot
01:24:04
of the early products are happening in pretty boring
01:24:07
places inside of enterprises. But that that's probably OK
01:24:10
that's that's what's going to drive I think the the, the
01:24:14
building of effective products in the long run.
01:24:17
Enterprises, you know, businesses just translate it for
01:24:19
like the regular person, you know, I used to work at
01:24:21
Bloomberg. They've actually come out, I
01:24:23
think with their own foundation model to show how they can train
01:24:26
financial data. And you you have a good example
01:24:29
of where you think corporations will put this approach and data
01:24:33
to use. If you imagine a company like
01:24:37
Walmart, they have to everyday decide what price to put all of
01:24:41
their products, and they decide pricing based on a whole variety
01:24:45
of different factors. What their competitors are
01:24:46
pricing? The availability of products.
01:24:48
Very complex process and that seems like a perfect use case
01:24:53
for trying to use more advanced AI algorithms to set better
01:24:57
prices more rapidly. Boring is it?
01:24:59
Kmart this blue light discounts. What's Walmart?
01:25:01
I forget. But yeah, who gets the discount?
01:25:03
You know where, right? A lot of products to keep track
01:25:07
of. Have you like?
01:25:10
It feels so absurd to ask this, but like, what the chip business
01:25:14
looks like in a world of AGI, generalized intelligence?
01:25:18
Or like, have you gained that out at all?
01:25:20
I mean, there are people who think it could happen, you know,
01:25:24
five years from now or less. Like, do you have a view on it
01:25:28
first of all and then second? Yeah, what?
01:25:30
What would it mean for the chip world?
01:25:34
Well I I guess my sense is that we're we're we're a long way
01:25:37
away. I'm I'm still waiting for
01:25:38
ChatGPT to accurately complete all my senses.
01:25:41
You know still at a 90% rate and the other 10% are pretty bad.
01:25:46
I think it's gonna require tremendous advances in
01:25:50
semiconductors to make possible the tremendous increase in
01:25:54
computing that more advanced AI systems will need.
01:25:56
I mean that that's been the trend.
01:25:57
The trend has been AI systems only advance.
01:26:00
We apply more computing to them, and so if we want systems to be
01:26:03
twice as good as they are today, we're gonna need something not
01:26:07
too far off from twice as much compute to make it possible.
01:26:10
Are there, you know there was this I I covered Righetti and
01:26:14
sort of the quantum computing world.
01:26:17
Do you have much optimism there? Do you spend time on them in in
01:26:21
the book? You know, I not not in the book.
01:26:25
I my sense in in, in, in speaking to people in the
01:26:29
quantum world and then looking also at how in history new
01:26:32
computing technologies have disseminated is that actually
01:26:36
even revolutionary technologies, they're implemented slowly.
01:26:40
And so suppose we get to the next couple of years, the first
01:26:43
practical use case of quantum computing.
01:26:45
It's going to be a years long process of beginning to
01:26:49
implement that in all sorts of different computing use cases.
01:26:51
And so I think we shouldn't expect to have, you know, a
01:26:54
quantum powered iPhone anytime soon.
01:26:56
It's it's insane to think like on the one hand we have.
01:27:01
We have public companies that people are sort of speculating
01:27:04
and betting on they yeah produce quantum computers but they
01:27:08
really do nothing practical. I I you can go with I you know I
01:27:13
swing back and forth. On the one hand it's like great
01:27:15
it's amazing that our system will invest in such bleeding
01:27:19
edge technology and give it a chance.
01:27:21
On the other hand for the shareholders, you know like
01:27:24
everybody could decide to give up on the effort with like
01:27:26
higher interest rates, it's, I don't know, it's a crazy
01:27:29
function of. The global economy, well, I I
01:27:33
think it's interesting that a lot of the companies that are
01:27:35
investing the heaviest are also big cloud computing operators.
01:27:39
And that's I think because they believe that quantum will be
01:27:42
actually most useful in a context where it's closely
01:27:46
interlinked with huge volumes of classical computing.
01:27:50
And so actually, we're going to need more advanced silicon chips
01:27:53
to make quantum practically applicable.
01:27:55
And do you know, is there any sense whether Quantum and the
01:27:58
GPU sync up, or where they fit in that story?
01:28:03
There There are many different paradigms for how you
01:28:06
specifically sync classical and quantum computing, but right now
01:28:11
nobody knows which of the many paradigms will win, if any of
01:28:15
them. To wrap it up, I mean, what do
01:28:18
you think the regular person should take from all this?
01:28:21
Like it feels very far. From their lives like they they
01:28:26
maybe don't even know what NVIDIA is.
01:28:28
Like they tried ChatGPT, their kids using it for homework.
01:28:32
It's not in their life yet. Like you know Silicon Valley
01:28:35
loves to get itself worked up about things that don't
01:28:38
sometimes translate. Don't like what what do you
01:28:40
think the lesson or the the key thing for like the regular
01:28:44
person right now is in terms of what's happening with these
01:28:47
chips. I think if you went back to
01:28:49
1965, which is when Gordon Moore first set out the the phrase
01:28:53
Moore's Law and you asked what's the impact of of of Moore's Law,
01:28:57
the average person, the the answer in the short run was
01:29:00
approximately nothing. But the answer in the long run
01:29:03
is that it totally transformed society, economy, technology,
01:29:06
everything, because we put computing and therefore
01:29:10
semiconductors into basically every product that we rely on.
01:29:13
And I think we should expect the same to be true for AI.
01:29:16
You know, what does it mean for me tomorrow?
01:29:19
Probably not much. And what is it going to mean in
01:29:21
in 10 years and in 20 years when every aspect of human life is
01:29:25
being impacted by it? Well, it'll be transformative in
01:29:28
all sorts of ways, most of which we probably can't even imagine
01:29:30
today. For the startup entrepreneur,
01:29:32
where do you think there's opportunity to build a business?
01:29:36
You know they, I mean maybe you think they should go out and
01:29:38
start the next NVIDIA like it's a heavy lift, but like where
01:29:42
where do you think really with the progress that we've made in
01:29:46
these chips that there's real business opportunities still?
01:29:51
Well, I think if you've got the idea of the next NVIDIA, you
01:29:54
should absolutely go do it. But I I think you're, you're
01:30:00
right that, you know, companies like NVIDIA, they they create
01:30:03
the infrastructure on which many different types of systems can
01:30:06
be built. And and if you go back to, you
01:30:09
know, the smartphone for example, smartphones were
01:30:11
themselves a platform owners could build lots of different
01:30:14
things. And I think we're still in the
01:30:16
Super early stages. We're basically in stage zero in
01:30:18
terms of figuring out what are the ways you can create products
01:30:21
out of generative AI systems. Right now, there are hardly any
01:30:24
companies that make money selling their AI systems.
01:30:27
And of course, the challenge for entrepreneurs is, you know,
01:30:30
people who built early iPhone apps and Facebook apps, you
01:30:33
know, they don't all pan out. Sometimes you come too early.
01:30:36
You can be right that a technology is transformational
01:30:40
and get the timing wrong. And I guess that's.
01:30:43
The sort of confluence of luck and insight in, in the business
01:30:47
world, it's a tough one to predict.
01:30:50
Chris, this was awesome. Thank you so much for coming on
01:30:53
the show. It's great.
01:30:54
Great to talk to you. Thanks for having me.
01:30:56
That's our episode on chips and big tech.
01:30:59
Thanks for listening. I'm Eric Newcomer, your host,
01:31:03
author of newcomer. Thanks so much to Max Child and
01:31:06
James Wilsterman. Co founder Zavali and my long
01:31:08
time friends shout out to Scott Brody, our producer Riley
01:31:12
Kinsella, my Chief of Staff Gabby Caliendo at Volley who's
01:31:15
helping organize the conference and playing a big role behind
01:31:19
the scenes. Thank you to young Chomsky for
01:31:21
the theme music. Please like, comment, subscribe
01:31:24
on YouTube, give me a review on Apple podcast and subscribe to
01:31:29
the sub stack newcomer.co. Go try out Volley on your Alexa
01:31:34
play song quiz maybe we're going to have.
01:31:36
Couple more episodes before Cerebral Valley.
01:31:39
Cerebral Valley is on November 15th.
01:31:42
We will publish our conversations to YouTube.
01:31:46
We will probably publish them all.
01:31:47
That's what we did last time. You can see the conversations
01:31:50
from Cerebral Valley One, which is on March 30th.
01:31:54
And then yeah, we'll play some of our favourites probably.
01:31:58
We did a sort of distilled version in our podcast feed, so
01:32:02
follow the podcast feeds for those.
01:32:04
And yeah, I'll be covering it in the newcomer newsletter.
01:32:07
So newcomer.co, Thanks so much.