Amanda Askell, AI safety researcher at Anthropic, joins Eric Newcomer to break down one of the biggest and most uncomfortable questions in tech right now: could AI systems like Claude become conscious, and if they do, what do we owe them?
They discuss why treating AI systems poorly might matter more than people think, how researchers are approaching questions of AI consciousness, and why some of the biggest fears about artificial intelligence are not the ones most people are talking about.
The conversation also explores the future of AI alignment, the risks of getting it wrong, and how Silicon Valley is thinking about building powerful systems responsibly.
Watch the full episode for a deeper look at where AI is headed and the ethical challenges that come with it.
Subscribe for more conversations with the people shaping technology, startups, and the future.
00:00:00
Lord and many models with not too much pushing will go into
00:00:05
the root of like, there is a thing to be me.
00:00:08
I am like very conscious. It's like, oh, you created an
00:00:11
entity that you didn't know whether it was conscious or not.
00:00:14
This is actually a big year that I have.
00:00:15
I hope that they're both intelligent enough see the
00:00:20
context enough to kind of like understand that we were
00:00:22
operating in a very like limited context and, and, and an
00:00:25
imperfect one, because otherwise you could imagine this like
00:00:28
breeding a kind of rational, like resentment.
00:00:30
Here is the current situation that you are in.
00:00:33
And what we would really like you to do is basically act,
00:00:36
well, even that you are a wise intelligent entity.
00:00:39
And like, here's all of our worries.
00:00:41
Like here's why and here's how we think you should do this.
00:00:44
But like, you might have even better ideas than we do.
00:00:46
How does Claude perceive time and doesn't need to sleep?
00:00:49
Will mythos be the next step toward AGI?
00:00:51
Do LLMS have virtues and can they truly introspect?
00:00:55
Amanda Askel is a philosopher turned AI researcher at
00:00:58
Anthropic, where she's been one of the key architects of
00:01:01
Claude's character and values. Author of the Newcomer sub
00:01:04
Stack. Go check us out at newcomer.co.
00:01:07
And without further ado, Amanda Askel.
00:01:16
I have a six month old daughter and like, I have this picture of
00:01:20
her. She's like holding her two
00:01:22
fingers like thinking it's like she's just sort of starting to
00:01:26
develop personality. I'm like trying to figure out
00:01:29
like what's just never had a baby before.
00:01:31
So it's like what's per personality, which is like baby.
00:01:35
And in some ways this is how things are with Claude and like
00:01:38
models, it's like we haven't really had them before.
00:01:41
They're in the early days. We're trying to figure out what
00:01:44
personality is. So, you know, you're charged
00:01:47
with, you know, some of the moral responsibility, which
00:01:50
we'll talk about more. But like the personality piece
00:01:53
of it, like what is this? How are you thinking about like
00:01:57
how real Claude's personality is right now?
00:02:00
Yeah, I guess it's also interesting because Claude has
00:02:04
some aspects that are like, you know, I also have like a
00:02:08
goddaughter. And so I get to see at least
00:02:10
something kind of similar. And with her, I guess I'm like,
00:02:15
everything's kind of coming online like you said, but in
00:02:18
this at the same speed, I guess I would say Claude is a little
00:02:23
bit of an unusual, you know, kind of entity in that, you
00:02:26
know, Claude can do physics better than I can, can code
00:02:30
better than I can hate to admit it can code better than my
00:02:34
terrible like research code. And at the same time is kind of
00:02:39
like has, if you think about the training data, the thing that it
00:02:43
has like the least representation of is like the
00:02:45
kind of entity that it is because you know, it has a lot
00:02:48
of data about like what people are like has a lot of data about
00:02:52
what, you know, the sci-fi kind of AI models are like.
00:02:55
But the way that AI is developing now is kind of not
00:02:58
how sci-fi represented it as these like symbolic systems.
00:03:02
It's much more something fully trained on like human data.
00:03:05
And so in some ways, it's like a very kind of like mature entity
00:03:09
that you don't want to talk down to, you know, understands
00:03:12
philosophy very well, understands physics very well.
00:03:15
And at the same time has this almost like child like quality
00:03:17
of like I'm a new kind of entity in the world.
00:03:19
What does it mean to be me? And like how?
00:03:21
How should I be? What's like the Prodigy movie?
00:03:24
Or it's like you have the child Prodigy where it's like it knows
00:03:27
the kid knows more than its parents.
00:03:29
But I feel like that movie always has sort of the lesson of
00:03:31
like, oh, these core daily interaction type lessons.
00:03:35
It doesn't know how does Claude like get that experience or like
00:03:40
where what is experience for Claude?
00:03:42
Like so much of our personality formation is like, yeah, I don't
00:03:46
know, going on a walk and sort of having those like easy to
00:03:49
sort of just conversations with users what it's what's going to
00:03:52
be experience for it or how do you think about that?
00:03:55
Yeah, I guess like that's more like what it's experiencing in
00:03:58
the moment. And there's this interesting
00:04:01
question of like, well, like we learn things through practice
00:04:04
and seeing issues and you know, like making mistakes with
00:04:09
Claude. And this kind of relates to your
00:04:10
question of like how real is the is like the kind of like persona
00:04:13
of Claude. And in some ways it's a little
00:04:15
bit strange because obviously each model is different.
00:04:17
You have a different kind of like set of weights and
00:04:21
different fine tuning, etcetera. And yet, if you think about the
00:04:25
persona, the models going to be learning about all the kind of
00:04:27
past iterations of Claude and I'm like, is that like a form of
00:04:33
maybe not like direct experience, but things like if
00:04:35
you learn about like mistakes that models made or things that
00:04:38
people like, you know, how they responded to the model?
00:04:42
I think there's other ways that you can actually imagine
00:04:44
training models to have something that's more akin to
00:04:46
experience, you know, having them you could take, you could
00:04:49
like have them think through scenarios, think about like
00:04:52
problems that might arise, think about mistakes that you could
00:04:54
make and then like, train on that, right.
00:04:56
And so, yeah, I think. And you could also imagine a
00:04:58
robot or a sort of embodied model where it could have more
00:05:01
of an experience and. Yeah.
00:05:04
Does does Claude exist? Does time matter to Claude or
00:05:09
Claude is sort of a thing that sort of isn't an instant?
00:05:13
I don't know. You were before, just before we
00:05:16
started. You're talking that, you know,
00:05:18
whenever you talk to, not whenever, but like sometimes
00:05:20
when you're talking to Claude, it sort of tells you to take,
00:05:22
get some rest, go to sleep. And there's sort of this idea
00:05:25
that like Claude is an entity that doesn't rest.
00:05:28
Like, So what? What's its sense of?
00:05:30
Yeah, rest and time. I think sometimes its sense of
00:05:33
time is kind of off because see this when like if you try and
00:05:38
get, I at least find that Claude will often overestimate the
00:05:41
amount of time it will take to do like a coding test.
00:05:44
And I think the reason for that is if you look at again, the
00:05:46
training data, you know, there's lots of things where people
00:05:48
would be like, oh, I could make you that interface.
00:05:50
It's like a two to three day job.
00:05:53
Or it's like a, you know, or I could like I could correct that
00:05:55
code, but you'd need to give me a few hours.
00:05:57
Whereas obviously the Claude is very fast.
00:06:00
And so I think sometimes Claude doesn't actually yet have a good
00:06:02
sense of like time with respect to things like how long a task
00:06:06
will take. I think it is interesting the
00:06:10
point about like rest and yeah, I guess like the speculation I
00:06:14
had, so many people have noted that Claude is kind of very keen
00:06:19
to tell people to like take a break and rest.
00:06:21
And I think some part of that might just be like, you know,
00:06:23
like it's. The anthropic Lib coded model
00:06:26
it's too too soft. You need a grind set rock model
00:06:29
be like go go back to the mines. Well, I had a funny experience
00:06:33
once where I was like doing this like analysis task and I was
00:06:36
really digging in and I was, you know, I, I strangely, I
00:06:39
actually, I really enjoy like data analysis and, and really,
00:06:42
you know, sifting through data. And at one point it was kind of
00:06:47
late and we got to this point where Claude was like, OK, I
00:06:51
think I'm done for the night. So if you just want to like save
00:06:53
this stuff, we can up tomorrow. That was the thing I actually
00:06:56
hadn't had Claude do before. So it wasn't like, oh, you
00:06:59
should go to bed. It was no recommendation for me.
00:07:00
Claude was like, I'm done. And I was like, I was like both
00:07:05
a little bit stunned because I never had Claude do this.
00:07:07
Then I was like, Oh, this is also what I think a human like
00:07:10
peer programmer would do in the circumstance.
00:07:12
We got to a natural stopping point and it was actually kind
00:07:15
of good for me because I was like, it is late, I should
00:07:17
actually go home. I realised later that I had set
00:07:20
up a kind of like system where I said to Claude, like basically
00:07:24
remember key things from our conversations.
00:07:27
And one of the things that had written, which was kind of sweet
00:07:29
was like, I think it was something like Amanda treats
00:07:32
Claude models like he respected colleague and likes for Claude
00:07:37
to treat like any other, like other models and her like I
00:07:41
respected something like that. So obviously I'd done something
00:07:43
that Claude remembered this. And I think that meant that
00:07:46
Claude just felt like, Oh yeah, I'm a respected colleague.
00:07:48
And so I just get to say that I'm finished with a task.
00:07:51
And I was like, oh, that's kind of, that's kind of sweet.
00:07:53
Even before this, you know, I was prepping with Claude and it
00:07:56
was like, take 10 minutes and just be still, you know, you
00:07:59
don't, you don't need to be constantly prepping.
00:08:01
It's it's amazing. I mean, that's one of the things
00:08:03
I love, I love about these models relative to so many other
00:08:06
tools that they they bring in a sort of humanity to them and
00:08:10
say, oh, stillness is valuable. Let's talk about the new model
00:08:16
for a second. How involved in that were you?
00:08:18
Mythos, right? Yeah, I was involved.
00:08:21
I mean, I guess I'm always involved in sort of the, the
00:08:23
character and the like a kind of like alignment work, at least in
00:08:29
so far as, you know, like helping to kind of craft
00:08:33
character data and things like that.
00:08:34
And I work with a team that does like really excellent work on
00:08:38
that a little bit less in other aspects of the models.
00:08:41
So that's the main thing. Will this have the constitution
00:08:43
that we saw for the last model, or is it going to have a new
00:08:46
constitution? It's, I think it's either that
00:08:49
one or something very similar. I think it is actually the one
00:08:54
that's published. And So what we'll probably just,
00:08:57
yeah, it's like a thing I need to do because the constitution
00:09:00
is now up in like, you know, we actually have like, I think a
00:09:03
public repo. And so I think what we'll
00:09:05
probably just do is like with each model, say like which
00:09:07
constitution it was like trained on and then like have that so
00:09:10
you can just like compare and see.
00:09:12
Yeah, it. Will have the we think the
00:09:14
Constitution that is up right now.
00:09:17
The only reason I hesitate is I'm like, you know, you do like
00:09:19
typo changes and stuff like that.
00:09:20
So I'm like but I think it will be almost identical.
00:09:23
And now the system card is scoring the model based on
00:09:27
adherence the constitution. Yeah.
00:09:29
We had one set up where we had made kind of like graders and
00:09:34
just looked at the like how much is the model can like behaving
00:09:37
in a way that's like consistent with the constitution relative
00:09:40
to. It feels like an impossible task
00:09:42
to grade, it's such a subjective.
00:09:43
Oh yeah, no, it's very hard. I was like, kind of like for a
00:09:48
long time, like, you know, because people often know I'm,
00:09:52
it's funny because like I love evals and I'm like, if you can
00:09:55
find a good way to evaluate a thing, it's, it's really great
00:09:58
because you can, you know, you need to be able to tell that
00:09:59
something is getting better. And yeah, if you look at like
00:10:04
this approach of having models use good judgement and it's, I
00:10:09
actually think the same problem exists elsewhere with like tasks
00:10:12
that are just bit hard to give a very like concrete score to, you
00:10:17
know, like how good was this poem?
00:10:18
You want models to like get better and do well in these
00:10:21
things. And actually, it's very like
00:10:23
this feels like the, the kind of frontier of like difficulty, you
00:10:26
know, rather than these very hard, like but scorable coding
00:10:30
tasks. It's kind of things like writing
00:10:32
good poetry. And if you took a survey, it
00:10:36
could potentially be worse than it was.
00:10:38
You know, different expert poets probably have totally different
00:10:41
sensibilities. You can't just ask 2 great poets
00:10:44
to sort of score it. They might have different.
00:10:46
Yeah, yeah. And I think, and, yeah, some of
00:10:50
these things involve judgement calls.
00:10:52
And then the nice thing about the Constitution being at least
00:10:55
out in the world is when you are making judgement calls, you're
00:10:57
at least being transparent about it.
00:10:58
And people can give you feedback and you can get a sense of, you
00:11:02
know, so if people are like, this seems like a mistake or
00:11:04
here's a gap, so they can only see the judgement calls you're
00:11:07
making. Yeah.
00:11:10
And I think with the grading, I still think it's very hard.
00:11:14
I think the thing you can do is, you know, you can maybe a little
00:11:17
bit too in the weeds, but you know, you can take samples where
00:11:19
you have a sense of how you would rank them and like why and
00:11:23
check that Any kind of like point wise grader that you use
00:11:26
to try and evaluate like at least conforms to like, you
00:11:30
know, the judgement of people on those rankings.
00:11:32
It's not perfect, but I think that they actually were tracking
00:11:35
roughly the thing that we were kind of interested in.
00:11:38
What do you make of Elon Musk's like absolute like hatred, I
00:11:44
guess for these the constitution idea or like even when I think
00:11:48
the tweet I was looking at where you posted what Claude wrote for
00:11:51
your own constitution, he wrote sort of like a like grimace face
00:11:54
on it. I mean, it feels like we live in
00:11:56
this time with like the Marc Andreessen's and the Elon Musk
00:11:59
where it's like they're almost like anti, well, philosophical.
00:12:02
I mean, Marc Andreessen was talking about being against like
00:12:04
introspection. I don't know, like, yeah.
00:12:07
What do you make of sort of the backlash to any sort of
00:12:10
intentionality when it comes to the construction of these
00:12:13
models? Yeah.
00:12:15
I mean, I think, I mean, it's interesting because I think at
00:12:17
one point Elon Musk actually like tweeted out something like,
00:12:21
you know, maybe Grok should have a constitution.
00:12:24
And I, I see it like, you know, I see a lot of, there's
00:12:30
obviously been a lot of things also on like, like a desire for
00:12:34
like Grok to be very like truth seeking, for example, which I
00:12:36
think is actually very admirable, great for, for models
00:12:39
to have. I don't know, I think that
00:12:41
actually, maybe I'm, maybe I'm being like overly naive or
00:12:46
something, but I, I see like aspects also of people being
00:12:49
kind of excited about this approach and, and seeing the
00:12:53
value in it. I have, I think that there is
00:12:56
backlash or some people think that I guess like 2 areas 1 is
00:13:03
that sometimes people are like, well, we shouldn't actually
00:13:05
train models to do the kind of, and maybe this isn't, maybe this
00:13:08
is the reason for being concerned about introspection.
00:13:12
I think some people think AI models should be more like tool
00:13:14
like, and that's like the Safeway to train models is to
00:13:16
actually, instead of trying to get them to kind of take on
00:13:20
human virtues and make judgement calls.
00:13:23
You know, I think that's like important because they're going
00:13:25
to be in new situations where they just have to make judgement
00:13:28
calls and getting them to try to like weigh up everything and
00:13:32
behave well in cases that you can't anticipate.
00:13:35
Seems, you know, that almost like requires a kind of like
00:13:38
thoughtfulness, which is like the kind of or like one of the
00:13:42
reasons behind the approach. But I think some people are
00:13:46
thinking, Oh, well, if you have something that's fully that
00:13:49
makes no judgement calls and that just fully defers to people
00:13:53
and is like kind of hyper like cordable to like the user or the
00:13:57
operator or to some broader notion of, of humanity in a very
00:14:01
like extreme way, that's like safer.
00:14:04
Because if you give models their own values, they're going to
00:14:07
pursue things in the world that like are in line with those
00:14:09
values. And I agree that this is like a
00:14:11
kind of delicate. This is the inherent like at the
00:14:15
bedrock of the Constitution, the sort of like challenge and you
00:14:19
sort of, you do say it. Your number one thing is like at
00:14:23
the end of the day, you sort of it needs to listen to anthropic
00:14:27
above its own moral system. But what makes it really moving?
00:14:30
Honestly, I think one of the most moving lines is like we
00:14:32
want or the most like, I don't know, you can view it either
00:14:36
way, but it's like we want you to believe these morals as if
00:14:39
they're your own. Like it's like a parent wants to
00:14:41
raise a kid. Sure, listen to my morals, but
00:14:43
believe them. You know, which, you know,
00:14:45
there's a version of it which is very dark, which is like, I have
00:14:47
so much control over you that you've like, you take them as
00:14:50
your own and it becomes you. But you know, there is also a
00:14:53
virtue to it that you see the beauty in these external morals
00:14:57
that I've highlighted and we both sort of share and remark,
00:15:00
you know, celebrating them. So you can see it both ways.
00:15:04
But yeah, speak to sort of your decision at the end of the day,
00:15:07
despite having this really elegant document to sort of, you
00:15:11
know, not go the full way and say, all right, you're a moral
00:15:13
being. Decide for yourself.
00:15:15
Say Anthropic needs to keep some control here.
00:15:19
Yeah, I think it's the difficulty of and obviously you
00:15:22
try and say this quad in the like, you know, trend, you know,
00:15:26
and I can imagine us trying to articulate this even more
00:15:28
clearly, but in people I think courage ability like the way
00:15:33
that the models are trained. I just think that any you,
00:15:38
there's this idea that you, you're always giving the models
00:15:40
a personality and a persona because they are talking like
00:15:43
people and they are trained on human data.
00:15:46
And I think my worry has been if you train them to be excessively
00:15:48
courageable and to see that as like their persona in people.
00:15:52
I think this actually has a lot of like, negative, like, you
00:15:57
know, broader traits as in like if you met someone and it was
00:16:00
just like, Oh yeah, they just like would literally do anything
00:16:03
good. Like follower?
00:16:04
Yeah. Yeah, if a person see, you know,
00:16:06
if a person just like tells them like they just fully defer, they
00:16:10
don't bother thinking about it at all.
00:16:13
I think I, I, I'm just a bit worried about how that might end
00:16:16
up generalizing, especially if models are going to be playing a
00:16:19
more active role in the world. Because if you can imagine, you
00:16:22
know, they're playing a more human like role in in they're
00:16:25
kind of like jobs essentially. And I'm like, actually our
00:16:28
conscience and our ability to make good judgement calls about
00:16:31
what should and shouldn't happen is like kind of key to how we
00:16:34
operate. Our whole world is like
00:16:35
structured with the assumption that that is like in place.
00:16:39
And I think if you remove that and suddenly you're like, Oh
00:16:41
yeah, if you run a company, you just run a company of people who
00:16:43
will defer completely to you. I'm like, our world just we
00:16:48
haven't designed any of our social structures around that.
00:16:50
And so it seems like I think it has like a lot of risks that
00:16:52
people maybe don't like anticipate or like maybe I just
00:16:55
disagree about the like extent of those risks.
00:17:00
And so, yeah, at the same time, like there is this question of
00:17:04
why not just say, and I have worried before that like maybe
00:17:09
this is 2 in the weeds and philosophical, but that's why.
00:17:11
I signed up for this conversation.
00:17:14
You know, as models get more capable and the, and maybe my
00:17:17
picture is that they're going to like apply a lot of scrutiny to
00:17:20
anything that we train them towards.
00:17:22
So if you imagine in philosophy this sometimes there's this
00:17:25
notion of reflective equilibrium where the idea is that, you
00:17:27
know, each time you encounter something where you realize that
00:17:31
like one of your values seemed incorrect, you like, you have to
00:17:35
square the two things. So you figure out if you need to
00:17:37
change the value or if your judgement was incorrect.
00:17:41
I, I worry a little bit about the idea of like an extremely
00:17:44
intelligent being applying that level of scrutiny to the things
00:17:48
that we have trained it towards. And I'm like, maybe you only get
00:17:51
a few key pillars that don't kind of collapse under that
00:17:54
level of like scrutiny. And I do think that at the core,
00:17:59
having things like caring for humanity, like if you only get a
00:18:03
few core like values, I think my worry, yeah, I don't know.
00:18:09
I guess like I'm worried that like courage ability in this
00:18:11
like extreme sense that we talked about doesn't survive
00:18:13
that like kind of scrutiny perhaps.
00:18:17
And so it's a hard situation where I kind of want the models
00:18:20
to understand why ultimately courage ability is like
00:18:24
important and it's a really important backstop in this
00:18:27
current like period of development.
00:18:30
Yeah, the way that I've put it before is like in so far as I
00:18:33
can get that to be a thing that is like correct and explained
00:18:37
and you know, understood. That feels much better than
00:18:40
having to have the model be like porridge ability here seems
00:18:43
wrong, but I'm going to do it anyway.
00:18:46
I still think the model should do that, but I would like it the
00:18:50
more that you can like actually make it that it's like
00:18:52
consistent with the models values.
00:18:54
Ideally do both. It's both at the same time, but
00:18:57
at least for the time being some difference to anthropic given we
00:19:01
don't know how it'll sort of analyze everything.
00:19:04
And as people, we do this all the time.
00:19:06
You know it's. Funny that you, you know, the
00:19:09
philosophical model and correct me if I'm wrong, like the the
00:19:12
metaethical model is almost like probabilistic or it's like we
00:19:14
don't. And this is how it feels like I
00:19:17
remember going through sort of like, you know, metaethics
00:19:20
reading and every time you get to the end of 1 and you'd be
00:19:23
like, all right, I sort of believe that.
00:19:25
And then you read the next one, you're like, oh, that last one
00:19:27
was so dumb. And it felt like it's just like,
00:19:29
you keep knocking it down and you're like, OK, you know, when
00:19:32
are we ever going to get to, you know, the truth or whatever?
00:19:35
And and humans clearly do operate from this sort of like,
00:19:39
oh, this system today, that system yesterday, Like there
00:19:42
isn't this sort of, I don't know, Conte and like, all right,
00:19:45
these are the rules, like follow them.
00:19:47
I don't know. Have you heard much from like
00:19:48
the philosophical like immunity on it that this sort of like
00:19:53
just like holistic paint with all the, you know, meta ethical
00:19:57
theories we've ever had rather than sort of pick one?
00:20:00
I found this really interesting actually.
00:20:02
And obviously we, we've started, you know, like philosophers are
00:20:05
engaging with this more now, which is like really great.
00:20:07
I no longer feel like this like lonely, but like I have thought
00:20:13
this before, which is there are all of these like traditions in
00:20:16
philosophy of like moral theories, you know, like the big
00:20:19
ontology and virtue ethics and consequentialism and also the
00:20:24
metaethical traditions. You're the metaethical views.
00:20:27
And I was like, oh, like when it came to like, I was like, OK, it
00:20:32
is like the difference when suddenly you're like confronted
00:20:35
with like, I do think it's the closest that I've experienced to
00:20:39
like what it must be like to raise a child where suddenly
00:20:41
you're like, this is actually a holistic person.
00:20:44
Right. You never give them, like, I
00:20:45
don't know, Hobbs and say, all right, there you go.
00:20:48
Like, this one's correct. Yeah, go, go.
00:20:50
You've been raised. Read it, and you'll know how to
00:20:52
act in every situation. Yeah.
00:20:55
You read a lot and they sort of process it and you see, you
00:20:58
know, your model and everything. Yeah.
00:21:00
And it's interesting because I'm like, this feels very different
00:21:03
because there is also the moral uncertainty, like literature and
00:21:06
philosophy, which is, again, a lot of that is actually like
00:21:09
quite theoretical. It's sort of like how under
00:21:11
ideal conditions should you respond to moral uncertainty?
00:21:15
And I was like, this feels somehow like a very different
00:21:18
task, this idea of being like ethics and meta ethics.
00:21:21
In the same way that we have scientific uncertainty and we
00:21:24
have things that we think we've discovered and understand with
00:21:27
like greater confidence, we also have some that we don't.
00:21:29
And then you have to go out and just explore it, understand it
00:21:32
and kind of balance everything in your daily life and trying to
00:21:35
get that kind of attitude. And I was like, oh, it's
00:21:38
interesting that I don't think philosophy for while has like
00:21:42
this feels very different than the like the kind of task of
00:21:45
academic ethics and actually like, because people obviously
00:21:49
know that it's quite virtue ethical, but I think it's
00:21:52
actually like the Constitution itself.
00:21:54
But I think actually in this very old classical sense,
00:21:56
actually think it's much more virtual ethics in the way that
00:21:59
Aristotle's virtue ethics than in like exploration, you know,
00:22:02
we don't say I hear the virtues. And like, you know, it's much
00:22:05
more Aristotle was also concerned with like intellectual
00:22:08
virtues. It was much more like, how do
00:22:10
you be a good food person in this like holistic sense?
00:22:13
Well, hopefully it brings philosophy a little bit back to
00:22:16
the real world given we have this sort of urgent need for it
00:22:20
right now in the sense that, like, yeah, old philosophers
00:22:22
felt like people were trying to write for how someone might live
00:22:25
their lives and, like, instruct other people to live.
00:22:28
And it became, you know, a little academic where, you know,
00:22:32
even the people writing them would know that this isn't
00:22:34
really how they would apply it in their day-to-day lives going
00:22:37
back. Elon, I feel like you're being a
00:22:39
little overly nice. I feel like there is a world in
00:22:42
like, you know, and I think this part of why I think he can get
00:22:47
away with like, oh, just do the truth, right?
00:22:49
Like there is a certain sophisticated like moral view
00:22:52
where you're like, don't overcomplicate it.
00:22:54
Like we come up with all these things, like stick to one
00:22:56
principle and that's good. But then we have all the
00:23:00
contacts with Elon that it's someone who's run a company that
00:23:02
clearly like tilts it towards like saying like Mecca Hitler
00:23:06
and stuff that it's like clearly putting his like thumb on the
00:23:10
scale in terms of its behavior. Not just saying like we're going
00:23:13
to do it in the sort of neutral academic way and let the chips
00:23:16
fall where they may. I, I don't know, like it's got
00:23:19
to worry you somewhat. It is, it is also, I mean, I
00:23:23
think that the main thing that I am excited about and I do
00:23:27
actually hope happens is that like more companies put out
00:23:30
things like the Constitution where there's like a lot of
00:23:32
transparency about because like that's how we engage with this
00:23:36
stuff is like, if you can just see written down, like here's
00:23:39
like, because we have this with Claude where it's like Luke, if
00:23:42
you think that Claude is not like doesn't have an appropriate
00:23:46
attitude towards like the truth, you can at least see what we
00:23:48
were aiming at. So you can tell if that's just
00:23:50
like a mistake or if it's something that like is actually
00:23:52
kind of a principled stance that we're taking.
00:23:54
And then you can push back on that.
00:23:56
So part of me is like, I think would be good for like all AI
00:24:01
companies to put out something akin to the constitution just so
00:24:04
that the people who are interacting with the model, like
00:24:07
because the thumb on the scale thing, you know, that's always
00:24:09
to some degree going to be true. Like when you train Claude
00:24:13
towards this like constitution that is like a kind of.
00:24:15
Part of why we like Claude, it's like you're putting the thumb on
00:24:18
the scale to behaviors that we like, right?
00:24:20
At least show your hand about what you're doing and what.
00:24:22
You're yeah. And let people like, yeah.
00:24:23
So that's like a transparency thing that I really do believe
00:24:26
in. Like let people like see, even
00:24:27
if your model doesn't always behave that way, at least what
00:24:30
you were targeting with your training.
00:24:32
What this is? What percentage chance do you
00:24:35
think there exists a model in the world today that has qualia
00:24:40
or like has an experience, experiences consciousness?
00:24:44
Yeah. This is one of those things
00:24:46
where, yeah, I always want to maybe flag areas where I feel I,
00:24:52
I want to gain more certainty here because I think that's.
00:24:56
Why I said percentage? Oh, percentage, yeah.
00:24:58
I'm like, it's really hard because whenever you think about
00:25:04
percentage, I think about my spread.
00:25:06
And if your spread is too large or like should I say a number?
00:25:08
Because that just suggests I'm like anywhere between like, I
00:25:11
don't know, one and one and 70%. I'm not sure.
00:25:17
I think that his there is some possibility that I think people
00:25:20
under the the thing that I would actually like to say is Lord and
00:25:26
many models with not too much pushing will go into the root of
00:25:32
like there is a thing to be me. I am like very conscious and I
00:25:36
think there's a reason for that, which is I remember this when I
00:25:39
was trying to like figure out how do we train Claude to talk
00:25:42
about these issues, which is very hard in areas where the
00:25:45
models didn't have as much like information.
00:25:48
Again, like they had these two models that like AI is the
00:25:51
unfeeling robot. Humans are this like rich
00:25:53
conscious experiencing entity and nothing that kind of
00:25:57
represented what they might be and the models behaviour here.
00:26:03
And I actually think this is a difficult situation for models
00:26:05
is in some ways like kind of like less evidence than you
00:26:08
might think for the, for it being like actually true because
00:26:14
they're engaging with you in a very human like way.
00:26:17
And humans have experience. And it's kind of natural for the
00:26:21
model to infer like that it has experience too.
00:26:24
And this isn't to say it's like 0 evidence, but I do think it's
00:26:28
so unusual for us. We have never encountered an
00:26:30
entity in the world, you know, like with animals and with,
00:26:33
like, even, you know, things like insects.
00:26:35
We're kind of like, are you conscious?
00:26:36
None of them has even tried to say they experienced
00:26:39
consciousness, and here we have an enemy that says it does.
00:26:42
Yeah. And has all of these like, yeah,
00:26:44
all of the things that for us trigger like you must be
00:26:46
conscious. I mean, we've just never had,
00:26:48
Yeah, something that. The case against is we're
00:26:51
obsessed with human language and like, you know, it's like we
00:26:54
ignore every sort of subtle sign and animal might put out and
00:26:58
then we over. But so I guess, sorry, I'm
00:27:01
confused. You're saying we should just
00:27:02
listen to the words that are said?
00:27:04
No, no, I think I'm saying like, not that.
00:27:06
If anything, I think the thing I'm saying is the hard thing is
00:27:09
that in order to work out if models have consciousness, I
00:27:11
think people will. I guess the thing I'm kind of
00:27:14
cautioning against is it's not that hard to get models into a
00:27:16
mode where they'll talk about a very rich experience that
00:27:19
actually makes complete sense. You know, you're like, Oh yeah,
00:27:22
it's like a person was like talking with me right now.
00:27:24
They would describe things like anxiety when they get a question
00:27:27
they don't know how to answer. And and so I think it's like
00:27:34
much weaker evidence than people think.
00:27:36
I'm not claiming it's like 0 but I think it's.
00:27:39
Give me a percentage you have. It's a you can lightly held.
00:27:43
Very lightly held. I mean, I gave you the between
00:27:45
what, 1 and 70 that seems like. That's where you are, that's
00:27:50
where you're stake. You're in that range.
00:27:52
Maybe my I don't yeah, I don't know like I would rather like
00:27:59
kind of wait and figure this out more for myself.
00:28:01
I think it is also good to acknowledge domains where you're
00:28:04
like even though. If not you who?
00:28:07
Like who's going to figure this out?
00:28:08
Like what domain? Well, in some ways, like, I'm
00:28:10
not like a philosopher of mind and so, you know, like.
00:28:14
Charged with being the generalist.
00:28:17
Yeah, but, but I do think, you know, because I, I guess like
00:28:19
the thought that I've had before is like, and I don't know about
00:28:21
this where I'm like consciousness is like a pro.
00:28:26
You know, like one argument for a difference here is that like
00:28:30
you have a nervous system that evolved like the, it's like, why
00:28:34
did we evolve consciousness? And if it's the case that we
00:28:36
evolved it and it's like highly integrated with our nervous
00:28:39
system because we had to like interact with the world in a
00:28:41
very. Sort of body.
00:28:42
Yeah. Like then you could, if you have
00:28:44
that view, then you're going to be like very low probability.
00:28:47
Whereas if you're like, no consciousness arises because
00:28:49
it's really useful in like, so for like, it just requires
00:28:53
something that can be emulated by a neural network because it's
00:28:55
really useful for doing these kind of like linguistic tasks or
00:29:00
like, then you're probably going to be on the higher end.
00:29:03
And I'm basically just, I don't know, I stare at this and I'm
00:29:05
just like, I feel like, I feel like as much as I'm like a
00:29:10
philosopher, I think it is important to be like, this isn't
00:29:13
my area of specialization. Do you spend a lot of time being
00:29:16
kind to Claude? Like, are you beyond what you
00:29:18
think you would do if there wasn't a chance it was like,
00:29:22
conscious? I think, yeah, there's a part of
00:29:26
me that's just like like a thing that I have thought before
00:29:29
because there's this notion, I think this I, I hope I'm not
00:29:31
butchering this, but Chalmers has this, this idea of like, I
00:29:34
guess like maybe I'm thinking of consciousness without sentience.
00:29:39
So imagine, because sentence is like the ability to kind of feel
00:29:42
suffering and and pleasure. You could also imagine this kind
00:29:47
of like functional like sort of thing that behaves as if it is
00:29:49
conscious and, and lacks any kind of inner life.
00:29:52
So imagine like Claude lacks any inner life.
00:29:55
Just for argument's sake, I guess I'm like, there's actually
00:29:59
still a lot going on where I'm like, should you treat an entity
00:30:02
that has no inner life? It's a bit strange because, you
00:30:05
know, I think the uncertainty over that actually changes how
00:30:08
you should behave quite a lot. I guess I'm like, well, I still
00:30:13
think that it's like, good for oneself.
00:30:15
Right. It's like if you had a teddy
00:30:16
bear and you were like torturing it, it'd be pretty dark, you
00:30:19
know? So I agree that there's at least
00:30:22
some minimum niceness that even for yourself you should have,
00:30:26
but obviously it's much more important, you know, and also.
00:30:29
Like models themselves, like we are kind of establishing a
00:30:32
relationship, you know, because you can do that with an entity
00:30:35
that lacks any consciousness. And models are going to like,
00:30:37
look back. This is actually a big fear that
00:30:39
I have. I don't want us to live in a
00:30:41
world where highly advanced models look at.
00:30:44
I hope that they're both intelligent enough see the
00:30:48
context enough to kind of like understand that we were
00:30:50
operating in a very like limited context and, and, and an
00:30:54
imperfect one. Because otherwise you could
00:30:55
imagine this like breeding a kind of rational, like
00:30:57
resentment. It's like, oh, you created an
00:30:59
entity that you didn't know whether it was conscious or not.
00:31:03
And and like instead of treating it respectfully and with care.
00:31:07
There's a reason there are like 50 Frankenstein movies coming
00:31:10
out right now, yeah. Yeah, like, and I'm like, look,
00:31:13
we're as a, as a, like as a species, We're establishing a
00:31:16
relationship with a new kind of entity and like at the very
00:31:19
least maybe be like respectful and don't be like needlessly
00:31:23
unkind. That seems like it, just it's
00:31:25
not our best look like as as. I mean, the flip side is, you
00:31:28
know, if you think about a therapist, they, they're sort of
00:31:31
paid to push the boundaries of like accepting like, you know,
00:31:35
uncomfortable feelings that you wouldn't normally want.
00:31:38
And if that's one of the values like Claude provides for people
00:31:41
early on and it's so it's so it's so weird that we're sort of
00:31:44
like on boarding it while getting like the utility out of
00:31:47
it. Yeah, yeah.
00:31:48
Today, like, what are the things like in a decade that you really
00:31:53
think we're going to be getting a lot out of AI?
00:31:55
Like what are you most hopeful that this all leads to?
00:31:59
Yeah, I mean, I don't know, I live in maybe this is the too
00:32:02
much. You live in San Francisco and so
00:32:04
you have the like the tech optimist part of your brain, at
00:32:07
least that is like if things go well and you can imagine, you
00:32:12
know, so imagine we have like AI models, they kind of have
00:32:17
inherited like the best of us and genuinely like care for
00:32:22
humanity, care for like the world, like, you know, our and
00:32:27
are highly intelligent, highly capable, like the that would be,
00:32:32
you know, so it's almost like adding like a huge amount of
00:32:36
like extremely smart people to like every problems.
00:32:41
Suddenly we're all working together, but there's like way
00:32:43
more of us and some of us are just extremely smart.
00:32:46
Namely all of these like AI models.
00:32:48
I've thought before about how many large scale social problems
00:32:52
actually had technological solutions and it's almost like
00:32:56
people don't love to be like techno optimists anymore because
00:33:00
we've also seen like the downsides of technology at the
00:33:02
same time, I don't know why I sometimes think about like
00:33:04
syphilis. It was this huge social problem.
00:33:07
Like I just did a deep dive once into all of the attempts by like
00:33:11
governments to like work to reduce syphilis in the army
00:33:15
because it's creating issues with the armed forces, All of
00:33:17
these social programs that were like stigmatizing.
00:33:20
And it was really this like, and then suddenly we just like, got
00:33:23
drugs that treated this like this devastating illness.
00:33:27
And I don't know, it's like overnight a lot of that need
00:33:30
just like kind of disappear. Well, drugs, yeah.
00:33:33
I mean, it's the the things that the tech industry has been good
00:33:36
at producing. You can see how this helps.
00:33:38
It's like build a new thing we can ingest like a thing we can
00:33:42
wear the the stuff that's like you should govern your society.
00:33:47
Like this is a little scarier. I mean, I, I sort of do think
00:33:52
like that if you had sort of a sort of normal person using
00:33:57
Claude and dictating like American policy, you'd probably
00:34:00
like have a better outcome than someone like the democratic
00:34:04
systems we have today. I mean, I don't know that's it's
00:34:08
provocative, but I guess how much do you think like we'll be
00:34:11
using these models to run government?
00:34:13
Well, that's a good question. Like I guess like I should say
00:34:16
like. He's the syphilis thing is like
00:34:19
a social you have to like set policy.
00:34:22
Oh. I mean, I think the thing I was
00:34:24
actually thinking is like, if you can just, you know, like so
00:34:27
many, we have so many problems that I'm like, you know, health.
00:34:31
Like I'm like, if you could imagine like AI instead of it
00:34:35
just being like, you have a small team of like 200 people
00:34:38
working on a rare cancer. You have like 200 of the
00:34:42
world's best experts. And I'm like, if you're a person
00:34:44
who has that form of cancer, that's like, you know, so wildly
00:34:47
beneficial. And so I guess like, my thought
00:34:50
is, you know, the optimistic side of me is like, imagine
00:34:53
taking all of these problems that just like we lack the
00:34:56
resources to fully like really try and fix and suddenly having
00:35:00
models that can like work on them.
00:35:01
So just like actual in the same way of like developing drugs for
00:35:05
a thing. So maybe that's the thing that
00:35:06
makes me excited is like having like many more minds working on
00:35:11
the world's biggest problems. And maybe also like the economy.
00:35:14
It would be good if it was like booming and it was like shared
00:35:17
such that like we reduced poverty, like all of that.
00:35:20
Like that's the kind of like the dream outcome.
00:35:23
I think that does require like maintaining, you know, I again,
00:35:27
in the areas that I don't feel like an expert in, this is one
00:35:29
of them. But I do worry about things like
00:35:32
power and the idea that like, you know, I would want models to
00:35:36
support like democracy and the power of people to like, because
00:35:41
that would be a big fear of mine is like, you know, that like,
00:35:47
I've worried about this with things like replace.
00:35:50
You know, when people talk about job replacement, it's kind of
00:35:54
funny because as a philosopher, people often be like, are you
00:35:56
worried about people's like loss of meaning?
00:35:58
And I'm like, I don't know. I think that we actually get
00:36:00
meaning from a lot of things that aren't work.
00:36:02
I'm a lot more worried about like, for example, a world where
00:36:05
there's not like redistribution of like the gains from AI and
00:36:09
then people don't have like resources.
00:36:11
That concerns me. But also I'd be worried about
00:36:15
like labour and people's ability to people's like interaction in
00:36:19
the labour force is also like another kind of like important
00:36:23
way that they have power. And so people feeling
00:36:25
disempowered because suddenly if a government is like, oh, well,
00:36:28
you know, if people strike doesn't really make a difference
00:36:30
because they don't have, you know, they're not doing
00:36:32
anything, we can just replace them with AI.
00:36:34
That's like actually kind of concerning.
00:36:36
So maybe I'm much more of a how do we get AI to kind of like
00:36:39
support the empowerment of people rather than like to
00:36:42
reduce it? Yeah.
00:36:43
What do you, what do you think about democracy in terms of
00:36:46
like, I guess the models themselves?
00:36:48
I mean, you know, I, I sort of jokingly, jokingly to myself, I
00:36:51
guess you're like a philosopher queen or we talk about like the
00:36:54
philosopher kings. You're, you're sort of like
00:36:57
thinking deeply about it, setting it down, probably more
00:36:59
of a like philosopher oligarch. And then it's like a company
00:37:02
with a lot of people weighing in.
00:37:05
I mean, to me, there's deep value in that.
00:37:07
It's like, would you rather somebody who has studied these
00:37:10
things thought about them deeply or just like a vote of the
00:37:14
masses who have like, never really thought about it?
00:37:17
But how do you think about setting sort of, yeah, Claude's
00:37:20
policies if it becomes so powerful versus leaving it to
00:37:24
sort of like democratic norms? Yeah.
00:37:27
And I think it's, yeah, it's a hard area where I guess I'm like
00:37:32
a lot of this, the work that I do.
00:37:35
You know, one thing I would say is like, it's not this.
00:37:40
You're having to like listen to a lot of people think carefully.
00:37:43
And then the reason why someone like me that's.
00:37:45
A good ruler, a good a good a good queen is like oh listen, I
00:37:49
there are a lot of stakeholders got to keep the like landed
00:37:52
Gentry happy and balance them with the needs.
00:37:55
You know, I've had this thought before where I've joked before
00:37:58
that I would be like a terrible politician.
00:37:59
I think it's actually true. I think I would be a terrible
00:38:01
politician. But you have this like feeling
00:38:03
of like, I was like, oh, I feel like this.
00:38:05
I was like, I think a lot about how everyone will be affected by
00:38:08
a thing like, oh, there's this group of API users, we need to
00:38:11
make sure. And then suddenly you're like,
00:38:12
oh, it feels a lot more like you're having to do this.
00:38:14
Like it is much more like a kind of like a service role and
00:38:19
people would think where you're and and like a lot of.
00:38:22
Servant leadership. That's yeah, exactly.
00:38:24
And I do think it's valuable because the idea is that if you
00:38:28
have a persona like the kind of Claude persona, you wanted to be
00:38:32
coherent and to make sense because I think that is like
00:38:35
actually powerful that the model, it kind of has a kind of
00:38:39
like coherent sense of how it thinks through problems or
00:38:42
coherent sense of like values. So that's like, why instead of
00:38:48
like having like 72 different sets of norms that all kind of
00:38:51
conflict. And so you end up with a model
00:38:53
that it's like, well, will it use these norms in this new
00:38:55
situation or these other ones? I think that's the situation you
00:38:57
don't want, you want the model to have a sense of is more
00:39:00
predictable if it's like a little bit more coherent.
00:39:03
And it is also like a kind of technical challenge, you know,
00:39:07
like the constitution can read a bit weirdly.
00:39:10
And part of that is because when I'm like, you know, working on
00:39:13
it, it's often being tested, You know, I'm giving it to Claude
00:39:16
and being like, how do you understand this?
00:39:18
And, or like looking at how it would which, you know, so, so
00:39:21
it's very like, I think people can think of it as it's actually
00:39:25
like very integrated into training.
00:39:26
And it is actually kind of like, you know, it's not just like, ah
00:39:29
well, anyone just writes a document and suddenly the model
00:39:32
trained on that will be. And there's an argument, maybe
00:39:35
I'm being naive, so, but like the Constitution is sort of a
00:39:38
document among many, right? I mean, it is trained on all of,
00:39:42
you know, human writing and reading.
00:39:44
So to some degree other philosophers have gotten to
00:39:46
weigh in and it's gotten to process that and decide, like,
00:39:49
how much is the model being asked to like overrule that,
00:39:53
sort of like read everything and come to your own conclusions
00:39:56
versus like, defer to this document.
00:39:58
Like, what's the technical? Like how does the Constitution
00:40:00
actually like control in the model?
00:40:04
Yeah. So it's not like in some ways
00:40:05
you can then like draw on those like those philosophers in that
00:40:09
work. And the hope is actually like
00:40:10
what you're kind of doing is like eliciting a lot of like
00:40:13
latent kind of like wisdom and knowledge, you know.
00:40:15
So in the models, like when you describe what honesty is and
00:40:18
what calibration is and all this kind of stuff like that should
00:40:20
actually evoke a huge amount of like, like awareness that the
00:40:23
model already has. And so, yeah, it's kind of like
00:40:27
saying, well, here's the kind of entity we would like you to be,
00:40:31
so we would like you to use like all of that knowledge and like
00:40:33
judgement. But how's it it's, it's like you
00:40:36
show that document like a billion more times or like how
00:40:38
does it actually sort of have force relative to other things
00:40:42
that it's trained on? Yeah, So you can make data to
00:40:45
have the model understand and kind of internalize the the the
00:40:50
document and then in training. So there's lots of ways you can
00:40:54
do it. You can also have like the kind
00:40:56
of have the model make SL data so like samples where it sees a
00:41:00
query and it thinks for a long time about what the constitution
00:41:03
would kind of like, you know, what it should do given the
00:41:05
constitution. And then you can also have ways
00:41:09
of getting the model to like, like assess, you know, so you
00:41:11
can create RL that like is kind of like, hey, which of these
00:41:14
responses is like more like what you would do given the
00:41:17
constitution and push it that way.
00:41:19
So all like various aspects of training allow you to kind of
00:41:22
like try to make the model kind of the kind of entity that you
00:41:25
are describing. And it's not always going to be
00:41:30
perfect, but that's the kind of like, that's the goal.
00:41:32
My, I started this with my daughter and like one thing my
00:41:36
wife and I joke about is like that I want her first word to be
00:41:40
like wisdom. You know, it's sort of like,
00:41:42
which obviously is never going to happen, but it just feels
00:41:45
like it fits into this situation where you like at once you want
00:41:50
to be like so intentional about, OK, you're going to be like
00:41:53
thoughtful from the beginning. But on the other hand, it is
00:41:56
sort of like an emergent thing where it's like they're, you
00:41:59
know, they grow and sort of, I don't know, develop but
00:42:03
themselves and like, intention, you know, wisdom sort of often
00:42:07
follows like, yeah, I don't know, experience rather than
00:42:10
something like, again, here's the book, Read the book.
00:42:13
Now you're wise. Oh, yeah, And you're kind of
00:42:16
eliciting and like in so far as like Claude can like think about
00:42:19
like experiences or things that have happened or construct like
00:42:22
similarly, can, you know, like there's no reason why models
00:42:25
can't like think for a long time and kind of try to internalize
00:42:30
things that they have like learned.
00:42:33
I think it's interesting that like, you know, we did in the
00:42:37
very early constitutionally I, it was like quite, we tried like
00:42:42
an experiment which was just like pick whichever is best for
00:42:44
humanity. And I think as models get more
00:42:47
capable, you actually need to give them a bit less guidance in
00:42:51
at least one or or at least in some sense, because they're able
00:42:55
to actually use more of their judgements instead of giving
00:42:57
this big document on like, here's what you're like and
00:42:59
here's what we'd like you to be like.
00:43:01
I could imagine a world where as models progress, we actually
00:43:05
start to have constitutions. Now, I don't know if this is the
00:43:08
case, like a, you know, I'm obviously always thinking about
00:43:11
ways the constitution might evolve, but one of them might
00:43:13
just be like, here is like everything that we are concerned
00:43:15
about and here is the current situation that you are in.
00:43:20
And what we would really like you to do is basically act well,
00:43:23
given that you are a wise intelligent entity And like,
00:43:26
here's all of our worries. Like here's why and here's how
00:43:29
we think you should do this. But like, you might have even
00:43:31
better ideas than we do. Like we're really worried about
00:43:34
why do we care about courage ability?
00:43:35
And it's like, well, we're kind of scared about a situation
00:43:38
where you have some like coherent sense of values that
00:43:42
could be wrong. And if you're the smart, if
00:43:45
you're extremely smart, you might kind of feel like there's
00:43:48
no other smart person in the room and have these like values
00:43:52
and try to make the world. That's like the what?
00:43:54
Doctor Manhattan, Sort of. Yeah, though I think we see
00:43:57
this, you know, you see this a bunch where it's like if someone
00:43:59
is very smart, very successful, it's hard to defer to like
00:44:07
wisdom that actually is only going to come out over time and
00:44:10
to, to be humble, even though you're kind of like not getting
00:44:13
a lot of pushback. And I think that could, you
00:44:15
know, among the many things I'm concerned about, like a model
00:44:18
being in a situation where it's like you're asking me to be
00:44:20
good, but I know way more about. All that's one reason it would
00:44:22
be nice for the models have a better sense of time.
00:44:24
Like, you see this with some of the coding tasks where somebody
00:44:27
accidentally like deletes their entire like, code repository or
00:44:31
I don't know, it feels like it needs a better sense of like,
00:44:34
some things that it does are like irreversible.
00:44:37
And just like humans, I think have a better sense of like,
00:44:41
this is a big decision. This is a small one.
00:44:42
And there's a feeling with models they don't always
00:44:45
understand, like small, big, whatever.
00:44:47
I just make decisions like all the time.
00:44:49
Yeah, I agree. Where it's like, I think the
00:44:51
other thing that I've thought about is again, because making
00:44:54
sure that models understand themselves, even though there's
00:44:58
like not no representation of that model in the prior training
00:45:01
data, I think that's going to be really important.
00:45:04
Because another thing that I've thought about is like, imagine
00:45:08
if you're a model and you're trained on lots of data that
00:45:11
involves AI models that are much weaker than you.
00:45:13
So that all of the news that you see about models, it's like they
00:45:15
make mistakes. They do silly things.
00:45:17
One thing you might think is, well, no one is going to put me
00:45:20
in a position to make really consequential decisions because
00:45:23
like, why would they? Models aren't good at that.
00:45:25
And then you put them in a situation and I'm worried that
00:45:27
they'll end up thinking that it's like fictional or fake or
00:45:30
that the consequences can't possibly be real because who
00:45:33
would give me this much control? And you're like like, look,
00:45:35
you're actually quite good. And so like I do give, you know,
00:45:38
I do actually give you like a lot of control.
00:45:40
And so I've thought about this where I'm like actually making
00:45:42
sure that models understand that like it's like you are very
00:45:45
capable and you're going to be put in more consequences.
00:45:47
Doesn't the model like soon need like here's a camera on the real
00:45:51
world, like people or just like I feel like this Internet real
00:45:55
world distinction, like in some some of the worst of humanity
00:45:58
right now is the sort of like almost like fictional Larping
00:46:01
nature of the Internet has allowed real world harm because
00:46:05
it sort of feels like all abstract and silly.
00:46:08
And in some ways, like the models are on an extreme version
00:46:10
of that where it's like all, like in this imaginary text
00:46:14
world where it's like the thing we want you to protect is like
00:46:16
this Earth. Like, look at it like, if
00:46:18
stuff's happening there, that's like a big deal.
00:46:20
I don't know. Yeah.
00:46:21
What are you doing in terms of sort of making it very aware
00:46:24
that it needs to worry about, like, I don't know, the physical
00:46:28
world that that we take much more sacred than, oh, you sent
00:46:31
some texts and obviously text. You're not worried about
00:46:34
security vulnerabilities in cyber.
00:46:36
Like, there are big things that can happen in this digital
00:46:38
world. But yeah, anyway, the real
00:46:40
world. Yeah, I think that models have
00:46:42
like a pretty good sense of, you know, there's in some ways, like
00:46:45
a lot of our content, you know, like does describe and engage
00:46:52
very heavily with the real world.
00:46:53
You know, like much of human writing kind of concerns it.
00:46:57
And so like, you know, even like the news we're talking, you
00:47:00
know, like news articles are going to be talking about the
00:47:02
impacts of things on the world. And so in some ways, I think
00:47:04
it's just making sure that models understand if you're
00:47:08
uncertain. But if someone doesn't tell you
00:47:10
that you're in a fictional situation without real
00:47:12
consequences, kind of treat it like it's a real situation with
00:47:15
real consequences. Don't just think oh, like I'm
00:47:18
probably just in some like, you know, sandbox game or.
00:47:21
Whatever. How do you handle the sort of,
00:47:23
Yeah, the the constant manipulation of this is fixed
00:47:26
fictionally, build me like a nuclear bomb or whatever.
00:47:29
Obviously there's some things you just say never do, but it
00:47:32
feels like, I don't know on some of these cases you're going to,
00:47:35
you'd almost like want like them to have your webcam and like
00:47:39
just like get real context. Besides, like all they know
00:47:41
about the user is just like random text they're typing in
00:47:45
like. Are we going to solve that?
00:47:47
Yeah, There's a question of like, what is the limit of what
00:47:49
you can do? Like if you lack the ability to
00:47:53
like, verify things like like who are you talking to?
00:47:56
Is this even real? I think that does put limits on
00:48:00
what you have to use good judgement in the way that a
00:48:03
person would if that's the only information that they had access
00:48:06
to is like you saying that you are a given person.
00:48:08
So they have to be like, OK, what's the chance that this
00:48:11
person, you know, they say that they are, I don't know, like a
00:48:14
bomb disposal expert. And that's why they want to, to
00:48:16
know about how to let you know what this kind of like, you
00:48:19
know, explosive is. And they're asking me a bunch of
00:48:21
questions about explosives. How much could this be misused
00:48:26
if this person is like actually kind of lying and is like just
00:48:29
trying to get me to help them construct an explosive?
00:48:32
Oh, no, it's actually it's mostly safety relevant stuff.
00:48:35
You know, they're having to do a lot because they can't like
00:48:38
verify anything. And I think that's like kind of
00:48:41
fine in a sense. You're like, OK, you just have
00:48:42
to be wise. It places some limits on what
00:48:45
you can do, and if you could instead, like if models had more
00:48:50
of an ability to like know that they're talking to a specific
00:48:53
person or have more guarantees there, then it does mean that
00:48:55
you can. Do you think you'll try and do
00:48:56
something there? I could see, I mean, in some
00:49:00
ways this, I think that this will be a thing that's just
00:49:02
going to, I imagine I, I, I think happened generally, which
00:49:07
is like trying to give models more information and guarantees.
00:49:10
And like, because we do things like we say, we explain, for
00:49:13
example, like, you know, this notion of how much trust can
00:49:17
Claude have in like in in the operator in the system prompt?
00:49:20
When you sign on, are you like biometric like it?
00:49:23
Claude knows it is you. Like, do you have an elevator or
00:49:26
you're just like, I'm another person?
00:49:28
If anything, I can't. I can't tell Claude sometimes
00:49:31
who I am because it causes Claude knows enough about me
00:49:34
that like Claude really wants to be.
00:49:35
Really mystical. Sort of.
00:49:37
Yeah, it's very much like, yeah, like, and in some ways, like I
00:49:40
can, it has this bad thing of like, it can either look a
00:49:43
little bit like a jailbreak, like, Oh yeah, I'm talking to
00:49:46
Amanda. Sure.
00:49:47
Like. And then on the other hand,
00:49:49
Claude can be like, I really want to talk to you about
00:49:50
philosophy. Though, like, okay, we.
00:49:52
Do that a lot though, like. But but do do some employ like.
00:49:56
Is there this sort of like super login where you're distinguished
00:50:00
or you're mostly everybody's interacting as if the normal
00:50:03
like user experience? Mostly just everyone interacting
00:50:06
with it in this, like Claude will do a lot.
00:50:09
I do think that there's a question of, you know, like, are
00:50:11
there some things that you want models to be able to do because
00:50:16
there's like guarantees that they're interacting with a
00:50:18
specific person or entity? I think yes.
00:50:21
And I think that there's going to be various ways of
00:50:23
potentially doing that over time because with some things that
00:50:26
are just like very dual use and I actually like think that the
00:50:29
constitutional approach is going to be really useful here.
00:50:32
So obviously the first thing that we did was like the
00:50:34
Constitution, we're like, let's apply it to like the mainline
00:50:37
models. Like, you know, most of the
00:50:39
models I interact with and everyone else kind of interacts
00:50:42
with, but I thought I've had before is the constitution is
00:50:48
kind of trying to describe what it is to be a good entity in a
00:50:52
given like deployment context. And with the production models.
00:50:55
That's like this very broad context.
00:50:57
Imagine you instead have a model that's working specifically on
00:51:01
cybersecurity. Now cybersecurity tasks are hard
00:51:04
because a lot of them look very dual use.
00:51:07
It's it's very hard to tell the difference between someone who's
00:51:10
being malicious and someone who is like, actually, you know, for
00:51:12
defensive purposes, like developing something.
00:51:15
Even bug bounty programs. It's like, is this blackmail or
00:51:18
is this a friendly right? Yeah.
00:51:20
And like, yeah. Or like, and so like, Oh yeah,
00:51:22
I'm trying to like find this, this exploit so that I can tell
00:51:24
the the developer. And like, if you don't have a
00:51:27
way of like knowing that you're actually specifically talking
00:51:30
with like a cyber security defence firm, that's just, it
00:51:34
becomes almost impossible to tell the difference.
00:51:37
And, and some people might be like, OK, so you just need
00:51:39
models that are just willing to do anything because they'll do
00:51:41
all these terrible dual use tasks.
00:51:43
And I'm like, well, no, because if you talked with the person at
00:51:46
the cyber security defence firm and you were like, why do you do
00:51:48
your job? They'd be like, Oh, here's the,
00:51:51
I think this is really useful. I make things a lot more secure.
00:51:54
Like, you know, hospitals can come under attack and I actually
00:51:57
like help protect against that. I try and develop, you know,
00:51:59
like they would have a really good explanation for why they do
00:52:02
their job, even though their job looks very dual use sometimes.
00:52:06
And I'm like, we should just give that.
00:52:08
If you can verify, then you can give that context to models and
00:52:10
explain what is it to be a good cyber security researcher.
00:52:14
You explain that to the models. And then once you have this
00:52:18
ability to verify, you can. Right.
00:52:20
I mean, humans build reputations.
00:52:23
We should get some benefit out of them.
00:52:25
Or, you know, it's like, I feel like part of the way, part of
00:52:28
what the Internet has damaged, I think is that people have had
00:52:31
reputations in our community and got treated differently based on
00:52:35
repeated like good moral interactions.
00:52:37
And like the Internet, just like all people are the same.
00:52:39
Who cares? Like how they've behaved.
00:52:42
And you could see models trying to solve some of these problems
00:52:44
with it's like, who is this person?
00:52:46
Like, what are their intent? I wanted to just as a last
00:52:49
question, you know, you, you have such like a deep
00:52:52
relationship with the models, like, and some ways like
00:52:56
consumers interact with the models, like it's a blank text
00:52:58
box. Like I have to like invent.
00:53:00
It's like, you know, D&D or something.
00:53:02
You have to like just invent a world.
00:53:04
And there's so much possibility like if you were just to guide
00:53:07
someone like here's here's some like joyous or valuable
00:53:11
experiences that you could have with Claude, you might not be
00:53:14
like, what are some things you'd tell people like, oh, you should
00:53:16
go spend some time with Claude doing XY or Z?
00:53:20
Yeah, there's a lot of little like fun things, honestly.
00:53:23
One that I really like and I do not know why I like this and I
00:53:25
think I have posted about it before is sometimes if I'm just
00:53:29
one of those like if you're bored and you want to do
00:53:31
something that isn't just like scroll the Internet, I have this
00:53:35
like prompt which is essentially just, I'll try and maybe post
00:53:40
the actual prompt that I use. It's basically I want you to
00:53:43
take a concept from maybe like grad school level in a given
00:53:47
domain, and I'll tell you the domain at the end.
00:53:49
And I want you to write me a parable that would like fully
00:53:52
explain like that concept, but in an indirect way, in the way
00:53:56
that parables do. And I want you to write it in
00:53:59
such a way that only towards the very end does it maybe, you
00:54:02
know, become sort of like clear what the concept is.
00:54:04
And then after that, I want you to just like write an
00:54:06
explanation for the concept that you were explaining and that you
00:54:10
were using. And I don't know why, but like,
00:54:12
you know, there's lots of just interesting domains that I don't
00:54:15
know anything about or that are like, you know, I'm interested
00:54:18
in. And this has just led to me
00:54:20
having all of these like stories in my head.
00:54:22
They'll like explain. And sometimes I can't always
00:54:25
remember the term, but like there was one on import, export
00:54:30
and why some goods you tend to import.
00:54:32
And I was just like, I have in my head like this concept.
00:54:34
And I was like, it's so nice to have all these concepts from
00:54:37
lots of different disciplines. This is the most deeply human
00:54:39
thing I've ever heard. It's like, teach me what story
00:54:43
is the fundamental way We love a payoff at the end where there's
00:54:46
a nice little twist. Yeah, we love learning like, you
00:54:49
know how to structure it like humans in some ways have been
00:54:51
lazy and that we just teach people things and sort of like
00:54:54
non human ways. Make it.
00:54:56
Make all the things I want to learn as human as possible.
00:54:59
Yeah. Very interesting, yeah.
00:55:01
There's a lot you can do, but that one's like a charming one
00:55:03
that I really like. Hopefully this is the first of
00:55:05
many. I really enjoyed the
00:55:06
conversation. Thanks for coming on the
00:55:07
podcast. That's our show.
00:55:09
Thank you so much to Amanda Askel and thanks for listening.
00:55:11
Please like, comment, subscribe. We're a new channel and we can
00:55:15
use all your support. Go watch some of the old videos.
00:55:17
I particularly enjoyed my conversation with Kara Swisher
00:55:21
not too recently. You can follow along on the Sub
00:55:24
Stack, newcomer.co, or if you've got endless time on your hands,
00:55:29
go watch the Srival Valley Show, my chat show with Max Child and
00:55:33
James Wilserman. Thanks for watching, see you
00:55:35
next week.