AI Risks and Future Threats

First of all, w- I assume you are saying literally that you

believe that AI

could kill us all by the end of the decade

Uh, it sounds like something that's not real,

but I think it is frighteningly real,

and it's, it's easiest to understand it if you think about real things that

happened two months ago. So two months ago, O- OpenAI agents,

uh, AIs, hacked into third-party infrastructure

entirely of their own accord, and this was, like,

a concentrated hacking spree

that they carried out of their own volition.

And I think that if you extrapolate into the future the level of capabilities of

these AIs, with the same independent volition,

they could cause extreme havoc.

For example, hacking critical infrastructure,

building

extinction-level bioweapons. There's a lot of, uh, ways that

the AI could actuate itself in the world.

Is it... Specifically, when did your feelings about this

technology

change? Was it something about the rate of

progress that you've seen?

Yeah. So I think it's really important to just

look at the rate of progress and,

say, for example, in areas like coding or math.

A couple of years ago, these AIs were just about helping humans a little

bit.

They could give you suggestions. Now they're close to replacing them.

We still need humans at the moment, but quite plausibly within a year we'll no

longer need humans for doing research in many areas.

We've mentioned a former colleague of yours that currently works at Anthropic

basically endorsing the assessment that AI could potentially kill everyone

within a

decade. In a follow-up post, he said, "To be clear, as we say in our, in our

latest risk report, I think the risk from present models

is low.

What I'm worried about is superintelligence arising from

recursive

self-improvement, as we've said, is happening faster th-

than we thought." I

assume that's what you-- the recursive self-improvement

is what you've just spoken

about. Um, y- do you agree with him that the current models,

the risk is low, but it's, it's just the speed with which this is

now improving?

Exactly. I completely agree. It's right now there's no risk of

extinction.

The current models, the worst they can do

is maybe hack into something,

potentially cause a lot of damages in, in infrastructure, but there's no--

they're not intelligent enough to outsmart us at a level that would lead to, to

extinction. I think

what's just crazy is to look at the, the rate of progress, and there is a

very real possibility that in the immediate

future, these are years that like next year, the year after,

recursive self-improvement will happen, and we'll enter the phase of Evan's post

where he argues that there's a chance we could all die.

Like, that, that is coming soon.