The Future of AI and Human Control

Anything this power is also powerful is also intimidating.

This moment is complicated as we feel both tremendous potential and very

understandable anxiety at the same time.

These models can help people discover new knowledge, new medicines, better

healthcare, stronger education, and build more productive economies.

They can be new tools for people to pursue dreams and ambition that today feel

out of reach.

The complexity of life has increased so dramatically for the average person.

AI can be a defensive tool against this complexity as much as it can be this

tool of unprecedented discovery and innovation.

On the other hand, as AI systems become more capable and more

autonomous, they could move faster than our

institutions, concentrate

power in too f- too few hands, or make decisions that people no

longer understand or control.

That concern becomes especially important as we approach systems that can improve

themselves and future versions of themselves, often called recursive

self-improvement.

As the process of building AI becomes more automated, the pace of AI

progress could accelerate rapidly.

This moment calls for extreme care.

I largely agree with Professor Bengio,

uh, although I'll say two instead of three.

I believe there are two ways that AI progress could go very badly

that we have to

avoid.

First, we could lose control of the future to AI.

The risk is that it moves so fast that people can no longer follow what's

happening

or intervene when needed. This would obviously be terrible.

The industry must not accept too much technological risk just because the

benefits

are too great and that they feel too important to slow down.

Beating companies in a competitive race is not a reason to make rash decisions,

nor

do we believe we are locked in a race where we are unable to do that.

We have unilaterally slowed down in the past. We will do so in the future.

We do not want to build systems. We think it's important

that everyone avoids

building systems where we cannot make the alignment, monitorability, and

safety guarantees that we must.

We need to understand what these systems are doing and have strong evidence that

they will do what people intend, even as they get very, very smart.

Actually, especially as they get very, very smart.

It doesn't matter whether people put the risk of catastrophe at ten percent or one

percent or twelve percent or point one percent.

None of these levels are remotely acceptable,

and we should not train models that

we cannot make an extremely strong case they will be able to keep under human

control.

So that's one way things can go wrong.

In the other direction, these systems could concentrate too much

power in too few

hands.

No one person or company or country should be able to use the most

powerful

A- AI models to impose their worldview on

everyone else.

A company or country that believes only it can be trusted with

this technology can

use that belief to justify almost anything else.

We have to reject that logic even when it's convenient for any one actor.

I believe there is a pragmatic path through both of these.

At OpenAI, we approach these questions with a clear

goal.

AI should give people more power over their own lives,

more freedom to live lives

the way they want, more ability to solve problems, big

or small, and to improve

their communities.

We don't want to fall into the trap of blind optimism.

We also don't want to fall into the trap of doomerism either.

Getting pulled into either extreme will not help us successfully figure out the

challenges in front of us, especially at a time when the world

and its people

face such enormous challenges that this technology can help solve.

So we try to walk the middle path between these two poles.

For us, that means three principles.

First,

as AI becomes more capable, people must remain at the center of AI

decision-making.

Alignment is not an abstract research question.

It is the work of ensuring that these systems reliably remain under

human control,

reflect human values, and help people guide the next stages of

development.

We are not trying to and must not automate human judgment

or human values.

Second, the benefits of scientific progress

and economic growth must be

by people and for people. AI should help researchers make

discoveries, doctors

treat patients, teachers teach students, entrepreneurs start companies, communities

solve problems that felt out of reach, and much more.

The continuous story of human progress is not that tools replace

human agency. It is that the right tools make people more

capable, and I believe

this is going to go much further than we believe possible.

Third, this technology must empower people

individually.

A great future will come from people realizing their potential

and building value

for themselves, their communities, their countries, and the broader world.