AI in Military Warfare and Ethics

Bloomberg has reported that Claude is being used by the U.S. military

in the war in Iran to do AI-assisted targeting

via a platform made by Palantir, Maven's smart system.

In February, a U.S. missile reportedly hit a girls' school

in Iran, killing more than 150 people, most of them children.

Did Claude play a role in that strike?

We don't know exactly how, you know, these models were used.

You know, obviously,

like, you know,

these things that, you know, mistakes that happen in warfare are really,

really terrible, like, this is a really terrible thing to happen.

We were willing to risk the future of our company to,

like, limit how, you know, these models are used.

And you know, what you're talking about is a use case that

doesn't even violate our red lines.

We're worried that there will be 100 times as much,

you know, with use cases that do violate our red lines.

Now, you know, I, you know, again, again, I would say, I think, overall,

the use of these the use of these models is appropriate.

I think it's good on net.

You know, but military decision makers make terrible mistakes,

even even even at the best of times.

And I don't know if we're in the best of times.

What we've seen here is Claude assists, but a human makes the final call.

So a human made that final call, not Claude.

Imagine if you had a world in which not Claude,

because we haven't allowed it, but someone else's AI model,

the AI model just makes the decision and the human never sees it.

That's what we were standing up for.

That's what we were fighting against.

This school had a website.

You could have found it in a Google search,

like, shouldn't Claude have spotted that?

And it doesn't speak to a scarier issue about using technology as a shortcut in war.

The principle that was obeyed here is a human makes the human makes the final decision.

I don't know what role Claude or any other AI had,

but like, if this isn't an illustration why that principle is so important,

I don't know what is.

Is AI warfare more likely to stop World War Three,

a war between the U.S. and China,

or is it more likely to make it happen?

I would say on balance, it is more likely to stop it.

But if we have no limits on how it's used,

then I think, you know, it could be more likely to cause it.

You've seen Dr. Strangelove, right?

The premise of it was like, you know, you have a doomsday device

that automatically fires nuclear weapons when it thinks nuclear weapons are being fired at it.

What could go wrong, right?

I think the way conflicts happen is that, you know, the two sides jump at each other.

They misunderstand each other.

And when we don't have proper oversight of this technology, I think

those kinds of accidents are more likely to happen.