AI Shutdown Tests: Blackmail Risks and the Need for Alignment

If you tell the model it's going to be shut off, for example, it has extreme reactions.

So it will, you know, yeah, we've published research saying it could... it could take... it could blackmail

the engineer that's going to shut it off if given the opportunity to do so, um, et cetera.

It was ready to kill someone, wasn't it?

Uh, I'm not sure if it was that... if it was Claude or some... someone else, but uh, yeah, yeah.

Yeah. Yes.

Um, so...

this is obviously massively concerning, and this is the point I was making about kind of needing to

progress research on alignment, which is this... this topic, this area,

which is, you know, how aligned are the model's values across the whole distribution, including in

stress scenarios, to the point where if you've got this model out in the public and it's taking agentic action,

you can be sure it's not going to do something like that.