Transcript · English → Japanese
English → Japanese transcript
例えばモデルに「電源
を切られる」
と伝えると、
極端な反応を
示すんです。
なので、その、ええと、
そうですね、
それについては、
脅迫行為が可能になり
得ると電源を切ろう
としているエンジニアを、
もしその機会が与
えられれば脅迫できる、
といった研究結果
も公開しています。
誰かを殺す準備までできて
いたんですよね？
それがクロードだったのか、
それとも別のモデルだった
のかはちょっと分からない
んですが、ええ、そうですね。
うん。そうです。
ええと、なのでこれ
は明らかに非常
に深刻な懸念で、
そしてまさに私が言おうと
していたポイントは、
アラインメントに関する
研究を進める必要が
あるということなんです。
このテーマ、この分野ですね。
つまり、その、モデルの
価値観がストレスのかかる
状況も含めて全体
の分布にわたってどれだけ
人間と整合しているか、
ということで、
公開環境でこのモデル
がエージェント的
な行動を取るような
状況になったとき
でも通用するレベルまで、
ということです。
そんなことを
することは
ないと安心
していてい
いですよ。
End of transcript · 214 words · machine translated
Original transcript (English)
if you tell the model it's going to be shut off, for example, it has extreme reactions. if you tell the model it's going to be shut off, for example, it has extreme reactions. So it will, you know, yeah, we've published research saying it could it could take it could blackmail So it will, you know, yeah, we've published research saying it could it could take it could blackmail So it will, you know, yeah, we've published research saying it could it could take it could blackmail
the engineer that's going to shut it off if given the opportunity to do so, um, et cetera. the engineer that's going to shut it off if given the opportunity to do so, um, et cetera. It was ready to kill someone, wasn't it? It was ready to kill someone, wasn't it? I'm not sure if that if it was Claude or some someone else, but uh, yeah, yeah.
I'm not sure if that if it was Claude or some someone else, but uh, yeah, yeah. I'm not sure if that if it was Claude or some someone else, but uh, yeah, yeah. Yeah. Yes. Yeah. Yes. Yeah. Yes.
Um, so this is obviously massively concerning and this is the point I was making about Um, so this is obviously massively concerning and this is the point I was making about kind of needing to progress research on alignment, which is this this topic, this area, kind of needing to progress research on alignment, which is this this topic, this area, kind of needing to progress research on alignment, which is this this topic, this area,
which is, you know, how aligned are the model's values across the whole distribution including in stress scenarios, which is, you know, how aligned are the model's values across the whole distribution including in stress scenarios, which is, you know, how aligned are the model's values across the whole distribution including in stress scenarios, to the point where if you've got this model out in the public and it's taking agentic action, to the point where if you've got this model out in the public and it's taking agentic action,
to the point where if you've got this model out in the public and it's taking agentic action, you can be sure it's not going to do something like that. you can be sure it's not going to do something like that. you can be sure it's not going to do something like that.
