Corrigibility is doing a ton of heavy lifting in this picture. Humans obviously wouldn’t approve of daemons, so your AI would just try really hard to not do that.
I’m a bit confused about how “corrigibility” is being used here. I thought it meant that the agent doesn’t resist correction, but here it seems to be used to mean something more like trying to only do things the overseer would approve of.
I thought we called the latter being “approval-directed” and that it was a separate idea from corrigibility. Am I confused?
Oops, I think I was conflating “corrigible agent” with “benign act-based agent”. You’re right that they’re separate ideas. I edited my original comment accordingly.
I’m a bit confused about how “corrigibility” is being used here. I thought it meant that the agent doesn’t resist correction, but here it seems to be used to mean something more like trying to only do things the overseer would approve of.
I thought we called the latter being “approval-directed” and that it was a separate idea from corrigibility. Am I confused?
Oops, I think I was conflating “corrigible agent” with “benign act-based agent”. You’re right that they’re separate ideas. I edited my original comment accordingly.