I’ve laid out a concrete example of this at https://www.lesswrong.com/posts/FgXjuS4R9sRxbzE5w/medical-image-registration-the-obscure-field-where-deep , following the “optimization on a scaffold level” route. I found a real example of a misaligned inner objective outside of RL, which is cool
I’ve laid out a concrete example of this at https://www.lesswrong.com/posts/FgXjuS4R9sRxbzE5w/medical-image-registration-the-obscure-field-where-deep , following the “optimization on a scaffold level” route. I found a real example of a misaligned inner objective outside of RL, which is cool