This sounds to me like an argument that inner optimizers are particularly likely to arise in imitation learning, because humans are pretty close to optimizers. Does that seem right?
I’m not sure how well this fits into the category of “inner optimizers”; I’m still organizing my thoughts on that (aiming to finish doing so within the week...). I’m also not sure that people are thinking about inner optimizers in the right way.
Also, note that the thing being imitated doesn’t have to be a human.
OTTMH, I’d say:
This seems more general in the sense that it isn’t some “subprocess” of the whole system that becomes a dangerous planning process.
This seems more specific in the sense that the boldest argument for inner optimizers is, I think, that they should appear in effectively any optimization problem when there’s enough optimization pressure.
This sounds to me like an argument that inner optimizers are particularly likely to arise in imitation learning, because humans are pretty close to optimizers. Does that seem right?
Yes, maybe? Elaborating...
I’m not sure how well this fits into the category of “inner optimizers”; I’m still organizing my thoughts on that (aiming to finish doing so within the week...). I’m also not sure that people are thinking about inner optimizers in the right way.
Also, note that the thing being imitated doesn’t have to be a human.
OTTMH, I’d say:
This seems more general in the sense that it isn’t some “subprocess” of the whole system that becomes a dangerous planning process.
This seems more specific in the sense that the boldest argument for inner optimizers is, I think, that they should appear in effectively any optimization problem when there’s enough optimization pressure.
Yeah, I agree with all of those clarifications.