I would love to see a more rigorous way to measure optimization power. It’s a very important concept, and in fact we probably care more about optimization power than intelligence. On some days I’d rather talk about the possibility of an “optimization power explosion” than an “intelligence explosion.”
I don’t think a formalization has been made. I think Carl once suggested we might define optimization power in terms of capacity to achieve narrow goals across a canonical set of environments, but that’s a long way from a formalization.
I would love to see a more rigorous way to measure optimization power. It’s a very important concept, and in fact we probably care more about optimization power than intelligence. On some days I’d rather talk about the possibility of an “optimization power explosion” than an “intelligence explosion.”
I don’t think a formalization has been made. I think Carl once suggested we might define optimization power in terms of capacity to achieve narrow goals across a canonical set of environments, but that’s a long way from a formalization.