Yeah, but I guess I’m still not communicating the part where a very large and important section of your brain doesn’t just adopt the goals you consciously give it. The stuff you said above is true, but irrelevant in this context/overwhelmed or undermined by the effect the post is pointing at, which makes me continue to feel like you’re not receiving the point I’m trying to convey.
I hundred percent agree that most of the brain is below conscious control and doesn’t adopt your goals. What I think Goodhearts law should be guiding us towards is how we set the bits of the brain that are.
For example of losing weight by measuring weight and using that as a metric is *literally* setting the metric and measure the same. I was trying to point out a way that a measure could be used in pursuit of a goal, but that it could not be a treated like a metric.
I didn’t get that impression from your post that you thought that there was any sort of conscious thing you could do before hand to try and head off the worst bits of Goodhearts. It was only post-hoc noticing things are going wrong. Like noticing “hey I am about to start on a poorly understood problem, that my subconscious brain will optimise wrong if I just set solving it as my goal. Maybe instead I should try to understand it first, using the measure I was about to use as a metric.”
Yeah, but I guess I’m still not communicating the part where a very large and important section of your brain doesn’t just adopt the goals you consciously give it. The stuff you said above is true, but irrelevant in this context/overwhelmed or undermined by the effect the post is pointing at, which makes me continue to feel like you’re not receiving the point I’m trying to convey.
We may be agreeing, but not being clear!
I hundred percent agree that most of the brain is below conscious control and doesn’t adopt your goals. What I think Goodhearts law should be guiding us towards is how we set the bits of the brain that are.
For example of losing weight by measuring weight and using that as a metric is *literally* setting the metric and measure the same. I was trying to point out a way that a measure could be used in pursuit of a goal, but that it could not be a treated like a metric.
I didn’t get that impression from your post that you thought that there was any sort of conscious thing you could do before hand to try and head off the worst bits of Goodhearts. It was only post-hoc noticing things are going wrong. Like noticing “hey I am about to start on a poorly understood problem, that my subconscious brain will optimise wrong if I just set solving it as my goal. Maybe instead I should try to understand it first, using the measure I was about to use as a metric.”