If you indeed were solving a narrower task — that is, only creating the most sense of pleasure-inducing picture with maximization of other parameters — and then looked back, puzzled as to why the hungry weren’t fed by this procedure, bringing Goodhart’s law into the discussion is madness; it stresses me out. The variable ‘people are hungry’ wasn’t important for this task at all. Oh, or was it important to you? Then why didn’t you specify it? You think it’s ‘obvious’?
The point of Goodhart’s Law is that you can only select for what you can measure. The burger is a good analogy because Instagram can’t measure taste or nutrition, so when Instagram is what optimizes burgers, you get burgers with a very appealing appearance but non-optimized taste and nutrition. If you have the ability to measure taste, then you can create good taste, but you run into subtler examples of Goodhart (EG, Starbucks coffee is optimized to taste good to their professional tasters, which is slightly different from tasting good to a general audience).
Just specifying the variable you’re interested in doesn’t solve this problem; you also have to figure out how to measure it. The problem is that measurements are usually at least slightly statistically distinct from the actual target variable, so that the statistical connection can fall apart under optimization.
I also take issue with describing optimizing the appearance of the burger as “narrower” than optimizing the burger quality. In general it is a different task, which may be narrower or broader.
The point of Goodhart’s Law is that you can only select for what you can measure. The burger is a good analogy because Instagram can’t measure taste or nutrition, so when Instagram is what optimizes burgers, you get burgers with a very appealing appearance but non-optimized taste and nutrition. If you have the ability to measure taste, then you can create good taste, but you run into subtler examples of Goodhart (EG, Starbucks coffee is optimized to taste good to their professional tasters, which is slightly different from tasting good to a general audience).
Just specifying the variable you’re interested in doesn’t solve this problem; you also have to figure out how to measure it. The problem is that measurements are usually at least slightly statistically distinct from the actual target variable, so that the statistical connection can fall apart under optimization.
I also take issue with describing optimizing the appearance of the burger as “narrower” than optimizing the burger quality. In general it is a different task, which may be narrower or broader.