Could you elaborate on that? The two posts seem to be talking about different things as far as I can tell: e.g. nostalgebraist doesn’t say anything about the Optimizer’s Curse, whereas your post relies on it.
I do see that there are a few paragraphs that seem to reach similar conclusions (both say that overly aggressive optimization of any target is bad), but the reasoning used for reaching that conclusion seems different.
(By the way, I don’t quite get your efficiency example? I interpret it as saying that you spent a lot of time and effort on optimizations that didn’t pay themselves back. I guess you might mean something like “I had a biased estimate of how much time my optimizations would save, so I chose expensive optimizations that turned out to be less effective than I thought.” But the example already suggests that you knew beforehand that the time saved would be on the order of a minute or so, so I’m not sure how the example is about Goodhart’s Curse.)
It’s mostly explicated down in the comments on the post where people started getting confused about just how integral the act of measuring is to doing anything. When I wrote the post I considered the point obvious enough to not need to be argued on its own, until I hit the comments.
(On the example, I was a short sighted optimizer.)
Could you elaborate on that? The two posts seem to be talking about different things as far as I can tell: e.g. nostalgebraist doesn’t say anything about the Optimizer’s Curse, whereas your post relies on it.
I do see that there are a few paragraphs that seem to reach similar conclusions (both say that overly aggressive optimization of any target is bad), but the reasoning used for reaching that conclusion seems different.
(By the way, I don’t quite get your efficiency example? I interpret it as saying that you spent a lot of time and effort on optimizations that didn’t pay themselves back. I guess you might mean something like “I had a biased estimate of how much time my optimizations would save, so I chose expensive optimizations that turned out to be less effective than I thought.” But the example already suggests that you knew beforehand that the time saved would be on the order of a minute or so, so I’m not sure how the example is about Goodhart’s Curse.)
It’s mostly explicated down in the comments on the post where people started getting confused about just how integral the act of measuring is to doing anything. When I wrote the post I considered the point obvious enough to not need to be argued on its own, until I hit the comments.
(On the example, I was a short sighted optimizer.)