This post has been a core part of how I think about Goodhart’s Law. However, when I went to search for it just now, I couldn’t find it, because I was using Goodhart’s Law as a search term, but it doesn’t appear anywhere in the text or in the comments.
So, I thought I’d mention the connection, to make this post easier for my future self and others to find. Also, other forms of this include:
Hypocrisy (optimizing for appearances of good rather than becoming good)
The importance of quick feedback loops for course correction (Even good estimates have error, and as you get closer to your goal those errors compound, and things come apart at the tails.)
Maybe it would be useful to map out as many of the forms of Goodhart’s Law as possible, Turchin style.
This post has been a core part of how I think about Goodhart’s Law. However, when I went to search for it just now, I couldn’t find it, because I was using Goodhart’s Law as a search term, but it doesn’t appear anywhere in the text or in the comments.
So, I thought I’d mention the connection, to make this post easier for my future self and others to find. Also, other forms of this include:
Campbell’s law: https://en.wikipedia.org/wiki/Campbell%27s_law
The Cobra effect: https://en.wikipedia.org/wiki/Cobra_effect
Teaching to the test: https://en.wikipedia.org/wiki/Teaching_to_the_test
Perverse incentives: https://en.wikipedia.org/wiki/Perverse_incentive
Principal-Agent Problem: https://en.wikipedia.org/wiki/Principal%E2%80%93agent_problem
Hypocrisy (optimizing for appearances of good rather than becoming good)
The importance of quick feedback loops for course correction (Even good estimates have error, and as you get closer to your goal those errors compound, and things come apart at the tails.)
Maybe it would be useful to map out as many of the forms of Goodhart’s Law as possible, Turchin style.