Goodhart effect is a truly deep insight IMO. Goodhart itself states that lots of great metrics and “signals”, and in general, communicatory material, will lose its value when adopted by social systems. This leads to second order effects, such as people and society defending against this effect. This can be seen in speech easily: if your friend did not depend on your speech for various purposes, your speech would be a truly great way to learn about your “true” self. But now that he does depend on your speech, he has to guard against you lying and optimizing your speech to manipulate him and shift risks unto him. So he cannot accept your noncommittal stance on, e.g., “murder,” and requires you to commit your position. I have heard there are studies (beware replication crisis and yadada) that women prefer men with virtue ethics over utilitarians; One reason might be that virtue ethics constrains people much more effectively than “consequentialism,” where you’re your own auditor.
Goodhart effect is a truly deep insight IMO. Goodhart itself states that lots of great metrics and “signals”, and in general, communicatory material, will lose its value when adopted by social systems. This leads to second order effects, such as people and society defending against this effect. This can be seen in speech easily: if your friend did not depend on your speech for various purposes, your speech would be a truly great way to learn about your “true” self. But now that he does depend on your speech, he has to guard against you lying and optimizing your speech to manipulate him and shift risks unto him. So he cannot accept your noncommittal stance on, e.g., “murder,” and requires you to commit your position. I have heard there are studies (beware replication crisis and yadada) that women prefer men with virtue ethics over utilitarians; One reason might be that virtue ethics constrains people much more effectively than “consequentialism,” where you’re your own auditor.