Also, as a related side-point, I’d add Cobra effect to the list.
Model: V is the true goal, but can’t be incentivized. R is an easily measured consequence of V which can be accomplished other ways. If any of the other ways to acheive R are easier than accomplishing the true goal, Agents will pursue that instead of the intended goal.
Example: A baskeball player wants to impress the NBA recruiters, so he pays the other team to lose the game. They do so by not showing up, thereby forfeiting and losing, as agreed.
Also, as a related side-point, I’d add Cobra effect to the list.
Model: V is the true goal, but can’t be incentivized. R is an easily measured consequence of V which can be accomplished other ways. If any of the other ways to acheive R are easier than accomplishing the true goal, Agents will pursue that instead of the intended goal.
Example: A baskeball player wants to impress the NBA recruiters, so he pays the other team to lose the game. They do so by not showing up, thereby forfeiting and losing, as agreed.