This post’s main contribution is the formalization of game-theoretic defection as gaining personal utility at the expense of coalitional utility.
Rereading, the post feels charmingly straightforward and self-contained. The formalization feels obvious in hindsight, but I remember being quite confused about the precise difference between power-seeking and defection—perhaps because popular examples of taking over the world are also defections against the human/AI coalition. I now feel cleanly deconfused about this distinction. And if I was confused about it, I’d bet a lot of other people were, too.
I think this post is valuable as a self-contained formal insight into the nature of defection. If I could vote on it, I’d give it a 4 (or perhaps a 3, if the voting system allowed it).
This post’s main contribution is the formalization of game-theoretic defection as gaining personal utility at the expense of coalitional utility.
Rereading, the post feels charmingly straightforward and self-contained. The formalization feels obvious in hindsight, but I remember being quite confused about the precise difference between power-seeking and defection—perhaps because popular examples of taking over the world are also defections against the human/AI coalition. I now feel cleanly deconfused about this distinction. And if I was confused about it, I’d bet a lot of other people were, too.
I think this post is valuable as a self-contained formal insight into the nature of defection. If I could vote on it, I’d give it a 4 (or perhaps a 3, if the voting system allowed it).