For this argument, consequentialism is like kinetic theory of gases. The point is not that it’s wrong and doesn’t work (where it should), but that it’s not a relevant tool for many purposes.
I started giving up on consequentialism when thinking about concepts of alignment like corrigibility and then membranes (respect for autonomy). They could in principle be framed as particular preferences, but that doesn’t appear to be a natural way of thinking about them, of formulating them more clearly. Even in decision theory, with the aim of getting certain outcomes to pass, my current preferred ontology of simulation-structure of things points more towards convincing other computations to move the world in certain ways than towards anticipating their behavior before they decide what it should be themselves. It’s still a sort of “consequentialism”, but the property of preferences being unchanging is not a centerpiece, and the updateless manipulation of everything else is more of a technical error (like two-boxing in ASP) than a methodology.
In human thinking, issues with consequentialism seem to be about losing sight of chasing the void. Reflectively endorsed hedonistic goals (in a broad sense, which could include enjoyment of achievement) are a bit of a dead end, denying the process of looking for different kinds of aims, sometimes cynical reveling in knowing the secrets of human nature.
Yeah, I’ve been thinking along similar lines. Consequentialism stumbles on the richness of other creatures, and ourselves. Stumbles in the sense that many of our wishes are natively expressed in our internal “creature language”, not the language of consequences in the world.
For this argument, consequentialism is like kinetic theory of gases. The point is not that it’s wrong and doesn’t work (where it should), but that it’s not a relevant tool for many purposes.
I started giving up on consequentialism when thinking about concepts of alignment like corrigibility and then membranes (respect for autonomy). They could in principle be framed as particular preferences, but that doesn’t appear to be a natural way of thinking about them, of formulating them more clearly. Even in decision theory, with the aim of getting certain outcomes to pass, my current preferred ontology of simulation-structure of things points more towards convincing other computations to move the world in certain ways than towards anticipating their behavior before they decide what it should be themselves. It’s still a sort of “consequentialism”, but the property of preferences being unchanging is not a centerpiece, and the updateless manipulation of everything else is more of a technical error (like two-boxing in ASP) than a methodology.
In human thinking, issues with consequentialism seem to be about losing sight of chasing the void. Reflectively endorsed hedonistic goals (in a broad sense, which could include enjoyment of achievement) are a bit of a dead end, denying the process of looking for different kinds of aims, sometimes cynical reveling in knowing the secrets of human nature.
Yeah, I’ve been thinking along similar lines. Consequentialism stumbles on the richness of other creatures, and ourselves. Stumbles in the sense that many of our wishes are natively expressed in our internal “creature language”, not the language of consequences in the world.