For the more general case, I would search for some excuses why I can be a good utilitarian while stopping the monster. And hope that I actually find a convincing argument. Maybe I think that most good in the world needs strong cooperation which is undermined by the existence of utility monsters.
I mean, if it’s about looking for post-hoc rationalizations, what’s even the point of pretending there’s a consistent ethical system? Might as well go “fuck the utility monster” and blast it to hell with no further justification than sheer human chauvinism. I think we need a bit of that in fact in the face of AI issues—some of the most extreme e/acc people seem indeed to think that an ASI would be such a utility monster and “deserves” to take over for… reasons, reasons that I personally don’t really give a toss about.
But then, there could still be a stereotypical Caren who actually has very strong emotions/qualia/the-thing-that-matters-for-utility.
Having no access to the internal experiences of anyone else, how do we even tell? With humans, we assume we can know because we assume they’re kinda like us. And we’re probably often wrong on that too! People seem to experience pain, for example, both physical and mental, on very different scales, both in terms of expression and of how it actually affects their functioning. Does this mean some people feel the same pain but can power through it, or does it mean they feel objectively less? Does the question even make sense? If you start involving non-human entities, we have essentially zero reference frame to judge. Outward behavior is all we have, and it’s not a lot to go by.
I mean, if it’s about looking for post-hoc rationalizations, what’s even the point of pretending there’s a consistent ethical system?
Hmm, I would not describe it as rationalization in the motivated reasoning sense.
My model of this process is that most of my ethical intuitions are mostly a black-box and often contradictory, but still in the end contain a lot more information about what I deem good than any of the explicit reasoning I am capable of. If however, I find an explicit model which manages to explain my intuitions sufficiently well, I am willing to update or override my intuitions.
I would in the end accept an argument that goes against some of my intuitions if it is strong enough. But I will also strive to find a theory which manages to combine all the intuitions into a functioning whole.
In this case, I have an intuition towards negative utilitarianism, which really dislikes utility monsters, but I also have noticed the tendency that I land closer to symmetric utilitarianism when I use explicit reasoning. Due to this, the likely options are that after further reflection I
would be convinced that utility monsters are fine, actually.
would come to believe that there are strong utilitarian arguments to have a policy against utility monsters such that in practice they would almost always be bad
would shift in some other direction
and my intuition for negative utilitarianism would prefer cases 2 or 3.
So the above description was what was going on in my mind, and combined with the always-present possibility that I am bullshitting myself, led to the formulation I used :)
I mean, if it’s about looking for post-hoc rationalizations, what’s even the point of pretending there’s a consistent ethical system? Might as well go “fuck the utility monster” and blast it to hell with no further justification than sheer human chauvinism. I think we need a bit of that in fact in the face of AI issues—some of the most extreme e/acc people seem indeed to think that an ASI would be such a utility monster and “deserves” to take over for… reasons, reasons that I personally don’t really give a toss about.
Having no access to the internal experiences of anyone else, how do we even tell? With humans, we assume we can know because we assume they’re kinda like us. And we’re probably often wrong on that too! People seem to experience pain, for example, both physical and mental, on very different scales, both in terms of expression and of how it actually affects their functioning. Does this mean some people feel the same pain but can power through it, or does it mean they feel objectively less? Does the question even make sense? If you start involving non-human entities, we have essentially zero reference frame to judge. Outward behavior is all we have, and it’s not a lot to go by.
Hmm, I would not describe it as rationalization in the motivated reasoning sense.
My model of this process is that most of my ethical intuitions are mostly a black-box and often contradictory, but still in the end contain a lot more information about what I deem good than any of the explicit reasoning I am capable of. If however, I find an explicit model which manages to explain my intuitions sufficiently well, I am willing to update or override my intuitions. I would in the end accept an argument that goes against some of my intuitions if it is strong enough. But I will also strive to find a theory which manages to combine all the intuitions into a functioning whole.
In this case, I have an intuition towards negative utilitarianism, which really dislikes utility monsters, but I also have noticed the tendency that I land closer to symmetric utilitarianism when I use explicit reasoning. Due to this, the likely options are that after further reflection I
would be convinced that utility monsters are fine, actually.
would come to believe that there are strong utilitarian arguments to have a policy against utility monsters such that in practice they would almost always be bad
would shift in some other direction
and my intuition for negative utilitarianism would prefer cases 2 or 3.
So the above description was what was going on in my mind, and combined with the always-present possibility that I am bullshitting myself, led to the formulation I used :)