the thing that is identical is that you are trading utilities across people,
This is either wrong (the utility functions of the people involved aren’t queried in the dust speck problem) or so generic as to be encompassed in the concept of “utility calculation”.
Aggregating utility functions across different people is an unsolved problem, but not necessarily an unsolvable one. One way of avoiding utility monsters would be to normalize utility functions. The obvious way to do that leads to problems such as arachnophobes getting less cake even if they like cake equally much, but IMO that’s better than utility monsters.
Are you saying that the demographic you are talking about is special in using prejudice as the marker of evilness (as opposed to religious affiliation or whatever), or in taking that sort of attitude at all?