I don’t see how I could agree with this conclusion :
But many people don’t like this, usually for reasons involving utility monsters. If you are one of these people, then you better learn to like it, because according to Harsanyi’s Social Aggregation Theorem, any alternative can result in the supposedly Friendly AI making a choice that is bad for every member of the population.
The Social Aggregation Theorem doesn’t just show that some particular way of aggregating utility functions other than by linear combination is bad; it shows that every way of aggregating utility functions other than by linear combination is bad.
I don’t see how I could agree with this conclusion :
If both ways are wrong, then you haven’t tried hard enough yet.
Well explained though.
The Social Aggregation Theorem doesn’t just show that some particular way of aggregating utility functions other than by linear combination is bad; it shows that every way of aggregating utility functions other than by linear combination is bad.