Making the hyperplane argument as before, we get a π which places positive weight on each individual. This is interpreted as each individual’s weight in the coalition. The collective decision must be the result of a (positive) linear combination of each individual’s cardinal utilities—and those cardinal utilities can in turn be constructed via an application of VNM to individual ordinal preferences.
What this says is that any Pareto-optimal outcome can be rationalized as maximizing a positive linear combination of individual utilities, not that it can be generated in this way. For example, Nash bargaining results in Pareto optimal outcomes, yet it can’t be specified as the unique maximization of some positive linear combination of individual utilities. After running the algorithm, the result is optimal according to some linear combination of individual utilities, but this is a rationalization rather than the actual generation procedure. (This also works as a criticism of Bayesianism)
I basically agree with this criticism, and would like to understand what the alternative to Bayesian decision theory which comes out of the analogy would be.
I think when several AIs with bounded utility functions decide to merge, they can reach any point on the Pareto frontier like this:
1) Allow linear combinations of utility functions. This lets you reach all “pointy” points.
2) Allow making a tuple of functions of type (1) whose values should be compared lexicographically (e.g. “maximize U+V, break ties by maximizing U”). This lets you reach some points on the edges of flat parts.
3) Allow the merging process to choose randomly which function of type (2) to give to the merged AI. This lets you reach the rest of the points on flat parts.
That’s a bit complicated, but I don’t think there’s a simpler way.
I don’t see why 2 is necessary given that any point on the Pareto frontier is a mixture of pointy points (intuition for this: any point on the face of a polyhedron is a mixture of that face’s corners). In any case, I agree with the basic mathematical point that you can get any Pareto optimal mixture of outcomes by mixing between non-negative linear combinations of utility functions.
Well, I was imagining a Pareto frontier that changes smoothly from flat to curved. Then we can’t quite get a pointy point exactly on the edge of the flat part. That’s what 2 is for, it gives us some of these points (though not all). But I guess that doesn’t matter if things are finite enough.
What this says is that any Pareto-optimal outcome can be rationalized as maximizing a positive linear combination of individual utilities, not that it can be generated in this way. For example, Nash bargaining results in Pareto optimal outcomes, yet it can’t be specified as the unique maximization of some positive linear combination of individual utilities. After running the algorithm, the result is optimal according to some linear combination of individual utilities, but this is a rationalization rather than the actual generation procedure. (This also works as a criticism of Bayesianism)
I basically agree with this criticism, and would like to understand what the alternative to Bayesian decision theory which comes out of the analogy would be.
I think when several AIs with bounded utility functions decide to merge, they can reach any point on the Pareto frontier like this:
1) Allow linear combinations of utility functions. This lets you reach all “pointy” points.
2) Allow making a tuple of functions of type (1) whose values should be compared lexicographically (e.g. “maximize U+V, break ties by maximizing U”). This lets you reach some points on the edges of flat parts.
3) Allow the merging process to choose randomly which function of type (2) to give to the merged AI. This lets you reach the rest of the points on flat parts.
That’s a bit complicated, but I don’t think there’s a simpler way.
I don’t see why 2 is necessary given that any point on the Pareto frontier is a mixture of pointy points (intuition for this: any point on the face of a polyhedron is a mixture of that face’s corners). In any case, I agree with the basic mathematical point that you can get any Pareto optimal mixture of outcomes by mixing between non-negative linear combinations of utility functions.
Well, I was imagining a Pareto frontier that changes smoothly from flat to curved. Then we can’t quite get a pointy point exactly on the edge of the flat part. That’s what 2 is for, it gives us some of these points (though not all). But I guess that doesn’t matter if things are finite enough.
Ok, that seems right.