Think of it more as the argument that “If we needed to build an agent to select our joint choice for us and we can articulate our desires and settle on a mutually agreeable solution, then we can find weights for our utility functions such that the agent only needs to know a weighted sum of utility functions,”
I still disagree with this. I’ll restate/expand the argument that I made at the top of the previous thread. Suppose we want to use NBS or KSBS to make the joint choice. We could:
Compute the Pareto frontier, apply NBS/KSBS to find the mutually agreeable solution, use the slope of the tangent at that point to derive a set of weights, use those weights to form a linear aggregation of our utility functions, program the linear aggregation into a VNM AI, have the VNM AI recompute that solution we already found and apply it, or
Input our utility functions into an AI separately, program it to compute the Pareto frontier and apply NBS/KSBS to find the mutually agreeable solution and directly apply that solution.
It seems to me that in 1 you’re manually doing all of the work to make the actual decision outside of the VNM framework, and then tacking on a VNM AI at the end to do more redundant work. Why would you do that instead of 2?
You disagree with the statement that we can, or you disagree with the implication that we should?
Why would you do that instead of 2?
In practice, I don’t think you would need to. The point of the theorem is that you always can if you want to, and I’m not sure why this result is interesting to Nisan.
(Note also that this approach works for other metaethical approaches besides NBS/KSBS, and that you don’t always have access to NBS/KSBS.)
Yeah, I thought you meant to imply “should”. If we’re just talking about “can”, then I agree (with some caveats that aren’t very important at this point).
I still disagree with this. I’ll restate/expand the argument that I made at the top of the previous thread. Suppose we want to use NBS or KSBS to make the joint choice. We could:
Compute the Pareto frontier, apply NBS/KSBS to find the mutually agreeable solution, use the slope of the tangent at that point to derive a set of weights, use those weights to form a linear aggregation of our utility functions, program the linear aggregation into a VNM AI, have the VNM AI recompute that solution we already found and apply it, or
Input our utility functions into an AI separately, program it to compute the Pareto frontier and apply NBS/KSBS to find the mutually agreeable solution and directly apply that solution.
It seems to me that in 1 you’re manually doing all of the work to make the actual decision outside of the VNM framework, and then tacking on a VNM AI at the end to do more redundant work. Why would you do that instead of 2?
You disagree with the statement that we can, or you disagree with the implication that we should?
In practice, I don’t think you would need to. The point of the theorem is that you always can if you want to, and I’m not sure why this result is interesting to Nisan.
(Note also that this approach works for other metaethical approaches besides NBS/KSBS, and that you don’t always have access to NBS/KSBS.)
Yeah, I thought you meant to imply “should”. If we’re just talking about “can”, then I agree (with some caveats that aren’t very important at this point).