What does merging utility functions look like and are you sure it’s not going to look the same as global free trade? It’s arguable that trade is just a way of breaking down and modularizing a big multifaceted problem over a lot of subagent task specialists (and there’s no avoiding having subagents, due to the light speed limit)
By the way I’d love to hear people giving my comment agreement karma explain what they’re agreeing with and how they know it’s true, because I was asking a question that I don’t know the answer to, and I really hope people don’t think that we know the answer, unless we do, in which case I’d like to hear it.
Taken literally, the only way to merge n utility functions into one without any other info (eg the preferences that generated the utility functions) is to do a weighted sum. There’s only n-1 free parameters.
So you think it’s computationally tractable? I think there are some other factors you’re missing. That’s a weighted sum of a bunch of vectors assigning numbers to all possible outcomes, either all possible histories+final states of the universe, or all possible experiences. And there are additional complications with normalizing utility functions; you don’t know the probability distribution of final outcomes (so you can’t take the integral of the utility functions) until you already know how the aggregation of normalized weighted utility functions is going to influence it.
What does merging utility functions look like and are you sure it’s not going to look the same as global free trade? It’s arguable that trade is just a way of breaking down and modularizing a big multifaceted problem over a lot of subagent task specialists (and there’s no avoiding having subagents, due to the light speed limit)
By the way I’d love to hear people giving my comment agreement karma explain what they’re agreeing with and how they know it’s true, because I was asking a question that I don’t know the answer to, and I really hope people don’t think that we know the answer, unless we do, in which case I’d like to hear it.
Taken literally, the only way to merge n utility functions into one without any other info (eg the preferences that generated the utility functions) is to do a weighted sum. There’s only n-1 free parameters.
So you think it’s computationally tractable? I think there are some other factors you’re missing. That’s a weighted sum of a bunch of vectors assigning numbers to all possible outcomes, either all possible histories+final states of the universe, or all possible experiences. And there are additional complications with normalizing utility functions; you don’t know the probability distribution of final outcomes (so you can’t take the integral of the utility functions) until you already know how the aggregation of normalized weighted utility functions is going to influence it.