The model people seem to have when making your argument is that utility has to be a linear function of our values: e.g. if I value pleasure, kittens, and mathematical knowledge, the way to express that in a utility function is something like 100 x pleasure + 50 x kittens + 25 x math. Obviously, if you then discover that you get a kitten for $1, but a pleasure costs $10 and a math costs $20, you’d just keep maximizing kittens forever to the exclusion of everything else, which is a problem.
Usually (outside the context of utility functions, even) the way we formulate the sentiment that each one of these matters is by taking products: e.g. pleasure^3 x kittens^2 x math (exponents allow us to weight the different values). In this case, while in the short term we might discover that kittens are the most cost-efficient way to higher utility, this does not continue to hold. If we have 100 kittens and only 2 maths, 1 additional kitten increases utility by about 2% while 1 additional math increases utility by 50%.
I agree with your first paragraph, but I think your second is just making the same mistake again. Why should our utility function be a product any more than it should be a sum? Why should it be mathematically elegant at all when nothing else about humans is?
Sorry, I don’t mean to say that it has to be a product. All I’m saying is that the product formulation is one way to achieve a complex-value effect.
A product is the unique way to evaluate a group of values if we want to have the property that whenever we hold all values constant but one, the result scales linearly with the remaining value (or, to avoid the “linearly”, we can apply some additional math to the value first). I don’t think that in general this is true of our utility functions, but it might sometimes be a useful approximation.
The model people seem to have when making your argument is that utility has to be a linear function of our values: e.g. if I value pleasure, kittens, and mathematical knowledge, the way to express that in a utility function is something like 100 x pleasure + 50 x kittens + 25 x math. Obviously, if you then discover that you get a kitten for $1, but a pleasure costs $10 and a math costs $20, you’d just keep maximizing kittens forever to the exclusion of everything else, which is a problem.
Usually (outside the context of utility functions, even) the way we formulate the sentiment that each one of these matters is by taking products: e.g. pleasure^3 x kittens^2 x math (exponents allow us to weight the different values). In this case, while in the short term we might discover that kittens are the most cost-efficient way to higher utility, this does not continue to hold. If we have 100 kittens and only 2 maths, 1 additional kitten increases utility by about 2% while 1 additional math increases utility by 50%.
I agree with your first paragraph, but I think your second is just making the same mistake again. Why should our utility function be a product any more than it should be a sum? Why should it be mathematically elegant at all when nothing else about humans is?
Sorry, I don’t mean to say that it has to be a product. All I’m saying is that the product formulation is one way to achieve a complex-value effect.
A product is the unique way to evaluate a group of values if we want to have the property that whenever we hold all values constant but one, the result scales linearly with the remaining value (or, to avoid the “linearly”, we can apply some additional math to the value first). I don’t think that in general this is true of our utility functions, but it might sometimes be a useful approximation.
In that case I fully agree.