Crudely, something like this: Divide the value-laden world up into individuals and into short time-slices. Rate the happiness of each individual in each time-slice on a scale from −1 to +1. (So we suppose there are limits to momentary intensity of satisfaction or dissatisfaction, which seems reasonable to me.) Now, let h be a tiny positive number, and assign overall utility 1/h tanh(sum of atanh(h * local utility)).
Because h is extremely small, for modest lifespans and numbers of agents this is very close to net utility = sum of local utility. But we never get |overall utility| > 1/h.
Now, this feels like an ad hoc trick rather than a principled description of how we “should” value things, and I am not seriously proposing it as what anyone’s utility function “should” (or does) look like. But I think one could make something of a case for an agent that works more like this: just add up local utilities linearly, but weight the utility for agent A in timeslice T in a way that decreases exponentially (with small constant) with the description-length of (A,T), where the way in which we describe things is inevitably somewhat referenced to ourselves. So it’s pretty easy to say “me, now” and not that much harder to say “my wife, an hour from now”, so these two are weighted similarly; but you need a longer description to specify a person and time much further away, and if you have 3^^^3 people then you’re going to get weights about as small as 1/3^^^3.
You have a matrix of (number of individuals) x (number of time-slices). Each matrix cell has value (“happiness”) that’s constrained to lie in the [-1..1] interval. You call the cell value “local utility”, right?
And then you, basically, sum up the cell values, re-scale the sum to fit into a pre-defined range and, in the process, add a transformation that makes sure the bounds are not sharp cut-offs, but rather limits which you approach asymptotically.
As to the second part, I have trouble visualising the language in which the description-length would work as you want. It seems to me it will have to involve a lot scaffolding which might collapse under its own weight.
“You have a matrix …”: correct. “And then …”: whether that’s correct depends on what you mean by “in the process”, but it’s certainly not entirely unlike what I meant :-).
Your last paragraph is too metaphorical for me to work out whether I share your concerns. (My description was extremely handwavy so I’m in no position to complain.) I think the scaffolding required is basically just the agent’s knowledge. (To clarify a couple of points: not necessarily minimum description length, which of course is uncomputable, but something like “shortest description the agent can readily come up with”; and of course in practice what I describe is way too onerous computationally but some crude approximation might be manageable.)
The basic issue is whether the utility weights (“description lengths”) reflect the subjective preferences. If they do, it’s an entirely different kettle of fish. If they don’t, I don’t see why “my wife” should get much more weight than “the girl next to me on a bus”.
I think real people have preferences whose weights decay with distance—geographical, temporal and conceptual. I think it would be reasonable for artificial agents to do likewise. Whether the particular mode of decay I describe resembles real people’s, or would make an artificial agent tend to behave in ways we’d want, I don’t know. As I’ve already indicated, I’m not claiming to be doing more than sketch what some kinda-plausible bounded-utility agents might look like.
Crudely, something like this: Divide the value-laden world up into individuals and into short time-slices. Rate the happiness of each individual in each time-slice on a scale from −1 to +1. (So we suppose there are limits to momentary intensity of satisfaction or dissatisfaction, which seems reasonable to me.) Now, let h be a tiny positive number, and assign overall utility 1/h tanh(sum of atanh(h * local utility)).
Because h is extremely small, for modest lifespans and numbers of agents this is very close to net utility = sum of local utility. But we never get |overall utility| > 1/h.
Now, this feels like an ad hoc trick rather than a principled description of how we “should” value things, and I am not seriously proposing it as what anyone’s utility function “should” (or does) look like. But I think one could make something of a case for an agent that works more like this: just add up local utilities linearly, but weight the utility for agent A in timeslice T in a way that decreases exponentially (with small constant) with the description-length of (A,T), where the way in which we describe things is inevitably somewhat referenced to ourselves. So it’s pretty easy to say “me, now” and not that much harder to say “my wife, an hour from now”, so these two are weighted similarly; but you need a longer description to specify a person and time much further away, and if you have 3^^^3 people then you’re going to get weights about as small as 1/3^^^3.
Let me see if I understand you correctly.
You have a matrix of (number of individuals) x (number of time-slices). Each matrix cell has value (“happiness”) that’s constrained to lie in the [-1..1] interval. You call the cell value “local utility”, right?
And then you, basically, sum up the cell values, re-scale the sum to fit into a pre-defined range and, in the process, add a transformation that makes sure the bounds are not sharp cut-offs, but rather limits which you approach asymptotically.
As to the second part, I have trouble visualising the language in which the description-length would work as you want. It seems to me it will have to involve a lot scaffolding which might collapse under its own weight.
“You have a matrix …”: correct. “And then …”: whether that’s correct depends on what you mean by “in the process”, but it’s certainly not entirely unlike what I meant :-).
Your last paragraph is too metaphorical for me to work out whether I share your concerns. (My description was extremely handwavy so I’m in no position to complain.) I think the scaffolding required is basically just the agent’s knowledge. (To clarify a couple of points: not necessarily minimum description length, which of course is uncomputable, but something like “shortest description the agent can readily come up with”; and of course in practice what I describe is way too onerous computationally but some crude approximation might be manageable.)
The basic issue is whether the utility weights (“description lengths”) reflect the subjective preferences. If they do, it’s an entirely different kettle of fish. If they don’t, I don’t see why “my wife” should get much more weight than “the girl next to me on a bus”.
I think real people have preferences whose weights decay with distance—geographical, temporal and conceptual. I think it would be reasonable for artificial agents to do likewise. Whether the particular mode of decay I describe resembles real people’s, or would make an artificial agent tend to behave in ways we’d want, I don’t know. As I’ve already indicated, I’m not claiming to be doing more than sketch what some kinda-plausible bounded-utility agents might look like.