Yeah I think non-Archimedian values are normatively irrational, i.e. make an AI less effective as an optimizer. The obvious problem is that when A is “infinitely preferable” to B, then even A’s most half-baked subgoal is infinitely preferable to B. Even if I gave the AI a wrapped box with B inside, it wouldn’t dedicate any resources to opening it—better to just flap its arms so that the butterfly effect will increase the chance of A by 1/googol.
better to just flap its arms so that the butterfly effect will increase the chance of A by 1/googol
Heh, yeah that’s roughly how I feel when noting Archimedeanity of my values. But then I wonder… maybe I wouldn’t “flap my arms just so” to increase P(A) because I’m running on hostile hardware that makes my belief probabilities coarse-grained.… i.e. maybe I’m forced to treat 1/googol like 0, and open the box with B in it. I certainly feel that way.
Such reflection leads me to think that humans aren’t precise enough for the difference between VNM utility and Hausner utility to really manifest decisively. When would a human really be convinced enough that EUBig(X) precisely equals EUBig(Y), so to start optimizing EUSmall? It seems like the difference between VNM and Hausner utility only happens in a measure-0 class of scenarios that humans couldn’t practically detect anyway. ETA: except maybe when there’s a time limit...
This is actually one reason I posted on Hausner utility: if you like it, then note it’s 0% likely to give you different answers from VNM utility, and then just use VNM because you’re not precise enough to know the difference :)
Is there any use for introducing the concept of “noise terms” in lexicographical expected value?
Here’s an illustration of what I mean by “noise terms”:
Suppose that you have a warm cup of coffee sitting in a room. How long will it take before the coffee is the same temperature as the room? Well, the rate of heat transfer between two objects is proportional to the temperature difference between them. That means that the temperature difference between the coffee and the room undergoes exponential decay, and exponential decay never actually reaches its final value. It’ll take infinitely long for the coffee to reach the same temperature as the room!
Except that the real world doesn’t quite work that way. If you try to measure temperature precisely enough, you’ll start finding that your thermometer readings are fluctuating. All those “random” molecular motions create noise. You can’t measure the difference between 100 degrees and 100 + epsilon degrees as long as epsilon is less than the noise level of the system. So the coffee really does reach room temperature, because the difference becomes so small that it disappears into the noise.
If the parameters that I use to calculate expected value are noisy, then as long as the difference between EUBig(X) and EUBig(Y) is small enough—below the noise level of the system—I can’t tell if it’s positive or negative, so I can’t know if I should prefer X to Y. As with the coffee in the room, the difference vanishes into the noise. So I’ll resort to the tiebreaker, and optimize EUSmall instead.
This certainly has an intuitive appeal. Where would the noise be? In the outcome values, or the probabilities, or both?
To have an effect, the noise would have to something that you somehow know can’t be eliminated by allocating more thinking/computing time (perhaps you’re on a time limit) or resources, else the rational thing to do would be to just “’think harder” to make sure you’re not sacrificing your EUBig...
It’s not really an issue of impracticality. It’s just that, any time you have a higher-level class of utility, the lower-level classes of utility stop being relevant to your decisions. No matter how precise the algorithm is. That’s why I say it’s extra complexity with no optimization benefit. Since the extra structure doesn’t even map better to my intuition about preference, I just Occam-shave it away.
any time you have a higher-level class of utility, the lower-level classes of utility stop being relevant to your decisions
Wait… certainly, if you lexicographically value (brightness, redness) of a light, and somehow manage to be in a scenario where you can’t make the light brighter, and somehow manage to know that, then the redness value becomes relevant.
What I mean is that the environment itself makes such precise situations rare (a non-practical issue), and an imprecise algorithm makes it hard to detect when, if ever, they occur (a practical issue).
Yeah I think non-Archimedian values are normatively irrational, i.e. make an AI less effective as an optimizer. The obvious problem is that when A is “infinitely preferable” to B, then even A’s most half-baked subgoal is infinitely preferable to B. Even if I gave the AI a wrapped box with B inside, it wouldn’t dedicate any resources to opening it—better to just flap its arms so that the butterfly effect will increase the chance of A by 1/googol.
Heh, yeah that’s roughly how I feel when noting Archimedeanity of my values. But then I wonder… maybe I wouldn’t “flap my arms just so” to increase P(A) because I’m running on hostile hardware that makes my belief probabilities coarse-grained.… i.e. maybe I’m forced to treat 1/googol like 0, and open the box with B in it. I certainly feel that way.
Such reflection leads me to think that humans aren’t precise enough for the difference between VNM utility and Hausner utility to really manifest decisively. When would a human really be convinced enough that EUBig(X) precisely equals EUBig(Y), so to start optimizing EUSmall? It seems like the difference between VNM and Hausner utility only happens in a measure-0 class of scenarios that humans couldn’t practically detect anyway. ETA: except maybe when there’s a time limit...
This is actually one reason I posted on Hausner utility: if you like it, then note it’s 0% likely to give you different answers from VNM utility, and then just use VNM because you’re not precise enough to know the difference :)
Is there any use for introducing the concept of “noise terms” in lexicographical expected value?
Here’s an illustration of what I mean by “noise terms”:
Suppose that you have a warm cup of coffee sitting in a room. How long will it take before the coffee is the same temperature as the room? Well, the rate of heat transfer between two objects is proportional to the temperature difference between them. That means that the temperature difference between the coffee and the room undergoes exponential decay, and exponential decay never actually reaches its final value. It’ll take infinitely long for the coffee to reach the same temperature as the room!
Except that the real world doesn’t quite work that way. If you try to measure temperature precisely enough, you’ll start finding that your thermometer readings are fluctuating. All those “random” molecular motions create noise. You can’t measure the difference between 100 degrees and 100 + epsilon degrees as long as epsilon is less than the noise level of the system. So the coffee really does reach room temperature, because the difference becomes so small that it disappears into the noise.
If the parameters that I use to calculate expected value are noisy, then as long as the difference between EUBig(X) and EUBig(Y) is small enough—below the noise level of the system—I can’t tell if it’s positive or negative, so I can’t know if I should prefer X to Y. As with the coffee in the room, the difference vanishes into the noise. So I’ll resort to the tiebreaker, and optimize EUSmall instead.
Does this make any sense?
I just added a paragraph about this. Good stuff!
This certainly has an intuitive appeal. Where would the noise be? In the outcome values, or the probabilities, or both?
To have an effect, the noise would have to something that you somehow know can’t be eliminated by allocating more thinking/computing time (perhaps you’re on a time limit) or resources, else the rational thing to do would be to just “’think harder” to make sure you’re not sacrificing your EUBig...
Is that knowledge plausible?
Okay, I just added a paragraph about this. It seems to me that time limits are the biggest noise factor for humans.
Good stuff!
It’s not really an issue of impracticality. It’s just that, any time you have a higher-level class of utility, the lower-level classes of utility stop being relevant to your decisions. No matter how precise the algorithm is. That’s why I say it’s extra complexity with no optimization benefit. Since the extra structure doesn’t even map better to my intuition about preference, I just Occam-shave it away.
Wait… certainly, if you lexicographically value (brightness, redness) of a light, and somehow manage to be in a scenario where you can’t make the light brighter, and somehow manage to know that, then the redness value becomes relevant.
What I mean is that the environment itself makes such precise situations rare (a non-practical issue), and an imprecise algorithm makes it hard to detect when, if ever, they occur (a practical issue).