Thanks! I think that this is a very useful example of an advance prediction of utility theory; and that gathering more examples like this is one of the most promising way to make progress on bridging the gap between Eliezer’s and most other people’s understandings of consequentialism.
Potentially important thing to flag here: at least in my mind, expected utility theory (i.e. the property Eliezer was calling “laser-like” or “coherence”) and consequentialism are two distinct things. Consequentialism will tend to produce systems with (approximate) coherent expected utilities, and that is one major way I expect coherent utilities to show up in practice. But coherent utilities can in-principle occur even without consequentialism (e.g. conservative vector fields in physics), and consequentialism can in-principle not be very coherent (e.g. if it just has tons of resources and doesn’t have to be very efficient to achieve a goal-state).
(I’m not sure whether Eliezer would agree with this. The thing-I-think-Eliezer-means-by-consequentialism does not yet have a good mathematical formulation which I know of, which makes it harder to check that two people even mean the same thing when pointing to the concept.)
My model of Eliezer says that there is some deep underlying concept of consequentialism, of which the “not very coherent consequentialism” is a distorted reflection; and that this deep underlying concept is very closely related to expected utility theory. (I believe he said at one point that he started using the word “consequentialism” instead of “expected utility maximisation” mainly because people kept misunderstanding what he meant by the latter.)
I don’t know enough about conservative vector fields to comment, but on priors I’m pretty skeptical of this being a good example of coherent utilities; I also don’t have a good guess about what Eliezer would say here.
I don’t know enough about conservative vector fields to comment, but on priors I’m pretty skeptical of this being a good example of coherent utilities; I also don’t have a good guess about what Eliezer would say here.
I think johnswentworth (and others) are claiming that they have the same ‘math’/‘shape’, which seems much more likely (if you trust their claims about such things generally).
Thanks! I think that this is a very useful example of an advance prediction of utility theory; and that gathering more examples like this is one of the most promising way to make progress on bridging the gap between Eliezer’s and most other people’s understandings of consequentialism.
Potentially important thing to flag here: at least in my mind, expected utility theory (i.e. the property Eliezer was calling “laser-like” or “coherence”) and consequentialism are two distinct things. Consequentialism will tend to produce systems with (approximate) coherent expected utilities, and that is one major way I expect coherent utilities to show up in practice. But coherent utilities can in-principle occur even without consequentialism (e.g. conservative vector fields in physics), and consequentialism can in-principle not be very coherent (e.g. if it just has tons of resources and doesn’t have to be very efficient to achieve a goal-state).
(I’m not sure whether Eliezer would agree with this. The thing-I-think-Eliezer-means-by-consequentialism does not yet have a good mathematical formulation which I know of, which makes it harder to check that two people even mean the same thing when pointing to the concept.)
My model of Eliezer says that there is some deep underlying concept of consequentialism, of which the “not very coherent consequentialism” is a distorted reflection; and that this deep underlying concept is very closely related to expected utility theory. (I believe he said at one point that he started using the word “consequentialism” instead of “expected utility maximisation” mainly because people kept misunderstanding what he meant by the latter.)
I don’t know enough about conservative vector fields to comment, but on priors I’m pretty skeptical of this being a good example of coherent utilities; I also don’t have a good guess about what Eliezer would say here.
I think johnswentworth (and others) are claiming that they have the same ‘math’/‘shape’, which seems much more likely (if you trust their claims about such things generally).