Thanks! FWIW, I don’t strongly expect that we’ll have to abandon VNM at this point, but I do still have a lot of uncertainty in the area.
I will be much more willing to discard objection 1 after resolving my confusion surrounding Pascal muggings, and I’ll be much more willing to discard objection 2 once I have a formalization of “corrigibility.” The fact that I still have some related confusions, combined with the fact that (in my experience) mathematical insights often reveal hidden assumptions in things I thought were watertight, combined with a dash of outside view, leads me to decently high credence (~15%, with high anticipated variance) that VNM won’t cut it.
Thanks! FWIW, I don’t strongly expect that we’ll have to abandon VNM at this point, but I do still have a lot of uncertainty in the area.
I will be much more willing to discard objection 1 after resolving my confusion surrounding Pascal muggings, and I’ll be much more willing to discard objection 2 once I have a formalization of “corrigibility.” The fact that I still have some related confusions, combined with the fact that (in my experience) mathematical insights often reveal hidden assumptions in things I thought were watertight, combined with a dash of outside view, leads me to decently high credence (~15%, with high anticipated variance) that VNM won’t cut it.
What’s the definition of preferences used in boundedly-rational agent research?