“You will die. No matter what actions you’ll take all the possible branches end with your death. Still, you try to pick optimal path, because that’s what your brain’s architecture know how to do: pick optimal branch. You try to salvage this approach by proposing more and more complicated goal functions: instead of final value, let’s look at the sum over time, or avg, or max, or maybe ascribe other value to death, or try to extend summation beyond it, or whatever. You brain is a hammer, and it needs a nail. But it never occurs to you, that life is not something one needs to optimize. This is not an instance of the problem your brain is built to solve, and it looks silly to me you try to fit it by force to your preferred format. This is your inductive bias, so strong you probably don’t get what I’m trying to say to you: yes, you’ll die, but this doesn’t count.”
(I’m surprised nobody wrote it for 12 years, or at least my eyes can’t see it)
“You will die. No matter what actions you’ll take all the possible branches end with your death. Still, you try to pick optimal path, because that’s what your brain’s architecture know how to do: pick optimal branch. You try to salvage this approach by proposing more and more complicated goal functions: instead of final value, let’s look at the sum over time, or avg, or max, or maybe ascribe other value to death, or try to extend summation beyond it, or whatever. You brain is a hammer, and it needs a nail. But it never occurs to you, that life is not something one needs to optimize. This is not an instance of the problem your brain is built to solve, and it looks silly to me you try to fit it by force to your preferred format. This is your inductive bias, so strong you probably don’t get what I’m trying to say to you: yes, you’ll die, but this doesn’t count.”
(I’m surprised nobody wrote it for 12 years, or at least my eyes can’t see it)