The alignment problem still has to get solved somehow in those branches, which almost all merely have slightly different versions of us doing mostly the same sorts of things.
What might be different in these branches is that world-ending AGIs have anomalously bad luck in getting started. But the vast majority of anthropic weight, even after selecting for winning branches, will be on branches that are pretty ordinary, and where the alignment problem still had to get solved the hard way, by people who were basically just luckier versions of us.
So even if we decide to stake our hope on those possibilities, it’s pretty much the same as staking hope on luckier versions of ourselves who still did the hard work. It doesn’t really change anything for us here and now; we still need to do the same sorts of things. It all adds up to normality.
If anthropic stuff actually works out like this, then this is great news for values over experiences, which will still be about as satisfiable as they were before, despite our impending doom. But values over world-states will not be at all consoled.
I suspect human values are a complicated mix of the two, with things like male-libido being far on the experience end (since each additional experience of sexual pleasure would correspond in the ancestral environment to a roughly linear increase in reproductive fitness), and things like maternal-love being far on the world-state end (since it needs to actually track the well-being of the children, even in cases where no further experiences are expected), and most things lying somewhere in the middle.
The alignment problem still has to get solved somehow in those branches, which almost all merely have slightly different versions of us doing mostly the same sorts of things.
What might be different in these branches is that world-ending AGIs have anomalously bad luck in getting started. But the vast majority of anthropic weight, even after selecting for winning branches, will be on branches that are pretty ordinary, and where the alignment problem still had to get solved the hard way, by people who were basically just luckier versions of us.
So even if we decide to stake our hope on those possibilities, it’s pretty much the same as staking hope on luckier versions of ourselves who still did the hard work. It doesn’t really change anything for us here and now; we still need to do the same sorts of things. It all adds up to normality.
Another consideration I thought of:
If anthropic stuff actually works out like this, then this is great news for values over experiences, which will still be about as satisfiable as they were before, despite our impending doom. But values over world-states will not be at all consoled.
I suspect human values are a complicated mix of the two, with things like male-libido being far on the experience end (since each additional experience of sexual pleasure would correspond in the ancestral environment to a roughly linear increase in reproductive fitness), and things like maternal-love being far on the world-state end (since it needs to actually track the well-being of the children, even in cases where no further experiences are expected), and most things lying somewhere in the middle.