I think many EAs/rationalists shouldn’t find this to be worse for humans than life today on the views they apparently endorse, because each human looks better off under standard approaches to intrapersonal aggregation: they get more pleasure, less suffering, more preference satisfaction (or we can imagine some kind of manipulation to achieve this), but at the cost of some important frustrated preferences.
EA/rationalists seems to me the community who gives more conscious attention to the problem of following your officially stated preferences down a cliff, a.k.a. Goodhart.
FWIW, I’m not sure if that’s true relative to the average person, and I’d guess non-consequentialist philosophers are more averse to biting bullets than the average EA and maybe rationalist.
I think many EAs/rationalists shouldn’t find this to be worse for humans than life today on the views they apparently endorse, because each human looks better off under standard approaches to intrapersonal aggregation: they get more pleasure, less suffering, more preference satisfaction (or we can imagine some kind of manipulation to achieve this), but at the cost of some important frustrated preferences.
EA/rationalists seems to me the community who gives more conscious attention to the problem of following your officially stated preferences down a cliff, a.k.a. Goodhart.
FWIW, I’m not sure if that’s true relative to the average person, and I’d guess non-consequentialist philosophers are more averse to biting bullets than the average EA and maybe rationalist.
I suspect that you read “conscious attention to the problem” in a different light than what I mean. To clarify:
Average person won’t go down a cliff on literal instructions, unless Moloch? Yes.
Average person will identify and understand such problem? No.
EA bites a bullet and does something weird? Yes.
EA bites a bullet because YAY BULLETS COOL? No.