I feel like this post provides arguments similar to those I would have given if I was mentally more organized. For months I’ve been asserting (without argument), ’don’t you see? -- without “absolute values” to steer us, optimizing over preferences is incoherent”. The incoherence stems from the fact that our preferences are mutable, and something we modify and optimize over a lifetime, and making distinctions between preferences given to us by genetics, our environmental history, or random chance is too arbitrary. There’s no reason to elevate one source of preferences as somehow more special or more uniquely “myself”.
I think that, despite my lack of mental organization on this topic, I experience this incoherence more immediately because my preferences for existence, consciousness and personal identity are not that strong. For example, I don’t care about existential risk because I don’t care about humankind existing in the abstract. If I must exist, then I would like my existence to have certain qualities. I don’t see things being “my way” in the long run, so I prefer a short life, optimizing the present as best I can. I have some hope that things will nevertheless work out, but this hope is more of a ‘watch and see’ than something that drives action.
In the long run, as we understand more and more what human preferences are, I think there will be just a few coherent options: the choice of non-existence, the choice to wire-head or the choice to repress (what I consider to be) the salient aspects of consciousness and self-awareness.
Overt self-awareness may have been an evolutionary mistake that needs to be corrected. Perhaps we encounter no intelligent life because there is such a small temporal gap between the development of intelligence, the meta-awareness of goals, and the desire to relinquish awareness.
What causes me to pause in this analysis is that persons here, clearly more intelligent than myself, do not have a similar impulse to lie down and quit. I infer that it is a difference in preferences, and I represent some subset of human preferences that are not meant to persist … because my preferences dissolve upon reflection.
I feel like these ideas are all connected and follow from the thesis of the post . Do they seem peripheral?
In the long run, as we understand more and more what human preferences are, I think there will be just a few coherent options: the choice of non-existence, the choice to wire-head or the choice to repress (what I consider to be) the salient aspects of consciousness and self-awareness.
In my imagination Less Wrong becomes really influential and spurs a powerful global movement, develops factions along these fault lines (with a fourth faction, clinging desperately to their moral nostalgia) and then self-destructs in a flame war to end all flame wars.
Maybe I’ll write a story.
You can pry my self-awareness from my cold, dead neurons.
It wouldn’t surprise me if strong preferences for existence, consciousness, and personal identity are partly physiologically based. And I mean fairly simple physiology, like neurotransmitter balance.
This doesn’t mean they should be changed.
It does occur to me that I’ve been trying to upgrade my gusto level by a combination of willpower and beating up on myself, and this has made things a lot worse.
I don’t think we’re talking about the same type of incoherence; but I wouldn’t want to have been deprived of these thoughts of yours because of that. Even though they’re the most depressing thing I’ve heard today.
I find that careful introspection always dissolves the conceptual frames within which my preferences are formulated but generally leaves the actionable (but not the non-actionable) preferences intact.
I don’t follow. Can you give examples? What’s a conceptual frame, and what’s an actionable vs. non-actionable preference? I infer the actionable/non-actionable distinction is related to the keep/don’t keep decision, but the terminology sounds to me like it just means “a preference you can satsify” vs. “a preference you can’t act to satisfy”.
Could you give an example of an actionable preference that stays intact? Preferably one that is not evolutionary, because I agree that those are mostly indissoluble.
I feel like this post provides arguments similar to those I would have given if I was mentally more organized. For months I’ve been asserting (without argument), ’don’t you see? -- without “absolute values” to steer us, optimizing over preferences is incoherent”. The incoherence stems from the fact that our preferences are mutable, and something we modify and optimize over a lifetime, and making distinctions between preferences given to us by genetics, our environmental history, or random chance is too arbitrary. There’s no reason to elevate one source of preferences as somehow more special or more uniquely “myself”.
I think that, despite my lack of mental organization on this topic, I experience this incoherence more immediately because my preferences for existence, consciousness and personal identity are not that strong. For example, I don’t care about existential risk because I don’t care about humankind existing in the abstract. If I must exist, then I would like my existence to have certain qualities. I don’t see things being “my way” in the long run, so I prefer a short life, optimizing the present as best I can. I have some hope that things will nevertheless work out, but this hope is more of a ‘watch and see’ than something that drives action.
In the long run, as we understand more and more what human preferences are, I think there will be just a few coherent options: the choice of non-existence, the choice to wire-head or the choice to repress (what I consider to be) the salient aspects of consciousness and self-awareness.
Overt self-awareness may have been an evolutionary mistake that needs to be corrected. Perhaps we encounter no intelligent life because there is such a small temporal gap between the development of intelligence, the meta-awareness of goals, and the desire to relinquish awareness.
What causes me to pause in this analysis is that persons here, clearly more intelligent than myself, do not have a similar impulse to lie down and quit. I infer that it is a difference in preferences, and I represent some subset of human preferences that are not meant to persist … because my preferences dissolve upon reflection.
I feel like these ideas are all connected and follow from the thesis of the post . Do they seem peripheral?
In my imagination Less Wrong becomes really influential and spurs a powerful global movement, develops factions along these fault lines (with a fourth faction, clinging desperately to their moral nostalgia) and then self-destructs in a flame war to end all flame wars.
Maybe I’ll write a story.
You can pry my self-awareness from my cold, dead neurons.
You can pry my self-awareness from my cold, dead neurons.
Yup, that’s pretty much the plan.
It wouldn’t surprise me if strong preferences for existence, consciousness, and personal identity are partly physiologically based. And I mean fairly simple physiology, like neurotransmitter balance.
This doesn’t mean they should be changed.
It does occur to me that I’ve been trying to upgrade my gusto level by a combination of willpower and beating up on myself, and this has made things a lot worse.
Did pjeby write a post against willpower? I think willpower is overrated. Cognitive behavioral therapy is better.
I don’t think we’re talking about the same type of incoherence; but I wouldn’t want to have been deprived of these thoughts of yours because of that. Even though they’re the most depressing thing I’ve heard today.
I find that careful introspection always dissolves the conceptual frames within which my preferences are formulated but generally leaves the actionable (but not the non-actionable) preferences intact.
I don’t follow. Can you give examples? What’s a conceptual frame, and what’s an actionable vs. non-actionable preference? I infer the actionable/non-actionable distinction is related to the keep/don’t keep decision, but the terminology sounds to me like it just means “a preference you can satsify” vs. “a preference you can’t act to satisfy”.
And, also, could you give an example of a conceptual frame which got dissolved?
Free will vs. determinism, deontology vs. utilitarianism.
Could you give an example of an actionable preference that stays intact? Preferably one that is not evolutionary, because I agree that those are mostly indissoluble.