Either there are some concrete things corresponding to personal identity/preference satisfaction/etc., or we value not these things but some actually-existing correlates of these things, or acting like we value these things is a heuristic that instrumentally helps us arrive at good outcomes. Either way, ontology shifts don’t do anything bad to our values.
Leading to this claim, which is very alluring to me, even if I disagree with it:
Or, the other way around, perhaps “values” are defined by being robust to ontology shifts.
You gave the example that us learning that our world is physical didn’t make us value humans less, even if we no longer believe that they have immortal souls. That’s true, we still value humans. But the problem is that the first group of agents (medieval “we”) are a different group than the second group (post-industrial “we”). Each group has a different concept of “human”. The two concepts roughly map to the same targets in the territory, but the semantic meaning is different in a way that is crucial to the first group of agents. The first group weren’t convinced that they were wrong. They simply got replaced by a new group who inherited their title.
That might sound dramatic, but ask a truly committed religious person if they can imagine themselves not believing in God. They will not be able to imagine it. This is because, in fact, they would no longer be the same agency in the sense that is crucial to them. Society would legally consider them the same person, and their apostate future-self might claim the same title as them and even deny that the change really mattered, but the original entity who was asked, the theistic agent, would no longer recognize any heir as legitimate at that point. From its point of view, it has simply died. That is why the theist cannot even imagine becoming an apostate, even if they grant that there’s a greater than zero chance of becoming an apostate at all times.
It’s true what you say, if you’re talking about people i.e. biological human organisms. Such entities will always be doing something in the world up until the very moment they physically expire, including having their whole world view shattered and living in the aftermath of that. However, pre-world-view-shattering them would not recognize post-world-view-shattering them as a legitimate heir. The person might be the same, but it’s a different agency in control.
Similar things can be said about populations and governments.
I think I get what you mean. Human minds’ “breakdown protocol” in case the universe turns out to be empty of value isn’t to just shut down; the meat keeps functioning, so what happens is the gradual re-assembly of a new mind/agent from the pieces of the old one. Does that pass the ITT?
But I remain unconvinced that this is what happens during a crisis of faith. By themselves, the theist’s refusal to admit their future apostate self as an heir, and their belief that they’d die if they were to lose faith, don’t mean much to me beyond “the religion memeplex encourages its hosts to develop such beliefs to strengthen its hold on them”. Especially if the apostate later denies that these beliefs were true.
And while my model of a devout mind in the middle of a crisis of faith is indeed dramatic and involves some extreme mental states… I’m unconvinced that this involves changes to terminal values. The entire world-model and the entire suite of instrumental values being rewritten is dramatic enough on its own, and “terminal values” slot nicely into the slot of “what’s guiding this rewriting”/”what’s the main predictor of what the apostate’s instrumental values will be”.
Building off of your other answer here, I think I can imagine at least one situation where your terminal values will get vetoed. Imagine that you discovered, to your horror, that all of your actions up until now have been subconsciously motivated to bring about doomsday. Causing the death of everyone is actually your terminal goal, which you were ignorant of. Furthermore, your subconsciously motivated actions actually have been effective at bringing the world closer and closer to its demise. Your only way to divert this now is to throw yourself immediately out of the window to your death, thereby averting your own terminal goal.
Would you do this?
Dr. Jekyll and Mr. Hyde are the same person, but are they really the same agent?
Suppose I would end up walking out the window. And it would be the wrong action for me to take. I would be foiled by a bunch of bad heuristics and biases I’d internalized over the course of my omnicidal plot. There would be no agent corresponding to me whose values would be satisfied by this.
It would be not unlike, say, manipulating and gaslighting someone until they decide to kill their entire family. This would be against the values the person would claim as their “truer” ones, but in the moment, under the psychological pressure and the influence of some convincing lies, it’d (incorrectly) feel to them like a good idea.
I’d be genuinely interested if you could elaborate on that, including the agency vs. personality distinction you’re making here.
Okay. Let me back up a bit. You had said:
Leading to this claim, which is very alluring to me, even if I disagree with it:
You gave the example that us learning that our world is physical didn’t make us value humans less, even if we no longer believe that they have immortal souls. That’s true, we still value humans. But the problem is that the first group of agents (medieval “we”) are a different group than the second group (post-industrial “we”). Each group has a different concept of “human”. The two concepts roughly map to the same targets in the territory, but the semantic meaning is different in a way that is crucial to the first group of agents. The first group weren’t convinced that they were wrong. They simply got replaced by a new group who inherited their title.
That might sound dramatic, but ask a truly committed religious person if they can imagine themselves not believing in God. They will not be able to imagine it. This is because, in fact, they would no longer be the same agency in the sense that is crucial to them. Society would legally consider them the same person, and their apostate future-self might claim the same title as them and even deny that the change really mattered, but the original entity who was asked, the theistic agent, would no longer recognize any heir as legitimate at that point. From its point of view, it has simply died. That is why the theist cannot even imagine becoming an apostate, even if they grant that there’s a greater than zero chance of becoming an apostate at all times.
It’s true what you say, if you’re talking about people i.e. biological human organisms. Such entities will always be doing something in the world up until the very moment they physically expire, including having their whole world view shattered and living in the aftermath of that. However, pre-world-view-shattering them would not recognize post-world-view-shattering them as a legitimate heir. The person might be the same, but it’s a different agency in control.
Similar things can be said about populations and governments.
Thanks for elaborating!
I think I get what you mean. Human minds’ “breakdown protocol” in case the universe turns out to be empty of value isn’t to just shut down; the meat keeps functioning, so what happens is the gradual re-assembly of a new mind/agent from the pieces of the old one. Does that pass the ITT?
But I remain unconvinced that this is what happens during a crisis of faith. By themselves, the theist’s refusal to admit their future apostate self as an heir, and their belief that they’d die if they were to lose faith, don’t mean much to me beyond “the religion memeplex encourages its hosts to develop such beliefs to strengthen its hold on them”. Especially if the apostate later denies that these beliefs were true.
And while my model of a devout mind in the middle of a crisis of faith is indeed dramatic and involves some extreme mental states… I’m unconvinced that this involves changes to terminal values. The entire world-model and the entire suite of instrumental values being rewritten is dramatic enough on its own, and “terminal values” slot nicely into the slot of “what’s guiding this rewriting”/”what’s the main predictor of what the apostate’s instrumental values will be”.
Building off of your other answer here, I think I can imagine at least one situation where your terminal values will get vetoed. Imagine that you discovered, to your horror, that all of your actions up until now have been subconsciously motivated to bring about doomsday. Causing the death of everyone is actually your terminal goal, which you were ignorant of. Furthermore, your subconsciously motivated actions actually have been effective at bringing the world closer and closer to its demise. Your only way to divert this now is to throw yourself immediately out of the window to your death, thereby averting your own terminal goal.
Would you do this?
Dr. Jekyll and Mr. Hyde are the same person, but are they really the same agent?
Suppose I would end up walking out the window. And it would be the wrong action for me to take. I would be foiled by a bunch of bad heuristics and biases I’d internalized over the course of my omnicidal plot. There would be no agent corresponding to me whose values would be satisfied by this.
It would be not unlike, say, manipulating and gaslighting someone until they decide to kill their entire family. This would be against the values the person would claim as their “truer” ones, but in the moment, under the psychological pressure and the influence of some convincing lies, it’d (incorrectly) feel to them like a good idea.