I think I get what you mean. Human minds’ “breakdown protocol” in case the universe turns out to be empty of value isn’t to just shut down; the meat keeps functioning, so what happens is the gradual re-assembly of a new mind/agent from the pieces of the old one. Does that pass the ITT?
But I remain unconvinced that this is what happens during a crisis of faith. By themselves, the theist’s refusal to admit their future apostate self as an heir, and their belief that they’d die if they were to lose faith, don’t mean much to me beyond “the religion memeplex encourages its hosts to develop such beliefs to strengthen its hold on them”. Especially if the apostate later denies that these beliefs were true.
And while my model of a devout mind in the middle of a crisis of faith is indeed dramatic and involves some extreme mental states… I’m unconvinced that this involves changes to terminal values. The entire world-model and the entire suite of instrumental values being rewritten is dramatic enough on its own, and “terminal values” slot nicely into the slot of “what’s guiding this rewriting”/”what’s the main predictor of what the apostate’s instrumental values will be”.
Building off of your other answer here, I think I can imagine at least one situation where your terminal values will get vetoed. Imagine that you discovered, to your horror, that all of your actions up until now have been subconsciously motivated to bring about doomsday. Causing the death of everyone is actually your terminal goal, which you were ignorant of. Furthermore, your subconsciously motivated actions actually have been effective at bringing the world closer and closer to its demise. Your only way to divert this now is to throw yourself immediately out of the window to your death, thereby averting your own terminal goal.
Would you do this?
Dr. Jekyll and Mr. Hyde are the same person, but are they really the same agent?
Suppose I would end up walking out the window. And it would be the wrong action for me to take. I would be foiled by a bunch of bad heuristics and biases I’d internalized over the course of my omnicidal plot. There would be no agent corresponding to me whose values would be satisfied by this.
It would be not unlike, say, manipulating and gaslighting someone until they decide to kill their entire family. This would be against the values the person would claim as their “truer” ones, but in the moment, under the psychological pressure and the influence of some convincing lies, it’d (incorrectly) feel to them like a good idea.
Thanks for elaborating!
I think I get what you mean. Human minds’ “breakdown protocol” in case the universe turns out to be empty of value isn’t to just shut down; the meat keeps functioning, so what happens is the gradual re-assembly of a new mind/agent from the pieces of the old one. Does that pass the ITT?
But I remain unconvinced that this is what happens during a crisis of faith. By themselves, the theist’s refusal to admit their future apostate self as an heir, and their belief that they’d die if they were to lose faith, don’t mean much to me beyond “the religion memeplex encourages its hosts to develop such beliefs to strengthen its hold on them”. Especially if the apostate later denies that these beliefs were true.
And while my model of a devout mind in the middle of a crisis of faith is indeed dramatic and involves some extreme mental states… I’m unconvinced that this involves changes to terminal values. The entire world-model and the entire suite of instrumental values being rewritten is dramatic enough on its own, and “terminal values” slot nicely into the slot of “what’s guiding this rewriting”/”what’s the main predictor of what the apostate’s instrumental values will be”.
Building off of your other answer here, I think I can imagine at least one situation where your terminal values will get vetoed. Imagine that you discovered, to your horror, that all of your actions up until now have been subconsciously motivated to bring about doomsday. Causing the death of everyone is actually your terminal goal, which you were ignorant of. Furthermore, your subconsciously motivated actions actually have been effective at bringing the world closer and closer to its demise. Your only way to divert this now is to throw yourself immediately out of the window to your death, thereby averting your own terminal goal.
Would you do this?
Dr. Jekyll and Mr. Hyde are the same person, but are they really the same agent?
Suppose I would end up walking out the window. And it would be the wrong action for me to take. I would be foiled by a bunch of bad heuristics and biases I’d internalized over the course of my omnicidal plot. There would be no agent corresponding to me whose values would be satisfied by this.
It would be not unlike, say, manipulating and gaslighting someone until they decide to kill their entire family. This would be against the values the person would claim as their “truer” ones, but in the moment, under the psychological pressure and the influence of some convincing lies, it’d (incorrectly) feel to them like a good idea.