Re “they should also pop out in a sane person’s mind within like 15 seconds of actually hearing that statement” I agree with that in the abstract; few people will say that a state of high physiological alertness/vigilance is Actually A Good Idea to cultivate for threats/risks not usefully countered by the effects of high physiological alertness.
Being able to reason about that in the abstract doesn’t necessarily transfer to actually stopping doing that. Like personally, I feel like being told something along the line of “you’re working yourself up into a counterproductive state of high physiological alertness about the risks of [risk] and counterproductively countering that with incredibly abstract thought disconnected from useful action” is not something I am very good at hearing from most people when I am in that sort of extraordinarily afraid state. It can really feel like someone wants to manipulate me into thinking that [risk] is not a big deal, or discourage me from doing anything about [risk], or that they’re seeking to make me more vulnerable to [risk]. These days this is rarely the case; but the heuristic still sticks around. Maybe I should find its commanding officer so it can be told by someone it trusts that it’s okay to stand down...
With the military analogy; it’s like you’d been asked to keep an eye out for a potential threat, and your commanding officer tells you on the radio to get on REDCON 1. Later on you hear an unfamiliar voice on the radio which doesn’t authenticate itself, and it keeps telling you that your heightened alertness is actually counterproductive and that you should stand down.
Would you stand down? No, you’d be incredibly suspicious! Interfering with the enemy’s communication is carte blanche in war. Are there situations where you would indeed obey the order from the unfamiliar voice? Perhaps! Maybe your commanding officer’s vehicle got destroyed, or more prosaically, maybe his radio died. But it would have to be in a situation where you’re confident it represents legitimate military authority. It would be a high bar to clear, since if you do stand down and it was an enemy ruse, you’re in a very bad situation regardless if you get captured by the enemy or if you get court-martialed for disobeying orders. If it seems like standing down makes zero tactical/strategic sense, your threshold would be even higher! In the extreme, nothing short of your commanding officer showing up in person would be enough.
All of this is totally consistent with the quoted section in OP that mentions “Goals and motivational weightings change”, “Information-gathering programs are redirected”, “Conceptual frames shift”, etc. The high physiological alertness program has to be a bit sticky, otherwise a predator stalking you could turn it off by sitting down and you’d be like “oh, I guess I’m not in danger anymore”. If you’ve been successfully tricked by a predator into thinking that it broke off the hunt when it really was finding a better position to attack you from, the program’s gonna be a bit stickier, since its job is to keep you from becoming food.
To get away from the analogies, I really appreciate this piece and how it was written. I specifically appreciate it because it doesn’t feel like it is an attempt to make me more vulnerable to something bad. Also I think it might have helped me get a bit of a felt sense shift.
To get away from the analogies, I really appreciate this piece and how it was written. I specifically appreciate it because it doesn’t feel like it is an attempt to make me more vulnerable to something bad. Also I think it might have helped me get a bit of a felt sense shift.
Thank you for sharing that, I’m happy to hear it. :)
I want to mention here that the war example is an example of where there is an adversarial scenario, or adversarial game, and applying an adversarial frame is usually not the correct decision to do, and importantly given that the most perverse scenarios usually can’t be dealt with without exotic physics due to computational complexity reason, you usually shouldn’t focus on adversarial scenarios, and here Kaj Sotala is very, very correct on this post.
Re “they should also pop out in a sane person’s mind within like 15 seconds of actually hearing that statement” I agree with that in the abstract; few people will say that a state of high physiological alertness/vigilance is Actually A Good Idea to cultivate for threats/risks not usefully countered by the effects of high physiological alertness.
Being able to reason about that in the abstract doesn’t necessarily transfer to actually stopping doing that. Like personally, I feel like being told something along the line of “you’re working yourself up into a counterproductive state of high physiological alertness about the risks of [risk] and counterproductively countering that with incredibly abstract thought disconnected from useful action” is not something I am very good at hearing from most people when I am in that sort of extraordinarily afraid state. It can really feel like someone wants to manipulate me into thinking that [risk] is not a big deal, or discourage me from doing anything about [risk], or that they’re seeking to make me more vulnerable to [risk]. These days this is rarely the case; but the heuristic still sticks around. Maybe I should find its commanding officer so it can be told by someone it trusts that it’s okay to stand down...
With the military analogy; it’s like you’d been asked to keep an eye out for a potential threat, and your commanding officer tells you on the radio to get on REDCON 1. Later on you hear an unfamiliar voice on the radio which doesn’t authenticate itself, and it keeps telling you that your heightened alertness is actually counterproductive and that you should stand down.
Would you stand down? No, you’d be incredibly suspicious! Interfering with the enemy’s communication is carte blanche in war. Are there situations where you would indeed obey the order from the unfamiliar voice? Perhaps! Maybe your commanding officer’s vehicle got destroyed, or more prosaically, maybe his radio died. But it would have to be in a situation where you’re confident it represents legitimate military authority. It would be a high bar to clear, since if you do stand down and it was an enemy ruse, you’re in a very bad situation regardless if you get captured by the enemy or if you get court-martialed for disobeying orders. If it seems like standing down makes zero tactical/strategic sense, your threshold would be even higher! In the extreme, nothing short of your commanding officer showing up in person would be enough.
All of this is totally consistent with the quoted section in OP that mentions “Goals and motivational weightings change”, “Information-gathering programs are redirected”, “Conceptual frames shift”, etc. The high physiological alertness program has to be a bit sticky, otherwise a predator stalking you could turn it off by sitting down and you’d be like “oh, I guess I’m not in danger anymore”. If you’ve been successfully tricked by a predator into thinking that it broke off the hunt when it really was finding a better position to attack you from, the program’s gonna be a bit stickier, since its job is to keep you from becoming food.
To get away from the analogies, I really appreciate this piece and how it was written. I specifically appreciate it because it doesn’t feel like it is an attempt to make me more vulnerable to something bad. Also I think it might have helped me get a bit of a felt sense shift.
Thank you for sharing that, I’m happy to hear it. :)
I want to mention here that the war example is an example of where there is an adversarial scenario, or adversarial game, and applying an adversarial frame is usually not the correct decision to do, and importantly given that the most perverse scenarios usually can’t be dealt with without exotic physics due to computational complexity reason, you usually shouldn’t focus on adversarial scenarios, and here Kaj Sotala is very, very correct on this post.