I like the idea that having some parts of you protected from yourself makes them indirectly protected from people or memes who have power over you (and want to optimize you for their benefit, not yours). Being irrational is better than being transparently rational when someone is holding a gun at your head. If you could do something, you would be forced to do it (against your interests), so it’s better for you if you can’t.
But, what now? It seems like rationality and introspection is a bit like defusing a bomb—great if you can do it perfectly, but it kills you when you do it halfways.
It reminds me of a fantasy book which had a system of magic where wizards could achieve 4 levels of power. Being known as a 3rd level wizard was a very bad thing, because all 4th level wizards were trying to magically enslave you—to get rid of a potential competitor, and to get a powerful slave (I suppose the magical cost of enslaving someone didn’t grow up proportionally to victim’s level).
To use an analogy, being biologically incapable of reaching 3rd level of magic might be an evolutionary advantage. But at the same time, it would prevent you from reaching the 4th level, ever.
The only difference between their presentation and mine is that I’m saying that for 99% of people, 99% of the time, taking ideas seriously is the wrong strategy
I kinda think this is true, and it’s not clear to me from the outset whether you should “go down the path” of getting access to level 3 magic given the negatives.
Probably good heuristics are proceeding with caution when encountering new/out there ideas, remembering you always have the right to say no, finding trustworthy guides, etc.
Related: Reason as memetic immune disorder
I like the idea that having some parts of you protected from yourself makes them indirectly protected from people or memes who have power over you (and want to optimize you for their benefit, not yours). Being irrational is better than being transparently rational when someone is holding a gun at your head. If you could do something, you would be forced to do it (against your interests), so it’s better for you if you can’t.
But, what now? It seems like rationality and introspection is a bit like defusing a bomb—great if you can do it perfectly, but it kills you when you do it halfways.
It reminds me of a fantasy book which had a system of magic where wizards could achieve 4 levels of power. Being known as a 3rd level wizard was a very bad thing, because all 4th level wizards were trying to magically enslave you—to get rid of a potential competitor, and to get a powerful slave (I suppose the magical cost of enslaving someone didn’t grow up proportionally to victim’s level).
To use an analogy, being biologically incapable of reaching 3rd level of magic might be an evolutionary advantage. But at the same time, it would prevent you from reaching the 4th level, ever.
Thanks for including that link—seems right, and reminded me of Scott’s old post Epistemic Learned Helplessness
I kinda think this is true, and it’s not clear to me from the outset whether you should “go down the path” of getting access to level 3 magic given the negatives.
Probably good heuristics are proceeding with caution when encountering new/out there ideas, remembering you always have the right to say no, finding trustworthy guides, etc.