I think the key issue with “I don’t know” (as with many other things) is that it should be a starting point rather than a stop sign. If you’re using it as a starting point, the phrase indicates that you don’t have enough relevant information to move forward but you would like to find out more. In everyday conversation, that means that you (or the person who asked you the question) are still in the information-gathering phase of the process, e.g. “I don’t know; let me google it” or “I don’t know; Rebecca might remember—have you asked her?” For rationalist-level problems, it means recognizing your ignorance (just as you should notice your confusion) rather than glossing over it or making up some plausible-sounding story that makes it seem like you understand what’s going on. “I don’t know” means that there’s work to be done in figuring it out better, and it’s worth taking a moment to see that you don’t have a clear answer yet before you start guessing.
The problem is when you take “I don’t know” to mean “it’s not worth thinking about”, or you turn it into a treasured mystery, or you conclude that you’re allowed to believe whatever you want (no need to worry about what’s true) because no one actually knows what’s true. You don’t know, and you’re fine with keeping it that way.
Eliezer actually discusses this briefly in his 12 virtues essay, where he says (in the first paragraph of the essay) that curiosity requires both recognizing your ignorance and seeking to relinquish it.
I think the key issue with “I don’t know” (as with many other things) is that it should be a starting point rather than a stop sign. If you’re using it as a starting point, the phrase indicates that you don’t have enough relevant information to move forward but you would like to find out more. In everyday conversation, that means that you (or the person who asked you the question) are still in the information-gathering phase of the process, e.g. “I don’t know; let me google it” or “I don’t know; Rebecca might remember—have you asked her?” For rationalist-level problems, it means recognizing your ignorance (just as you should notice your confusion) rather than glossing over it or making up some plausible-sounding story that makes it seem like you understand what’s going on. “I don’t know” means that there’s work to be done in figuring it out better, and it’s worth taking a moment to see that you don’t have a clear answer yet before you start guessing.
The problem is when you take “I don’t know” to mean “it’s not worth thinking about”, or you turn it into a treasured mystery, or you conclude that you’re allowed to believe whatever you want (no need to worry about what’s true) because no one actually knows what’s true. You don’t know, and you’re fine with keeping it that way.
Eliezer actually discusses this briefly in his 12 virtues essay, where he says (in the first paragraph of the essay) that curiosity requires both recognizing your ignorance and seeking to relinquish it.