It was the notion that you could actually in real life follow Science and fail miserably, that Eliezer₁₈ didn’t really, emotionally believe was possible.
Oh, of course he said it was possible. Eliezer₁₈ dutifully acknowledged the possibility of error, saying, “I could be wrong, but...”
But he didn’t think failure could happen in, you know, real life. You were supposed to look for flaws, not actually find them.
...
No one begins to truly search for the Way until their parents have failed them, their gods are dead, and their tools have shattered in their hand.
This sounds like an instance of https://www.lesswrong.com/posts/wustx45CPL5rZenuo/no-safe-defense-not-even-science .
I hadn’t thought of that. Not sure whether it’s the same thing, but thanks for the comment.