Where people have vague mental models that can be used to argue anything, they usually end up believing whatever they started out wanting to believe.
“Humility” is a virtue that is often misunderstood. This doesn’t mean we should discard the concept of humility, but we should be careful using it.
It seems to me to be the case that when confronting rationalists, those who have a belief they’re motivated to continue to hold will attempt to manipulate rationalists into withdrawing skepticism or risk social disapproval. For example, when creationists ask something like “how can you be sure you’re absolutely right about evolution?”, I believe the actual intention is not to induce humility on the part of the evolutionist, but to appeal and warning for the evolutionist not to risk the creationist’s disapproval.
So, it’s crucial to identify the difference between when someone else wants you to be humble, and when someone wants you to be socially modest so you don’t frustrate them by challenging their beliefs.
There’s better discussion than what I can produce on when humility is and isn’t useful in the comments of the SEQ RERUN of this post
NOTE: edited for simplicity and grammar.
The following was comment from discussion of the original post, and I thought it was great. Does anyone know how I might go about designing and practicing such calibration tests well?