So in my case I would consider elite common sense about cryptography to be “Ask Bruce Schneier”, who might or might not have declined to talk to those companies or consult with them. That’s much narrower than trying to poll an upper crust of Ivy League graduates, from whom I would not expect a particularly good answer. If Bruce Schneier didn’t answer I would email Dad and ask him for the name of a trusted cryptographer who was friends with the Yudkowsky family, and separately I would email Jolly and ask him what he thought or who to talk to.
But then if Scott Aaronson, who isn’t a cryptographer, blogged about the issue saying the cryptographers were being silly and even he could see that, I would either mark it as unknown or use my own judgment to try and figure out who to trust. If I couldn’t follow the object-level arguments and there was no blatantly obvious meta-level difference, I’d mark it unresolvable-for-now (and plan as if both alternatives had substantial probability). If I could follow the object-level arguments and there was a substantial difference of strength which I perceived, I wouldn’t hesitate to pick sides based on it, regardless of the eliteness of the people who’d taken the opposite side, so long as there were some elites on my own side who seemed to think that yes, it was that obvious. I’ve been in that epistemic position lots of times.
I’m honestly not sure about what your version is. I certainly don’t get the impression that one can grind well-specified rules to get to the answer about polling the upper 10% of Ivy League graduates in this case. If anything I think your rules would endorse my ‘Bruce Schneier’ output more strongly than the 10%, at least as I briefly read them.
I think we don’t disagree about whether elite common sense should defer to cryptography experts (I assume this is what Bruce Schneier is a stand-in for). Simplifying a bit, we are disagreeing about the much more subtle question of whether, given that elite common sense should defer to cryptography experts, in a situation where the current views of cryptographers are unknown, elite common sense recommends adopting the current views of cryptographers. I say elite common sense recommends adopting their views if you know them, but going with what e.g. the upper crust of Ivy League graduates would say if they had access to your information if you don’t know about the opinions of cryptographers. I also suspect elite common sense recommends finding out about the opinions of elite cryptographers if you can. But Wei Dai’s example was one in which you didn’t know and maybe couldn’t find out, so that’s why I said what I said. Frankly, I’m pretty flummoxed about why you think this is the “No True Scotsman” fallacy. I feel that one of us is probably misunderstanding the other on a basic level.
A possible confusion here is that I doubt the cryptographers have very different epistemic standards as opposed to substantive knowledge and experience about cryptography and tools for thinking about it.
I certainly don’t get the impression that one can grind well-specified rules to get to the answer about polling the upper 10% of Ivy League graduates in this case.
I agree with this, and tried to make this clear in my discussion. I went with a rough guess that would work for a decent chunk of the audience rather than only saying something very abstract. It’s subtle, but I think reasonable epistemic frameworks are subtle if you want them to have much generality.
So in my case I would consider elite common sense about cryptography to be “Ask Bruce Schneier”, who might or might not have declined to talk to those companies or consult with them. That’s much narrower than trying to poll an upper crust of Ivy League graduates, from whom I would not expect a particularly good answer. If Bruce Schneier didn’t answer I would email Dad and ask him for the name of a trusted cryptographer who was friends with the Yudkowsky family, and separately I would email Jolly and ask him what he thought or who to talk to.
But then if Scott Aaronson, who isn’t a cryptographer, blogged about the issue saying the cryptographers were being silly and even he could see that, I would either mark it as unknown or use my own judgment to try and figure out who to trust. If I couldn’t follow the object-level arguments and there was no blatantly obvious meta-level difference, I’d mark it unresolvable-for-now (and plan as if both alternatives had substantial probability). If I could follow the object-level arguments and there was a substantial difference of strength which I perceived, I wouldn’t hesitate to pick sides based on it, regardless of the eliteness of the people who’d taken the opposite side, so long as there were some elites on my own side who seemed to think that yes, it was that obvious. I’ve been in that epistemic position lots of times.
I’m honestly not sure about what your version is. I certainly don’t get the impression that one can grind well-specified rules to get to the answer about polling the upper 10% of Ivy League graduates in this case. If anything I think your rules would endorse my ‘Bruce Schneier’ output more strongly than the 10%, at least as I briefly read them.
I think we don’t disagree about whether elite common sense should defer to cryptography experts (I assume this is what Bruce Schneier is a stand-in for). Simplifying a bit, we are disagreeing about the much more subtle question of whether, given that elite common sense should defer to cryptography experts, in a situation where the current views of cryptographers are unknown, elite common sense recommends adopting the current views of cryptographers. I say elite common sense recommends adopting their views if you know them, but going with what e.g. the upper crust of Ivy League graduates would say if they had access to your information if you don’t know about the opinions of cryptographers. I also suspect elite common sense recommends finding out about the opinions of elite cryptographers if you can. But Wei Dai’s example was one in which you didn’t know and maybe couldn’t find out, so that’s why I said what I said. Frankly, I’m pretty flummoxed about why you think this is the “No True Scotsman” fallacy. I feel that one of us is probably misunderstanding the other on a basic level.
A possible confusion here is that I doubt the cryptographers have very different epistemic standards as opposed to substantive knowledge and experience about cryptography and tools for thinking about it.
I agree with this, and tried to make this clear in my discussion. I went with a rough guess that would work for a decent chunk of the audience rather than only saying something very abstract. It’s subtle, but I think reasonable epistemic frameworks are subtle if you want them to have much generality.