It might be useful to think of this as an empirical claim about diamonds.
I think this statement encapsulates some worries I have.
If it’s important how the human defines a property like “the same diamond,” then assuming that the sameness of the diamond is “out there in the diamond” will get you into trouble—e.g. if there’s any optimization pressure to find cases where the specifics of the human’s model rear their head. Human judgment is laden with the details of how humans model the world, you can’t avoid dependence on the human (and the messiness that entails) entirely.
Or to phrase it another way: I don’t have any beef with a narrow approach that says “there’s some set of judgments for which the human is basically competent, and we want to elicit knowledge relevant to those judgments.” But I’m worried about a narrow approach that says “let’s assume that humans are basically competent for all judgments of interest, and keep assuming this until something goes wrong.”
It just feels to me like this second approach is sort of… treating the real world as if it’s a perturbative approximation to the platonic realm.
I think this statement encapsulates some worries I have.
If it’s important how the human defines a property like “the same diamond,” then assuming that the sameness of the diamond is “out there in the diamond” will get you into trouble—e.g. if there’s any optimization pressure to find cases where the specifics of the human’s model rear their head. Human judgment is laden with the details of how humans model the world, you can’t avoid dependence on the human (and the messiness that entails) entirely.
Or to phrase it another way: I don’t have any beef with a narrow approach that says “there’s some set of judgments for which the human is basically competent, and we want to elicit knowledge relevant to those judgments.” But I’m worried about a narrow approach that says “let’s assume that humans are basically competent for all judgments of interest, and keep assuming this until something goes wrong.”
It just feels to me like this second approach is sort of… treating the real world as if it’s a perturbative approximation to the platonic realm.