What exactly is “rationality level,” and how would it be measured?
This is a harder question than “which one of two given behaviors is more rational in a given setting?”, that’s why I suggested starting with the latter. Once you accumulate enough answers like that, you can start assembling them into a more general metric.
Gotta start somewhere. I proposed a step that may or may not lead in the right direction, leveling a criticism that it does not solve the whole problem at once is not very productive. Even the hardest problems tend to yield to incremental approaches. If you have a better idea, by all means, suggest it.
leveling a criticism that it does not solve the whole problem at once is not very productive.
I’m not trying to be negative for the sake of being negative, or even for the sake of criticizing your proposal—I was disagreeing with your prediction that CFAR will have an easier time of meeting GiveWell’s requirements.
(I actually like your proposal quite a bit, and I think it’s an avenue that CFAR should investigate. But I still think that the verification problem is hard, and hence I predict that CFAR will not be very good at providing GiveWell with a workable rationality metric.)
This is a harder question than “which one of two given behaviors is more rational in a given setting?”, that’s why I suggested starting with the latter. Once you accumulate enough answers like that, you can start assembling them into a more general metric.
I maintain that this is a very hard problem. We know what the correct answers to various cognitive bias quizzes are (e.g. the conjunction fallacy questions questions), but it’s not clear that aggregating a lot of these tests corresponds to what we really mean when we say “rationality”.
Gotta start somewhere. I proposed a step that may or may not lead in the right direction, leveling a criticism that it does not solve the whole problem at once is not very productive. Even the hardest problems tend to yield to incremental approaches. If you have a better idea, by all means, suggest it.
I’m not trying to be negative for the sake of being negative, or even for the sake of criticizing your proposal—I was disagreeing with your prediction that CFAR will have an easier time of meeting GiveWell’s requirements.
(I actually like your proposal quite a bit, and I think it’s an avenue that CFAR should investigate. But I still think that the verification problem is hard, and hence I predict that CFAR will not be very good at providing GiveWell with a workable rationality metric.)