Currently there seems to be a dearth of mutual understanding between various groups of experts, and the recent discussions I read here and elsewhere between Eliezer Yudkowsky, Paul Christiano, Robin Hanson and others don’t even seem to crux well, let alone represent the opponents’ position faithfully (as confirmed by the other side). Charity has never been Eliezer’s core strength, but I would have expected others at MIRI to help him there. So, before you can put a reasonable confidence interval on the question “Are these risks actually likely to materialize?” you may have to do some legwork to get the relevant experts to double-crux or something.
An “expert alignment” problem, if you will, that needs to be solved before the AI alignment problem.
This is actually another related area of my research: To the extent that we cannot get people to sit down and agree on double cruxes, can we still assign some reasonable likelihoods and/or uncertainty estimates for those likelihoods? After all, we do ultimately need to make decisions here! Or if it turns out that we literally cannot use any numbers here, how do we best make decisions anyway?
It’s an interesting question, I think Scott A explored it as https://slatestarcodex.com/2019/06/03/repost-epistemic-learned-helplessness/ . But it would likely be inferior to figuring out a way for people to either double-crux, or at least do some kind of adversarial collaboration. Seems a lot easier than the problem we are trying to address, so what hope is there for the bigger problem if this one remains unresolved?
Currently there seems to be a dearth of mutual understanding between various groups of experts, and the recent discussions I read here and elsewhere between Eliezer Yudkowsky, Paul Christiano, Robin Hanson and others don’t even seem to crux well, let alone represent the opponents’ position faithfully (as confirmed by the other side). Charity has never been Eliezer’s core strength, but I would have expected others at MIRI to help him there. So, before you can put a reasonable confidence interval on the question “Are these risks actually likely to materialize?” you may have to do some legwork to get the relevant experts to double-crux or something.
An “expert alignment” problem, if you will, that needs to be solved before the AI alignment problem.
This is actually another related area of my research: To the extent that we cannot get people to sit down and agree on double cruxes, can we still assign some reasonable likelihoods and/or uncertainty estimates for those likelihoods? After all, we do ultimately need to make decisions here! Or if it turns out that we literally cannot use any numbers here, how do we best make decisions anyway?
It’s an interesting question, I think Scott A explored it as https://slatestarcodex.com/2019/06/03/repost-epistemic-learned-helplessness/ . But it would likely be inferior to figuring out a way for people to either double-crux, or at least do some kind of adversarial collaboration. Seems a lot easier than the problem we are trying to address, so what hope is there for the bigger problem if this one remains unresolved?