I have several short comments about part 3, short not because there is little to say, but because I want to make the points and do not have time to discuss them in depth right now.
1) If multi-agent systems are more likely to succeed in achieving GAI, we should shut up about why they are important. I’m concerned about unilateralist curse, and would ask that someone from MIRI weigh in on this.
2) I agree that multi-agent systems are critical, but for different (non-contradictory) reasons—I think multi-agent systems are likely to be less safe and harder to understand. See draft of my forthcoming article here: https://arxiv.org/abs/1810.10862
3) If this is deemed to be important, the technical research directions point to here are under-specified and too vague to be carried out. I think concretizing them would be useful. (I’d love to chat about this, as I have ideas in this vein. If you are interested in talking, feel free to be in touch—about.me/davidmanheim .)
The technical research direction specification can be in all cases “expanded” from the “seed idea” described here. (We are already working on some of those.) I’m not sure if it’s the best thing to publish now—to me, it seems better to do some iterations on “specify—try to work on it” first, before publishing the expansions.
I have several short comments about part 3, short not because there is little to say, but because I want to make the points and do not have time to discuss them in depth right now.
1) If multi-agent systems are more likely to succeed in achieving GAI, we should shut up about why they are important. I’m concerned about unilateralist curse, and would ask that someone from MIRI weigh in on this.
2) I agree that multi-agent systems are critical, but for different (non-contradictory) reasons—I think multi-agent systems are likely to be less safe and harder to understand. See draft of my forthcoming article here: https://arxiv.org/abs/1810.10862
3) If this is deemed to be important, the technical research directions point to here are under-specified and too vague to be carried out. I think concretizing them would be useful. (I’d love to chat about this, as I have ideas in this vein. If you are interested in talking, feel free to be in touch—about.me/davidmanheim .)
Nice! We should chat about that.
The technical research direction specification can be in all cases “expanded” from the “seed idea” described here. (We are already working on some of those.) I’m not sure if it’s the best thing to publish now—to me, it seems better to do some iterations on “specify—try to work on it” first, before publishing the expansions.