I do mean it rather seriously. There are theoretical frameworks already in play, directed at solving the two major subproblems that you identify, i.e. creating raw superintelligence, and (let’s say) identifying a friendly value system. I actually find it conceivable that the required breakthroughs are not far away, in the same way that e.g. imminent solutions to math’s Millennium Problems are conceivable—someone just needs to have the right insights.
Disclaimer: it’s hard to see too many levels above your own. This may still be underestimating the difficulty of friendliness.
I’d like to note that I’m skeptical of this for a few reasons:
1) this doesn’t even come close to making sense if you aren’t already as knowledgeable as one of the MIRI staff, in which case an 80-day resolve cycle to this effect might actually produce something. I think it is very unlikely that the entire problem can be resolved without an insane amount of intuition (both mathematical and normal). Even if a promising solution was found in that time frame, however,
2) there’s insufficient time for verification. Proving your intuitively-appealing idea will be safe is different than making strong arguments it will.
someone just needs to have the right insights.
Saying “there may be a short inferential distance to the full solution” (as if step-count is the main quantifier of difficulty / time required) misses the difficulty that some of these steps may entail. Yes, the challenge may be different than that posed to a rationalist in 1300 who realizes the horror of death and wants to live. Even if he had complete knowledge of what steps to take, it would be incredibly difficult (and probably impossible) for him to single-handedly build the machinery required to advance science beyond today’s frontier so as to ensure he continued living. That’s for the case in which he spends every waking moment taking an optimal action (with respect to a scientific gradient even more advanced than today’s). Yes, in that situation, there are too many insights and actions requiredfor him to survive.
In that sense, yes—the problem is, perhaps, literally able to be solved by one person’s efforts, but I still don’t think it’s a reasonable challenge. Maybe if you’re already on or near Eliezer’s level, this extended resolve cycle would be useful for generating new ideas.
I do mean it rather seriously. There are theoretical frameworks already in play, directed at solving the two major subproblems that you identify, i.e. creating raw superintelligence, and (let’s say) identifying a friendly value system. I actually find it conceivable that the required breakthroughs are not far away, in the same way that e.g. imminent solutions to math’s Millennium Problems are conceivable—someone just needs to have the right insights.
Disclaimer: it’s hard to see too many levels above your own. This may still be underestimating the difficulty of friendliness.
I’d like to note that I’m skeptical of this for a few reasons:
1) this doesn’t even come close to making sense if you aren’t already as knowledgeable as one of the MIRI staff, in which case an 80-day resolve cycle to this effect might actually produce something. I think it is very unlikely that the entire problem can be resolved without an insane amount of intuition (both mathematical and normal). Even if a promising solution was found in that time frame, however,
2) there’s insufficient time for verification. Proving your intuitively-appealing idea will be safe is different than making strong arguments it will.
Saying “there may be a short inferential distance to the full solution” (as if step-count is the main quantifier of difficulty / time required) misses the difficulty that some of these steps may entail. Yes, the challenge may be different than that posed to a rationalist in 1300 who realizes the horror of death and wants to live. Even if he had complete knowledge of what steps to take, it would be incredibly difficult (and probably impossible) for him to single-handedly build the machinery required to advance science beyond today’s frontier so as to ensure he continued living. That’s for the case in which he spends every waking moment taking an optimal action (with respect to a scientific gradient even more advanced than today’s). Yes, in that situation, there are too many insights and actions required for him to survive.
In that sense, yes—the problem is, perhaps, literally able to be solved by one person’s efforts, but I still don’t think it’s a reasonable challenge. Maybe if you’re already on or near Eliezer’s level, this extended resolve cycle would be useful for generating new ideas.