Are you advocating as option A, ‘deduce a full design by armchair thought before implementing anything’? The success probability of that isn’t 1%. It’s zero, to as many decimal places as makes no difference.
We’re probably talking past each other. I’m saying “no you don’t get to build lots of general AIs in the process of solving the alignment problem and still stay alive” and (I think) you’re saying “no you don’t get to solve the alignment problem without writing a ton of code, lots of it highly highly related to AI”. I think both of those are true.
Are you advocating as option A, ‘deduce a full design by armchair thought before implementing anything’? The success probability of that isn’t 1%. It’s zero, to as many decimal places as makes no difference.
We’re probably talking past each other. I’m saying “no you don’t get to build lots of general AIs in the process of solving the alignment problem and still stay alive” and (I think) you’re saying “no you don’t get to solve the alignment problem without writing a ton of code, lots of it highly highly related to AI”. I think both of those are true.
Right, yes, I’m not suggesting the iterated coding activity can or should include ‘build an actual full-blown superhuman AGI’ as an iterated step.