I don’t think screwing up permanently becomes a real concern until the invention of nuclear weapons, and that’s such a long ways ahead of the starting point for this exercise that I don’t think we can influence how it goes.
Surely we can have nontrivial influence both on variables relating to specific technologies like nukes, and on general variables along the lines of “caution about technology”.
We don’t want them to advance quickly; we want them to advance with a low probability of screwing up permanently.
I don’t think screwing up permanently becomes a real concern until the invention of nuclear weapons, and that’s such a long ways ahead of the starting point for this exercise that I don’t think we can influence how it goes.
Surely we can have nontrivial influence both on variables relating to specific technologies like nukes, and on general variables along the lines of “caution about technology”.