″...under the assumption that the subset of dangerous satisficing outputs D is much smaller than the set of all satisficing outputs S, and that we are able to choose a number m such that |D|≪m<|S|.”
I highly doubt that D≪S is true for anything close to a pivotal act since most pivotal acts at some point involve deploying technology that can trivially take over the world.
For anything less ambitious the proposed technique looks very useful. Strict cyber- and physical security will of course be necessary to prevent the scenario Gwern mentions.
″...under the assumption that the subset of dangerous satisficing outputs D is much smaller than the set of all satisficing outputs S, and that we are able to choose a number m such that |D|≪m<|S|.”
I highly doubt that D≪S is true for anything close to a pivotal act since most pivotal acts at some point involve deploying technology that can trivially take over the world.
For anything less ambitious the proposed technique looks very useful. Strict cyber- and physical security will of course be necessary to prevent the scenario Gwern mentions.