Now it is true that human programmers spend a lot of time
testing their programs and “simulating” them in debuggers,
but there is no reason that all the world’s programs could
not be delivered without doing any of that: those techniques
are simply not necessary to delivering code that is assured
to have the properties desired by our civilization.
For example, if there were enough programmers with the
necessary skills, every program could be delievered with a
mathematical proof that it has the properties that it was
intended to have, and this would completely eliminate the
need to use testing or debugging. [...]
In your dreams—testing is far more comprehensive and
effective that attempting to prove what your program can do.
If there were a lot more programmers, they would probably be writing more programs—not exhastively proving irrelevant facts about existing programs that have already been properly tested.
There are in fact not enough programmers with the necessary
skills to deliver such “correctness proofs” for all the
programs that the world’s programmers currently deliver, but
superintelligences will not suffer from that limitation.
IMHO they will almost never resort to testing and debugging
the programs they create. They will instead use more
efficient techniques.
It seems unlikely. There really is no substitute for testing.
In your dreams—testing is far more comprehensive and effective that attempting to prove what your program can do.
If there were a lot more programmers, they would probably be writing more programs—not exhastively proving irrelevant facts about existing programs that have already been properly tested.
It seems unlikely. There really is no substitute for testing.