Whatever “alignment” means, the “impossibility problem” you refer to could be any of
An aligned system is impossible.
A provably aligned system is impossible.
There is no general deterministic algorithm to determine whether or not an arbitrary system is aligned.
An unaligned system is possible.
In analogy with the halting problem, 3. is the good one; 1. and 2. are obviously false, and 4. is true.
More meta, 3. could itself be unprovable.
However, a proof or disproof (or even a proof of undecidability) of 3. has no consequences for which the metaphor of nuclear fission bombs would not be absurd, so perhaps you means something completely different, and you’ve just phrased it in a confusing way? Or do you think 1. or 2. might be true?
Why would 3 be important? 3 is true of the halting problem, yet we still create and use lots of software that needs to halt, and the trueness of 3 for the halting problem doesn’t seem to be an issue in practice.
Whatever “alignment” means, the “impossibility problem” you refer to could be any of
An aligned system is impossible.
A provably aligned system is impossible.
There is no general deterministic algorithm to determine whether or not an arbitrary system is aligned.
An unaligned system is possible.
In analogy with the halting problem, 3. is the good one; 1. and 2. are obviously false, and 4. is true.
More meta, 3. could itself be unprovable.
However, a proof or disproof (or even a proof of undecidability) of 3. has no consequences for which the metaphor of nuclear fission bombs would not be absurd, so perhaps you means something completely different, and you’ve just phrased it in a confusing way? Or do you think 1. or 2. might be true?
Why would 3 be important? 3 is true of the halting problem, yet we still create and use lots of software that needs to halt, and the trueness of 3 for the halting problem doesn’t seem to be an issue in practice.