This problem sounds awfully similar to the halting problem to me. If we can’t tell whether a Turing machine will eventually terminate without actually running it, how could we ever tell if a Turing machine will experience consciousness without running it?
Has anyone attempted to prove the statement “Consciousness of a Turing machine is undecideable”? The proof (if it’s true) might look a lot like the proof that the halting problem is undecideable. Sadly, I don’t quite understand how that proof works either, so I can’t use it as a basis for the consciousness problem. It just seems that figuring out if a Turing machine is conscious, or will ever achieve consciousness before halting, is much harder than figuring out if it halts.
The halting problem doesn’t imply that we can never tell whether a particular program halts without actually running it. (You can think of many simple programs which definitely halt, and other simple programs which are definitely infinite loops.)
It means, instead, that there exist relatively short but extremely pathological Turing machines, such that no Turing machine can be built that could solve the halting problem for every Turing machine. (Indeed, the idea of the proof is that a reputed halting-problem-solver is itself pathological, as can be seen by feeding it a modified version of itself as input.)
But these pathological ones are not at all the kind of Turing machines we would create to do any functional task; the only reason I could think for us to seek them out would be to find Busy Beaver numbers.
Has anyone attempted to prove the statement “Consciousness of a Turing machine is undecideable”? The proof (if it’s true) might look a lot like the proof that the halting problem is undecideable.
Your conjecture seems to follow from Rice’s theorem, assuming the personhood of a running computation is a property of the partial function its algorithm computes. Also, I think you can prove your conjecture by taking a certain proof that the Halting Problem is undecidable and replacing ‘halts’ with ‘is conscious’. I can track this down if you’re still interested.
But this doesn’t mess up Eliezer’s plans at all: you can have “nonhalting predicates” that output “doesn’t halt” or “I don’t know”, analogous to the nonperson predicates proposed here.
This problem sounds awfully similar to the halting problem to me. If we can’t tell whether a Turing machine will eventually terminate without actually running it, how could we ever tell if a Turing machine will experience consciousness without running it?
Has anyone attempted to prove the statement “Consciousness of a Turing machine is undecideable”? The proof (if it’s true) might look a lot like the proof that the halting problem is undecideable. Sadly, I don’t quite understand how that proof works either, so I can’t use it as a basis for the consciousness problem. It just seems that figuring out if a Turing machine is conscious, or will ever achieve consciousness before halting, is much harder than figuring out if it halts.
The halting problem doesn’t imply that we can never tell whether a particular program halts without actually running it. (You can think of many simple programs which definitely halt, and other simple programs which are definitely infinite loops.)
It means, instead, that there exist relatively short but extremely pathological Turing machines, such that no Turing machine can be built that could solve the halting problem for every Turing machine. (Indeed, the idea of the proof is that a reputed halting-problem-solver is itself pathological, as can be seen by feeding it a modified version of itself as input.)
But these pathological ones are not at all the kind of Turing machines we would create to do any functional task; the only reason I could think for us to seek them out would be to find Busy Beaver numbers.
Your conjecture seems to follow from Rice’s theorem, assuming the personhood of a running computation is a property of the partial function its algorithm computes. Also, I think you can prove your conjecture by taking a certain proof that the Halting Problem is undecidable and replacing ‘halts’ with ‘is conscious’. I can track this down if you’re still interested.
But this doesn’t mess up Eliezer’s plans at all: you can have “nonhalting predicates” that output “doesn’t halt” or “I don’t know”, analogous to the nonperson predicates proposed here.
Um, I happened to write an explanation of the Halting Problem proof in a comment over here. Please tell me which parts seem unclear to you.