I was having a conversation with a religious friend of mine who argued that even if materialism was true, we would never be able to fully understand or replicate human intelligence because a physical system cannot understand itself—it would need to use the resources contained within it to perform that understanding, excluding the possibility of full understanding.
I countered with the following argument. Assume you are what your neurons are doing, and suppose you wish to extend your consciousness to fully grasp yourself (be aware of the larger systems functioning of your neuronal circuits, as well as possibly the smaller biochemical details, and the larger conceptual maps). Since consciousness offers us gestalt parallel information processing, and we will assume it can be extended to arbitrarily large concurrent information flow, one could create a (much larger) co-brain which consciously perceives all the functioning of your original brain. Now you can identify with your old consciousness and the newly added (much more expansive) co-consciousness.
The problem is that now you do not understand the full brain & co-brain system. But you can perform the process again, adding a co-co-brain which gives a realtime gestalt understanding of the co-brain consciousness. Since this process may be performed to arbitrarily large nesting levels, we can say that any physical system that is like the brain is ω-self-aware, with n-self-aware referring to the nesting level n. Since we do not expect the neural structures required to encode an n-self-aware system in an n+1-self-aware system to be any functionally different, we can say we’ve satisfactorily produced a physical system with full understanding of itself. Denying this would be equivalent to claiming we do not understand the natural numbers because we have not written every one of them down.
Ask LW: ω-self-aware systems
I was having a conversation with a religious friend of mine who argued that even if materialism was true, we would never be able to fully understand or replicate human intelligence because a physical system cannot understand itself—it would need to use the resources contained within it to perform that understanding, excluding the possibility of full understanding.
I countered with the following argument. Assume you are what your neurons are doing, and suppose you wish to extend your consciousness to fully grasp yourself (be aware of the larger systems functioning of your neuronal circuits, as well as possibly the smaller biochemical details, and the larger conceptual maps). Since consciousness offers us gestalt parallel information processing, and we will assume it can be extended to arbitrarily large concurrent information flow, one could create a (much larger) co-brain which consciously perceives all the functioning of your original brain. Now you can identify with your old consciousness and the newly added (much more expansive) co-consciousness.
The problem is that now you do not understand the full brain & co-brain system. But you can perform the process again, adding a co-co-brain which gives a realtime gestalt understanding of the co-brain consciousness. Since this process may be performed to arbitrarily large nesting levels, we can say that any physical system that is like the brain is ω-self-aware, with n-self-aware referring to the nesting level n. Since we do not expect the neural structures required to encode an n-self-aware system in an n+1-self-aware system to be any functionally different, we can say we’ve satisfactorily produced a physical system with full understanding of itself. Denying this would be equivalent to claiming we do not understand the natural numbers because we have not written every one of them down.
Does anyone see any trouble with this argument?