The information content of a mind cannot exceed the amount of information necessary to specify a representation of that same mind. Therefore, while the AI can understand in principle that it is made up of transistors etc., its self-representation necessary has some blank areas.
This is strictly true if you’re talking about the working memory that is part of a complete model of your “mind”. But a mind can access an unbounded amount of externally stored data, where a complete self-representation can be stored.
A Turing Machine of size N can run on an unbounded-size tape. A von Neumann PC with limited main memory can access an unbounded-size disk.
Although we can only load a part of the data into working memory at a time, we can use virtual memory to run any algorithm written in terms of the data as a whole. If we had an AI program, we could run it on today’s PCs and while we could run out of disk space, we couldn’t run out of RAM.
This is strictly true if you’re talking about the working memory that is part of a complete model of your “mind”. But a mind can access an unbounded amount of externally stored data, where a complete self-representation can be stored.
A Turing Machine of size N can run on an unbounded-size tape. A von Neumann PC with limited main memory can access an unbounded-size disk.
Although we can only load a part of the data into working memory at a time, we can use virtual memory to run any algorithm written in terms of the data as a whole. If we had an AI program, we could run it on today’s PCs and while we could run out of disk space, we couldn’t run out of RAM.