Programmers are also supposed to search the space of Turing machines, which seems really hard. Programming in Brainfuck is hard. All the software written in higher level languages are points of a mere subspace… If optimizing in this subspace has proven to be so effective, I don’t think we have a reason to worry about uncompressible subspaces containing the only working solution for our problems, namely more intelligent AI designs.
Programmers are also supposed to search the space of Turing machines, which seems really hard. Programming in Brainfuck is hard. All the software written in higher level languages are points of a mere subspace… If optimizing in this subspace has proven to be so effective, I don’t think we have a reason to worry about uncompressible subspaces containing the only working solution for our problems, namely more intelligent AI designs.