I think that this infinite regress can be converted into a loop. Given an infinite sequence of layers, in which the job of layer n+1 is to optimise layer n. Each layer is a piece of programming code. After the first couple of layers, these layers will start to look very similar. You could have layer 3 being a able to optimize both layer 2 and layer 3.
One model is that your robot just sits and thinks for an hour. At the end of that hour, it designs what it thinks is the best code it can come up with, and runs that. To the original AI, anything outside the original hour is external, it is answering the question “what pattern of bits on this hard disk will lead to the best outcome.” It can take all these balances and tradeoffs into account in whatever way it likes. If it hasn’t come up with any good ideas yet, it could copy its code, add a crude heuristic that makes it run randomly when thinking (to avoid the preditors) and think for longer.
I think that this infinite regress can be converted into a loop. Given an infinite sequence of layers, in which the job of layer n+1 is to optimise layer n. Each layer is a piece of programming code. After the first couple of layers, these layers will start to look very similar. You could have layer 3 being a able to optimize both layer 2 and layer 3.
One model is that your robot just sits and thinks for an hour. At the end of that hour, it designs what it thinks is the best code it can come up with, and runs that. To the original AI, anything outside the original hour is external, it is answering the question “what pattern of bits on this hard disk will lead to the best outcome.” It can take all these balances and tradeoffs into account in whatever way it likes. If it hasn’t come up with any good ideas yet, it could copy its code, add a crude heuristic that makes it run randomly when thinking (to avoid the preditors) and think for longer.