You’re right, but let me revise that slightly. In a simulated universe, some forms of self-improvement are possible, but others are cut off. Specifically, all forms of self-improvement which require more resources than you provide in the simulated universe are cut off. The problem is that that includes most of the interesting ones, and it’s entirely possible that it will self-modify into something bad but only when you give it more hardware.
You’re right, but let me revise that slightly. In a simulated universe, some forms of self-improvement are possible, but others are cut off. Specifically, all forms of self-improvement which require more resources than you provide in the simulated universe are cut off. The problem is that that includes most of the interesting ones, and it’s entirely possible that it will self-modify into something bad but only when you give it more hardware.