While you can envision something, it doesn’t mean that envisioned is logically coherent/possible/trivial to achieve. In one fantasy novel protagonists travel to the world were physical laws make it impossible to light matches. It’s very easy to imagine that you try to light match again and again and fail, but “impossibility to light matches” implies such drastic changes in physical laws that Earth’s life probably can’t sustain itself here, because heads of matches contain phosphorous and phosphorous is vital for bodily processes (and I don’t even go for universe-wide consequences of different physical constants).
So it’s very easy to imagine terminal where you print “how to solve alignment?”, press “enter”, get solution after an hour and everybody lives happily ever after. But I can’t imagine how this thing should work without developing agency, if I don’t say in some moment “here happens Magic that prevents this system from developing agency”.
While you can envision something, it doesn’t mean that envisioned is logically coherent/possible/trivial to achieve. In one fantasy novel protagonists travel to the world were physical laws make it impossible to light matches. It’s very easy to imagine that you try to light match again and again and fail, but “impossibility to light matches” implies such drastic changes in physical laws that Earth’s life probably can’t sustain itself here, because heads of matches contain phosphorous and phosphorous is vital for bodily processes (and I don’t even go for universe-wide consequences of different physical constants).
So it’s very easy to imagine terminal where you print “how to solve alignment?”, press “enter”, get solution after an hour and everybody lives happily ever after. But I can’t imagine how this thing should work without developing agency, if I don’t say in some moment “here happens Magic that prevents this system from developing agency”.