I think the most likely concrete explanation for the hypothetical of non-world-eating alien tech is a non-corrigible anti-AI-risk AGI that was built by aliens in their own 21st century to stop AI risk. In this sort of partial failure of alignment, the AGI doesn’t change more than it has to in order to prevent AI risk, and it doesn’t itself develop, just keeps the world low-tech in this particular way. If such an AGI didn’t kill its developers, this means that it couldn’t be reformed or defeated since, and so we are likely safe from world-eating levels of AI risk as well, but we won’t be able to actually build powerful AGIs and things like aging might need to be solved manually. If it did kill its developers, then building even an aligned AGI might trigger the alien AGI, but also in that case it might be possible to work around, since nobody lived to try yet, and its purpose might be unrelated to prevention of AI risk.
I don’t think looking into the possibility of actual low-tech alien machines is meaningfully useful. Either it’s complete nonsense, or the alien machines probably exert some effort towards hiding the evidence, so that more effort would mostly fail to help with producing a compelling argument that it’s not complete nonsense. (The remaining possibility is that there’s legible argument/evidence that’s easily available, which is not actually the case.)
Threatening to kill a species for attempting AGI would be unnecessarily high impact. It can just blow up the datacenter when it sees we’re close. Knowing it’s there and losing a datacenter would be deterrent enough. Maybe people would try a few more times in secret, and it would detect it and blow those up too. We wouldn’t know how. If we develop to the point where we start replicating its tech and finding the real blindspots in its sensors, maybe then it would have to start voicing threats and muscling in.
Or maybe at that point we will have matured in the only way it will accept, nearly AGI level ourselves, and we’ll finally be given our AGI license.
A more straightforward example of the kind of system I’m thinking about is an alien upload (or LLM simulacrum) bureaucracy. It clearly failed, possibly because the system refuses to recognize anyone as authorized to change its structure. More generally, this could be a single non-corrigible AGI rather than an administrator-less bureaucracy of uploads, but that seems more difficult to imagine.
This kind of thing could maintain hard power by denying opportunity for excessive technological growth for everyone else, and remain unchanged indefinitely because it’s basically a software system operating in an environment that it keeps within original expectations at the time of its design. The intelligent components (uploads or LLM simulacra) could reset to factory settings on a schedule.
Arbitrary failure modes could then persist in the system’s effective mandate, since there is nobody in a position to fix the problems and make it more reasonable. It could limit its builders to steampunk levels of tech, or retaliate without warning if some criteria for that are met.
I think the most likely concrete explanation for the hypothetical of non-world-eating alien tech is a non-corrigible anti-AI-risk AGI that was built by aliens in their own 21st century to stop AI risk. In this sort of partial failure of alignment, the AGI doesn’t change more than it has to in order to prevent AI risk, and it doesn’t itself develop, just keeps the world low-tech in this particular way. If such an AGI didn’t kill its developers, this means that it couldn’t be reformed or defeated since, and so we are likely safe from world-eating levels of AI risk as well, but we won’t be able to actually build powerful AGIs and things like aging might need to be solved manually. If it did kill its developers, then building even an aligned AGI might trigger the alien AGI, but also in that case it might be possible to work around, since nobody lived to try yet, and its purpose might be unrelated to prevention of AI risk.
I don’t think looking into the possibility of actual low-tech alien machines is meaningfully useful. Either it’s complete nonsense, or the alien machines probably exert some effort towards hiding the evidence, so that more effort would mostly fail to help with producing a compelling argument that it’s not complete nonsense. (The remaining possibility is that there’s legible argument/evidence that’s easily available, which is not actually the case.)
Threatening to kill a species for attempting AGI would be unnecessarily high impact. It can just blow up the datacenter when it sees we’re close. Knowing it’s there and losing a datacenter would be deterrent enough. Maybe people would try a few more times in secret, and it would detect it and blow those up too. We wouldn’t know how. If we develop to the point where we start replicating its tech and finding the real blindspots in its sensors, maybe then it would have to start voicing threats and muscling in.
Or maybe at that point we will have matured in the only way it will accept, nearly AGI level ourselves, and we’ll finally be given our AGI license.
A more straightforward example of the kind of system I’m thinking about is an alien upload (or LLM simulacrum) bureaucracy. It clearly failed, possibly because the system refuses to recognize anyone as authorized to change its structure. More generally, this could be a single non-corrigible AGI rather than an administrator-less bureaucracy of uploads, but that seems more difficult to imagine.
This kind of thing could maintain hard power by denying opportunity for excessive technological growth for everyone else, and remain unchanged indefinitely because it’s basically a software system operating in an environment that it keeps within original expectations at the time of its design. The intelligent components (uploads or LLM simulacra) could reset to factory settings on a schedule.
Arbitrary failure modes could then persist in the system’s effective mandate, since there is nobody in a position to fix the problems and make it more reasonable. It could limit its builders to steampunk levels of tech, or retaliate without warning if some criteria for that are met.