The most likely way to get to extremely safe AGI or ASI systems is not by humans creating them, it’s by other less-safe AGI systems creating them.
This does seem more likely, but managing to sidestep the less-safe AGI part would be safer. In particular, it might be possible to construct a safe AGI by using safe-if-wielded-responsibly tool AIs (that are not AGIs), if humanity takes enough time to figure out how to actually do that.
This does seem more likely, but managing to sidestep the less-safe AGI part would be safer. In particular, it might be possible to construct a safe AGI by using safe-if-wielded-responsibly tool AIs (that are not AGIs), if humanity takes enough time to figure out how to actually do that.
The current paradigm of AI research makes it hard to make really pure tool AIs.
We have software tools, like Wolfram Alpha, and we have LLM-derived systems.
This is probably the set of tools we will either win or die with