I think the observation that it just isn’t obvious that ems will come before de novo AI is sufficient to worry about the problem in the case that they don’t. Possibly while focusing more capabilities development towards creating ems (whatever that would look like)?
Also, would ems actually be powerful and capable enough to reliably stop a world-destroying non-em AGI, or an em about to make some world-destroying mistake because of its human-derived flaws? Or would we need to arm them with additional tools that fall under the umbrella of AGI safety anyway?
I think the observation that it just isn’t obvious that ems will come before de novo AI is sufficient to worry about the problem in the case that they don’t. Possibly while focusing more capabilities development towards creating ems (whatever that would look like)?
Also, would ems actually be powerful and capable enough to reliably stop a world-destroying non-em AGI, or an em about to make some world-destroying mistake because of its human-derived flaws? Or would we need to arm them with additional tools that fall under the umbrella of AGI safety anyway?