You talk about “governance by Friendly AGI” as if it’s a solved problem we’re just waiting to deploy, not speculation that might simply not be feasible even if we solve AGI alignment, which itself is plausibly unsolvable in the near term. You also conflate AI safety research with AI governance regimes. And note that the problems with governance generally aren’t a lack of intelligence by those in charge, it’s largely conflicting values and requirements. And with that said, you talk about modern liberal governments as if they are the worst thing we’ve experienced, “riddled with brokenness,” as if that’s the fault of the people in charge, not the deeply conflicting mandates that the populace gives them. And to the extent that the systemic failure is the fault of the untrustworthy incentives of those in charge, why would controllable or aligned AGI fix that?
Yes, stasis isn’t safe by default, but undirected progress isn’t a panacea, and governance certainly isn’t any closer to solved just because we have AI progress.
You talk about “governance by Friendly AGI” as if it’s a solved problem we’re just waiting to deploy, not speculation that might simply not be feasible even if we solve AGI alignment, which itself is plausibly unsolvable in the near term. You also conflate AI safety research with AI governance regimes. And note that the problems with governance generally aren’t a lack of intelligence by those in charge, it’s largely conflicting values and requirements. And with that said, you talk about modern liberal governments as if they are the worst thing we’ve experienced, “riddled with brokenness,” as if that’s the fault of the people in charge, not the deeply conflicting mandates that the populace gives them. And to the extent that the systemic failure is the fault of the untrustworthy incentives of those in charge, why would controllable or aligned AGI fix that?
Yes, stasis isn’t safe by default, but undirected progress isn’t a panacea, and governance certainly isn’t any closer to solved just because we have AI progress.