This is an important consideration. I don’t think that government power travels inevitably to bad hands, but I do think it happens far too often. Strengthening democracy is the one useful move I can think of here, but that’s pretty vague.
Not pushing for nationalization doesn’t seem like a useful response. It will be soft-nationalized sooner or later; takeoff is going to be too slow for the AGI to outwit the US national security apparatus before they figure out what a big deal it is. Pushing for nationalization or not might affect when it’s done, giving some modicum of control.
Other questions I wish I (or people advocating for any policy w.r.t. AGI) had answers to include (i.a.) “How could I/we/anyone ensure that the resulting AGI project actually benefits everyone? Who, in actual concrete practice, would end up effectively having control over the AGI? How could (e.g.) the public hold those people accountable, even as those people gain unassailable power? How do we ensure that those people are not malevolent to begin with, and also don’t become corrupted by power? What kinds of oversight mechanisms could be built, and how?”
I agree that “strengthening democracy” sounds nice, and also that it’s too vague to be actionable. Also, what exactly would be the causal chain from “stronger democracy” (whatever that means) to “command structure in the nationalized AGI project is trustworthy and robustly aligned to the common good”?
If you have any more concrete ideas in this domain, I’d be interested to read about them!
This is an important consideration. I don’t think that government power travels inevitably to bad hands, but I do think it happens far too often. Strengthening democracy is the one useful move I can think of here, but that’s pretty vague.
Not pushing for nationalization doesn’t seem like a useful response. It will be soft-nationalized sooner or later; takeoff is going to be too slow for the AGI to outwit the US national security apparatus before they figure out what a big deal it is. Pushing for nationalization or not might affect when it’s done, giving some modicum of control.
I notice that I have almost no concrete model of what that sentence means. A couple of salient questions[1] I’d be very curious to hear answers to:
What concrete ways exist for affecting when (and how) nationalization is done? (How, concretely, does one “push” for/against nationalization of AGI?)
By what concrete causal mechanism could pushing for nationalization confer a modicum of control; and control over what exactly, and to whom?
Other questions I wish I (or people advocating for any policy w.r.t. AGI) had answers to include (i.a.) “How could I/we/anyone ensure that the resulting AGI project actually benefits everyone? Who, in actual concrete practice, would end up effectively having control over the AGI? How could (e.g.) the public hold those people accountable, even as those people gain unassailable power? How do we ensure that those people are not malevolent to begin with, and also don’t become corrupted by power? What kinds of oversight mechanisms could be built, and how?”
I agree that “strengthening democracy” sounds nice, and also that it’s too vague to be actionable. Also, what exactly would be the causal chain from “stronger democracy” (whatever that means) to “command structure in the nationalized AGI project is trustworthy and robustly aligned to the common good”?
If you have any more concrete ideas in this domain, I’d be interested to read about them!