I am optimistic about developing AGI collaboratively, especially through AI researchers cooperating. I’m not sure whether external incentives from government are the right way to achieve this—it seems likely that such regulation would be aimed at the wrong problems if it originated from government and not from AI researchers themselves. I’m more optimistic about some AI researchers developing guidelines and incentive structures themselves, that researchers buy into voluntarily, that maybe later get codified into law by governments, or adopted by companies for their AI research.
I would definitely want AI developers to participate in figuring out this stuff! Like I said in the post, the system is supposed to support them in creating the kind of an environment they want, rather than imposing something unwanted from the outside.
That said, voluntary arrangements only work to the extent that everyone has an incentive to follow them. For things like arms races, the fact that everyone has an incentive to participate in an arms race even if nobody wanted to, preventing people from opting into voluntary arrangements intended to avoid the race, is exactly the problem that this kind of thing is trying to help avoid.
I guess I’m confused about the path by which you hope to get the external incentives to be created. I’m advocating for a voluntary version that some people buy into (but not everyone because incentives), that later gets codified into actual external incentives (eg. law), whereas I read your post as suggesting that we push for external incentives like law, and when we’re drafting them we make sure to get input from AI researchers. These seem very different to me.
I guess I’m confused about the path by which you hope to get the external incentives to be created.
The way I’m thinking of it, this sentence seems to imply that AI development wouldn’t be facing any external incentives right now. But everyone is always operating under some set of external incentives, which unavoidably shape their behavior. And if they’re not intentionally designed ones, they are likely to be bad ones.
So the way I’d phrase it now, my proposal is neither “push for external incentives like law and get input from AI researchers in drafting them”, nor “establish voluntary codes to buy into and make them into external incentives later”. Rather it’s “get the AI researchers to give their input on what the current external incentives are like and how they could be better, and then use whatever policy instruments are available to shift those incentives to be more like the better ones”.
E.g. to take the specific example of liability legislation; there are already existing laws that are going to be applied if an AI system gets out of control and kills people. Is that existing legal framework, and the way it’s likely to be applied, good or bad for encouraging the kinds of behavior we’d like to see from AI developers? I don’t know, but at least I know that it was never designed with this specific intent in mind, so there may be things to improve on there.
I am optimistic about developing AGI collaboratively, especially through AI researchers cooperating. I’m not sure whether external incentives from government are the right way to achieve this—it seems likely that such regulation would be aimed at the wrong problems if it originated from government and not from AI researchers themselves. I’m more optimistic about some AI researchers developing guidelines and incentive structures themselves, that researchers buy into voluntarily, that maybe later get codified into law by governments, or adopted by companies for their AI research.
I would definitely want AI developers to participate in figuring out this stuff! Like I said in the post, the system is supposed to support them in creating the kind of an environment they want, rather than imposing something unwanted from the outside.
That said, voluntary arrangements only work to the extent that everyone has an incentive to follow them. For things like arms races, the fact that everyone has an incentive to participate in an arms race even if nobody wanted to, preventing people from opting into voluntary arrangements intended to avoid the race, is exactly the problem that this kind of thing is trying to help avoid.
I guess I’m confused about the path by which you hope to get the external incentives to be created. I’m advocating for a voluntary version that some people buy into (but not everyone because incentives), that later gets codified into actual external incentives (eg. law), whereas I read your post as suggesting that we push for external incentives like law, and when we’re drafting them we make sure to get input from AI researchers. These seem very different to me.
The way I’m thinking of it, this sentence seems to imply that AI development wouldn’t be facing any external incentives right now. But everyone is always operating under some set of external incentives, which unavoidably shape their behavior. And if they’re not intentionally designed ones, they are likely to be bad ones.
So the way I’d phrase it now, my proposal is neither “push for external incentives like law and get input from AI researchers in drafting them”, nor “establish voluntary codes to buy into and make them into external incentives later”. Rather it’s “get the AI researchers to give their input on what the current external incentives are like and how they could be better, and then use whatever policy instruments are available to shift those incentives to be more like the better ones”.
E.g. to take the specific example of liability legislation; there are already existing laws that are going to be applied if an AI system gets out of control and kills people. Is that existing legal framework, and the way it’s likely to be applied, good or bad for encouraging the kinds of behavior we’d like to see from AI developers? I don’t know, but at least I know that it was never designed with this specific intent in mind, so there may be things to improve on there.
Ah, I see, that makes sense, thanks for clarifying!