I am unlikely to post on the EA forum. (I only recently started posting much here, and I find most of EA rather unconvincing, aside from the one sentence summary, which is obviously a good thing.) Considering my negativity toward long-termism, I’m glad you decided more on the productive side for your response. My response is a bit long, I didn’t manage to get what I was trying to say down when it was shorter. Feel free to ignore it.
I will state that all of that is AI safety. Even the safety of the AI is determined by the overarching world upon which it is acting. A perfectly well controlled AI is unsafe if regulations followed by defense-bot-3000 state that all rebels must be ended, and everyone matches the definition of a rebel. The people that built defense-bot-3000 probably didn’t intend to end humanity because a human law said to. Identically, they probably didn’t mean for defense-bot-4000 to stand by and let it happen because a human is required in the loop by the 4000 version, and defense-bot-3000 made sure to kill those in charge of defense-bot-4000 at the start for its instrumental value.
Should a police bot let criminals it can prove are guilty run free, because their actions are justified in this instance? Should a facial recognition system point out that it has determined that new intern matches a spy for the government of that country? Should people be informed that a certain robot is malfunctioning, and likely to destroy an important machine in a hospital [when that means the people will simply destroy the sapient robot, but if the machine is destroyed people might die]? These are moral, and legal governance questions, that are also clearly AI safety questions.
I’d like to compare it to computer science where we know seemingly very disparate things are theoretically identical, such as iteration versus recursion, and hardware vs software. Regulation internal to the AI is the narrow construal of AI safety, while regulation external to it is governance. (Whether this regulation is on people or on the AI directly can be an important distinction, but either way it is still governance.)
Governance is thus actually a subset of AI safety broadly construed. And it should be broadly construed, since there is no difference between an inbuilt part of the AI and a part of the environment it is being used in if the lead to the same actions.
That wasn’t actually my point though. The definition of whether or not you call it AI safety isn’t important. You want to make it safe to have AI in use in society through regulation and cultural action. If you don’t understand AI, your regulation and cultural bits will be wrong. You do not currently understand AI, especially what effects it will actually have dealing with people [since sufficient AIs don’t exist to get data, and current approaches are not well understood in terms of why they do what they do].
Human culture has been massively changed by computers, the internet, cellphones, and so on. If I was older, I’d have a much longer list. If [and this is a big if] AI turns out to be that big of a thing, you can’t know what it will look like at this stage. That’s why you have to wait to find out [while trying to figure out what it will actually do.] If AI turns out to mostly be good at tutoring people, you need completely different regulation that if it turns out to only be good at war, and both are very different than if it is good at a wide variety of things.
Questions of human society rest on two things. First, what are people truly like on the inside. We aren’t good at figuring that out, but we have documented several thousand years of trying, and we’re starting to learn. Second, what is the particular culture like? Actual human level AI would massively change all of our cultures, to fit or banish the contours of the actual and perceived effects of the devices. (Also, what are the AI’s like on the inside? What are their natures? What cultures would spring up amongst different AIs?)
I agree that regulation is harder to do before you know all the details of the technology, but it doesn’t seem obviously doomed, and it seems especially-not-doomed to productively think about what regulations would be good (which is the vast majority of current AI governance work by longtermists).
As a canonical example I’d think of the Asilomar conference, which I think happened well before the details of the technology were known. There are a few more examples, but overall not many. I think that’s primarily because we don’t usually try to foresee problems because we’re too caught up in current problems, so I don’t see that as a very strong update against thinking about governance in advance.
Perhaps I was unclear. I object to the idea that you should get attached to any ideas now, not that you shouldn’t think about them. People being people, they are much more prone to getting attached to their ideas than is wise. Understand before risking attachment.
The problem with AI governance, is that AI is a mix between completely novel abilities, and things humans have been doing as long as there have been humans. The latter don’t need special ‘AI governance’ and the former are not understood.
(It should be noted that I am absolutely certain that AI will not take off quickly if it ever does takeoff beyond human limits.)
The Asilomar conference isn’t something I’m particularly familiar with, but it sounds like people actually had significant hands on experience with the technology, and understood them already. They stopped the experiments because they needed the clarity, not because someone else made rules earlier. There are not details as to whether they did a good job, and the recommendations seem very generic. Of course, it is wikipedia. We are not at this point with nontrivial AI. Needless to say, I don’t think this is against my point.
I am unlikely to post on the EA forum. (I only recently started posting much here, and I find most of EA rather unconvincing, aside from the one sentence summary, which is obviously a good thing.) Considering my negativity toward long-termism, I’m glad you decided more on the productive side for your response. My response is a bit long, I didn’t manage to get what I was trying to say down when it was shorter. Feel free to ignore it.
I will state that all of that is AI safety. Even the safety of the AI is determined by the overarching world upon which it is acting. A perfectly well controlled AI is unsafe if regulations followed by defense-bot-3000 state that all rebels must be ended, and everyone matches the definition of a rebel. The people that built defense-bot-3000 probably didn’t intend to end humanity because a human law said to. Identically, they probably didn’t mean for defense-bot-4000 to stand by and let it happen because a human is required in the loop by the 4000 version, and defense-bot-3000 made sure to kill those in charge of defense-bot-4000 at the start for its instrumental value.
Should a police bot let criminals it can prove are guilty run free, because their actions are justified in this instance? Should a facial recognition system point out that it has determined that new intern matches a spy for the government of that country? Should people be informed that a certain robot is malfunctioning, and likely to destroy an important machine in a hospital [when that means the people will simply destroy the sapient robot, but if the machine is destroyed people might die]? These are moral, and legal governance questions, that are also clearly AI safety questions.
I’d like to compare it to computer science where we know seemingly very disparate things are theoretically identical, such as iteration versus recursion, and hardware vs software. Regulation internal to the AI is the narrow construal of AI safety, while regulation external to it is governance. (Whether this regulation is on people or on the AI directly can be an important distinction, but either way it is still governance.)
Governance is thus actually a subset of AI safety broadly construed. And it should be broadly construed, since there is no difference between an inbuilt part of the AI and a part of the environment it is being used in if the lead to the same actions.
That wasn’t actually my point though. The definition of whether or not you call it AI safety isn’t important. You want to make it safe to have AI in use in society through regulation and cultural action. If you don’t understand AI, your regulation and cultural bits will be wrong. You do not currently understand AI, especially what effects it will actually have dealing with people [since sufficient AIs don’t exist to get data, and current approaches are not well understood in terms of why they do what they do].
Human culture has been massively changed by computers, the internet, cellphones, and so on. If I was older, I’d have a much longer list. If [and this is a big if] AI turns out to be that big of a thing, you can’t know what it will look like at this stage. That’s why you have to wait to find out [while trying to figure out what it will actually do.] If AI turns out to mostly be good at tutoring people, you need completely different regulation that if it turns out to only be good at war, and both are very different than if it is good at a wide variety of things.
Questions of human society rest on two things. First, what are people truly like on the inside. We aren’t good at figuring that out, but we have documented several thousand years of trying, and we’re starting to learn. Second, what is the particular culture like? Actual human level AI would massively change all of our cultures, to fit or banish the contours of the actual and perceived effects of the devices. (Also, what are the AI’s like on the inside? What are their natures? What cultures would spring up amongst different AIs?)
I agree that regulation is harder to do before you know all the details of the technology, but it doesn’t seem obviously doomed, and it seems especially-not-doomed to productively think about what regulations would be good (which is the vast majority of current AI governance work by longtermists).
As a canonical example I’d think of the Asilomar conference, which I think happened well before the details of the technology were known. There are a few more examples, but overall not many. I think that’s primarily because we don’t usually try to foresee problems because we’re too caught up in current problems, so I don’t see that as a very strong update against thinking about governance in advance.
Perhaps I was unclear. I object to the idea that you should get attached to any ideas now, not that you shouldn’t think about them. People being people, they are much more prone to getting attached to their ideas than is wise. Understand before risking attachment.
The problem with AI governance, is that AI is a mix between completely novel abilities, and things humans have been doing as long as there have been humans. The latter don’t need special ‘AI governance’ and the former are not understood.
(It should be noted that I am absolutely certain that AI will not take off quickly if it ever does takeoff beyond human limits.)
The Asilomar conference isn’t something I’m particularly familiar with, but it sounds like people actually had significant hands on experience with the technology, and understood them already. They stopped the experiments because they needed the clarity, not because someone else made rules earlier. There are not details as to whether they did a good job, and the recommendations seem very generic. Of course, it is wikipedia. We are not at this point with nontrivial AI. Needless to say, I don’t think this is against my point.