The Libertarian “brass” rule is a case in point: “Do not unto others as you would not have them do unto you,” which may be summarized as “Do no harm.”
Suppose you had perfect omniscience. (I’m not saying an AI would, I’m just setting up a hypothetical.) It might be the case that whenever you consider doing something, you notice that it has some harmful effect in the future on someone you consider morally important. You then end up not being able to do anything, including not being able to do nothing- because doing nothing also leads to harm in the future. So we can’t just ban all harm; we need to somehow proportionally penalize harm, so that it’s better to do less harm than more harm. But there are good things that are worth purchasing with harm, and so then we’re back into tradeoff territory and maximizing profit instead of just minimizing cost.
(Indeed, the function of morality seems to mostly be to internalize externalities, rather than simply minimize negative externalities. Rules like “do no harm” serve for this purpose by making you consider harm to others before you act, which hopefully prevents you from doing things that are net negative while still allowing you to do things that are net positive.)
The brass rule does not require rendering assistance.
Humans have some idea of commission and omission: consider the difference between me running my car into you, you walking into the path of my car, and me not grabbing you to prevent you from walking into the path of a car. The first would be murder, the second possibly manslaughter and possibly not, and the third is not a crime. But that’s a human-sized sense of commission and omission. It’s not at all clear that AGIs will operate on the same scale.
When one takes a system-sized viewpoint, commission and omission become very different. The choice to not add a safety feature that makes accidents less likely does make the system-designer responsible for those accidents in some way, but not in a way that maps neatly on to murder, manslaughter, and nothing.
It seems like AGIs are more likely to operate on a system-sized viewpoint than a human-sized viewpoint. It’s not enough to tell Google “don’t be evil” and trust that their inborn human morality will correct translate “evil.” What does it mean for an institution the size and shape of Google to be evil? They need to make many tradeoffs that people normally do not have to consider, and thus may not have good intuitions for.
“[Y]ou notice that [a proposed action] has some harmful effect in the future on someone you consider morally important. You then end up not being able to do anything …”
Not being able to do that thing, yes, and you shouldn’t do it—unless you can obviate the harm. A case in point is the AGI taking over management of all commodity production and thus putting the current producers out of work. But how is that harmful to them? They can still perform the acts if they wish. They can’t earn a living, you say? Well, then, let the AGI support them. Ah, but then, you suppose, they can’t enjoy the personal worth that meaningful employment reinforces. The what? Let’s stick to the material, please.
“You then end up not being able to do nothing—because doing nothing also leads to harm in the future.”
That does not follow. Doing nothing is always an option under the brass rule. Morally you are not the cause of any harm that then occurred, if any.
Commission vs. omission [of causative actions]: Omitting an action may indeed allow an entity to come to harm, but this is not a moral issue unless acting would harm that entity or another, perhaps to a lesser degree. Commission—taking action—is the problematic case. I repeat: a coded moral system should be restrictive, not promotional. Preventing external harm may be desirable and admirable but is never morally imperative, however physically imperative it may be.
“[Not adding a safety feature] does make the system-designer responsible for [the resulting] accidents in some way …”
Only by the “Good Samaritan” moral code, in which this society is so dolefully steeped. I prefer Caveat emptor. It may be that when AGIs are the principal operators of harmful equipment, the obsession with safety will moderate.
Not being able to do that thing, yes, and you shouldn’t do it—unless you can obviate the harm.
The relevant scenario is one in which all possible actions lead to some harm somewhere. Suppose the AGI designed to cure cancer uses electrical power to run a molecular simulation; then it’s causing someone to die due respiratory illness from inhaling coal dust, or to die from falling off a ladder installing a solar panel, or to die in a mine, or so on. Suppose it doesn’t; then people are dying due to cancer, and it’s abandoning the duty it cares deeply about.
Typically, this problem gets solved by either not thinking about it, rounding small numbers to 0, or by taxes. Consequentialist restrictive moralities operate by “taxes”—if you want to use that electricity that someone died for, you need to need to be using it for something good enough to offset that cost.
For example, coal costs about a hundred lives per TWh; American per capita power consumption is about 1.7kW, and so we’re looking at about one death every 670 person-years of energy consumption. It’s a small number, but there are clear problems with rounding it down to zero: if we do, half of zero is still zero, and there’s no impetus to reduce the pollution or switch to something cleaner. And not thinking about it is even more dangerous!
Only by the “Good Samaritan” moral code, in which this society is so dolefully steeped. I prefer Caveat emptor.
Do you think it is sensible to leave sharp knives around unattended infants? If so, yikes, and if not, why not?
Clearly the infant’s choices led to it cutting itself, but we wouldn’t call that the infant’s informed consent, because we don’t think infants can provide informed consent, because we don’t think they can be informed of the consequences of their actions. Most libertarian reasoning assumes that we are not dealing with infants, but instead with “responsible adults,” who can reason about their situations and make choices and “deserve what they get.”
But when we’re designing systems, there’s a huge information asymmetry, and the system designer often picks what information the user is looking at. To replace the field of human factors research with caveat emptor is profoundly anti-life and anti-efficiency.
And note that we haven’t touched at all on the Good Samaritan issue—there’s a different underlying relationship than the bystander-victim relationship, or the merchant-merchant relationship. The designer-user relationship is categorically different, and we can strongly endorse moral obligations there without also endorsing them in the other two scenarios. (Hayek’s information cost seems relevant.)
(The relevance is that most AI-human interactions will be closer to designer-user or parent-infant than merchant-merchant, and what constitutes trickery and harm might look very different.)
Suppose you had perfect omniscience. (I’m not saying an AI would, I’m just setting up a hypothetical.) It might be the case that whenever you consider doing something, you notice that it has some harmful effect in the future on someone you consider morally important. You then end up not being able to do anything, including not being able to do nothing- because doing nothing also leads to harm in the future. So we can’t just ban all harm; we need to somehow proportionally penalize harm, so that it’s better to do less harm than more harm. But there are good things that are worth purchasing with harm, and so then we’re back into tradeoff territory and maximizing profit instead of just minimizing cost.
(Indeed, the function of morality seems to mostly be to internalize externalities, rather than simply minimize negative externalities. Rules like “do no harm” serve for this purpose by making you consider harm to others before you act, which hopefully prevents you from doing things that are net negative while still allowing you to do things that are net positive.)
Humans have some idea of commission and omission: consider the difference between me running my car into you, you walking into the path of my car, and me not grabbing you to prevent you from walking into the path of a car. The first would be murder, the second possibly manslaughter and possibly not, and the third is not a crime. But that’s a human-sized sense of commission and omission. It’s not at all clear that AGIs will operate on the same scale.
When one takes a system-sized viewpoint, commission and omission become very different. The choice to not add a safety feature that makes accidents less likely does make the system-designer responsible for those accidents in some way, but not in a way that maps neatly on to murder, manslaughter, and nothing.
It seems like AGIs are more likely to operate on a system-sized viewpoint than a human-sized viewpoint. It’s not enough to tell Google “don’t be evil” and trust that their inborn human morality will correct translate “evil.” What does it mean for an institution the size and shape of Google to be evil? They need to make many tradeoffs that people normally do not have to consider, and thus may not have good intuitions for.
“[Y]ou notice that [a proposed action] has some harmful effect in the future on someone you consider morally important. You then end up not being able to do anything …”
Not being able to do that thing, yes, and you shouldn’t do it—unless you can obviate the harm. A case in point is the AGI taking over management of all commodity production and thus putting the current producers out of work. But how is that harmful to them? They can still perform the acts if they wish. They can’t earn a living, you say? Well, then, let the AGI support them. Ah, but then, you suppose, they can’t enjoy the personal worth that meaningful employment reinforces. The what? Let’s stick to the material, please.
“You then end up not being able to do nothing—because doing nothing also leads to harm in the future.”
That does not follow. Doing nothing is always an option under the brass rule. Morally you are not the cause of any harm that then occurred, if any.
Commission vs. omission [of causative actions]: Omitting an action may indeed allow an entity to come to harm, but this is not a moral issue unless acting would harm that entity or another, perhaps to a lesser degree. Commission—taking action—is the problematic case. I repeat: a coded moral system should be restrictive, not promotional. Preventing external harm may be desirable and admirable but is never morally imperative, however physically imperative it may be.
“[Not adding a safety feature] does make the system-designer responsible for [the resulting] accidents in some way …”
Only by the “Good Samaritan” moral code, in which this society is so dolefully steeped. I prefer Caveat emptor. It may be that when AGIs are the principal operators of harmful equipment, the obsession with safety will moderate.
The relevant scenario is one in which all possible actions lead to some harm somewhere. Suppose the AGI designed to cure cancer uses electrical power to run a molecular simulation; then it’s causing someone to die due respiratory illness from inhaling coal dust, or to die from falling off a ladder installing a solar panel, or to die in a mine, or so on. Suppose it doesn’t; then people are dying due to cancer, and it’s abandoning the duty it cares deeply about.
Typically, this problem gets solved by either not thinking about it, rounding small numbers to 0, or by taxes. Consequentialist restrictive moralities operate by “taxes”—if you want to use that electricity that someone died for, you need to need to be using it for something good enough to offset that cost.
For example, coal costs about a hundred lives per TWh; American per capita power consumption is about 1.7kW, and so we’re looking at about one death every 670 person-years of energy consumption. It’s a small number, but there are clear problems with rounding it down to zero: if we do, half of zero is still zero, and there’s no impetus to reduce the pollution or switch to something cleaner. And not thinking about it is even more dangerous!
Do you think it is sensible to leave sharp knives around unattended infants? If so, yikes, and if not, why not?
Clearly the infant’s choices led to it cutting itself, but we wouldn’t call that the infant’s informed consent, because we don’t think infants can provide informed consent, because we don’t think they can be informed of the consequences of their actions. Most libertarian reasoning assumes that we are not dealing with infants, but instead with “responsible adults,” who can reason about their situations and make choices and “deserve what they get.”
But when we’re designing systems, there’s a huge information asymmetry, and the system designer often picks what information the user is looking at. To replace the field of human factors research with caveat emptor is profoundly anti-life and anti-efficiency.
And note that we haven’t touched at all on the Good Samaritan issue—there’s a different underlying relationship than the bystander-victim relationship, or the merchant-merchant relationship. The designer-user relationship is categorically different, and we can strongly endorse moral obligations there without also endorsing them in the other two scenarios. (Hayek’s information cost seems relevant.)
(The relevance is that most AI-human interactions will be closer to designer-user or parent-infant than merchant-merchant, and what constitutes trickery and harm might look very different.)