Why is it a problem? Taboo “blackmail” and figure out what distinction you’re really trying to identify. I suspect it’s just violation of norms. Likely with added spice of confusion that there is some moral difference between action and inaction.
I think it goes beyond violation of norms. It has to do with the sum over the entire interaction between the two parties, as opposed to a single tree-branch of that interaction. In one case, you are in a bad situation, and then someone comes along, and offers to relieve it for a price. In the other case, you are in an okay situation and then someone comes along and puts you into a bad situation, then offers to relieve it.
This can also be expressed in terms of your regret of the other party’s presence in your life. Would you regret having ever met the trader who sells you something you greatly need? No. You’re better off for that person having been around. Would you regret having ever met the extortionist who puts you into a bad situation and then sells you relief? Yes. You’d be better off if they had never been there.
It matters that the same person designed the situation, causing the disutility in order to be able to offer to relieve it. Why? Because an extortionist has to optimize for creating disutility. They have to create problems that otherwise wouldn’t be there. They have to make the world worse; otherwise they wouldn’t be able to offer their own restraint as a “service”.
Tolerance of extortion allows the survival of agents who go around dumping negative utility on people.
Contrast the insurer with the protection racket. The insurer doesn’t create the threat of fire. They may warn you about it, vividly describe to you how much better off you’d be with insurance if your house burns down. But the protection racket has to actually set some people’s stuff on fire, or at least develop the credible ability to do so, in order to be effective at extracting protection money.
Confused as well by insurers who are protection rackets. The Mafia did provide some pluses as well as costs, and governments with emergency services will lock you up if you fail to pay the taxes that support them.
Tolerance of extortion allows the survival of agents who go around dumping negative utility on people.
This is backwards. Tolerance of dumping negative utility allows extortion. The problem is the ability/willingness to cause harm, not the monetization.
Unpack further, please. Are you trying to understand legal or colloquial implications of the words in the English language, or are you more concerned with clusterings of activities for some additional shared attributes?
As I understand it, the idea is that we want to design an AI that is difficult or impossible to blackmail, but which makes a good trading partner.
In other words there are a cluster of behaviors that we do NOT want our AI to have, which seem blackmailish to us, and a cluster of behaviors that we DO want it to have, which seem tradeish to us. So we are now trying to draw a line in conceptual space between them so that we can figure out how to program an AI appropriately.
As I understand it, the idea is that we want to design an AI that is difficult or impossible to blackmail, but which makes a good trading partner.
You and Stuart seem to have different goals. You want to understand and prevent some behaviors (in which case, start by tabooing culturally-dense words like “blackmail”). He wants to understand linguistic or legal definitions (so tabooing the word is counterproductive).
“You want to understand and prevent some behaviors (in which case, start by tabooing culturally-dense words like “blackmail”)”
In a sense, that’s exactly what Stuart was doing all along. The whole point of this post was to come up with a rigorous definition of blackmail, i.e. to find a way to say what we wanted to say without using the word.
Now I’m really confused. Are you trying to define words, or trying to understand (and manipulate) behaviors? I’m hearing you say something like “I don’t know what blackmail is, but I want to make sure an AI doesn’t do it”. This must be a misunderstanding on my part.
I guess you might be trying to understand WHY some people don’t like blackmail, so you can decide whether you want to to guard against it, but even that seems pretty backward.
You make it sound like those two things are mutually exclusive. They aren’t. We are trying to define words so that we can understand and manipulate behavior.
“I don’t know what blackmail is, but I want to make sure an AI doesn’t do it.” Yes, exactly, as long as you interpret it in the way I explained it above.* What’s wrong with that? Isn’t that exactly what the AI safety project is, in general? “I don’t know what bad behaviors are, but I want to make sure the AI doesn’t do them.”
*”In other words there are a cluster of behaviors that we do NOT want our AI to have, which seem blackmailish to us, and a cluster of behaviors that we DO want it to have, which seem tradeish to us. So we are now trying to draw a line in conceptual space between them so that we can figure out how to program an AI appropriately.”
It’s a badly formulated question, likely to lead to confusion.
there are a cluster of behaviors that we do NOT want our AI to have, which seem blackmailish to us
So, can you specify what this cluster is? Can you list the criteria by which a behaviour would be included in or excluded from this cluster? If you do this, you have defined blackmail.
“It’s a badly formulated question, likely to lead to confusion.” Why? That’s precisely what I’m denying.
“So, can you specify what this cluster is? Can you list the criteria by which a behaviour would be included in or excluded from this cluster? If you do this, you have defined blackmail.”
That’s precisely what I (Stuart really) am trying to do! I said so, you even quoted me saying so, and as I interpret him, Stuart said so too in the OP. I don’t care about the word blackmail except as a means to an end; I’m trying to come up with criteria by which to separate the bad behaviors from the good.
I’m honestly baffled at this whole conversation. What Stuart is doing seems the opposite of confused to me.
To avoid reinventing the wheel, I suggest looking at legal definitions of blackmail, as well as reading a couple of law research articles on the topic. Lawyers and judges had to deal with this issue for a very long time and they need to have criteria which produce a definite answer.
I think there’s also an element that if you know how to identify and resist blackmail yourself, not only do you win more, but also it’s easier for you to choose to not blackmail others.
Why is it a problem? Taboo “blackmail” and figure out what distinction you’re really trying to identify. I suspect it’s just violation of norms. Likely with added spice of confusion that there is some moral difference between action and inaction.
I think it goes beyond violation of norms. It has to do with the sum over the entire interaction between the two parties, as opposed to a single tree-branch of that interaction. In one case, you are in a bad situation, and then someone comes along, and offers to relieve it for a price. In the other case, you are in an okay situation and then someone comes along and puts you into a bad situation, then offers to relieve it.
This can also be expressed in terms of your regret of the other party’s presence in your life. Would you regret having ever met the trader who sells you something you greatly need? No. You’re better off for that person having been around. Would you regret having ever met the extortionist who puts you into a bad situation and then sells you relief? Yes. You’d be better off if they had never been there.
It matters that the same person designed the situation, causing the disutility in order to be able to offer to relieve it. Why? Because an extortionist has to optimize for creating disutility. They have to create problems that otherwise wouldn’t be there. They have to make the world worse; otherwise they wouldn’t be able to offer their own restraint as a “service”.
Tolerance of extortion allows the survival of agents who go around dumping negative utility on people.
Contrast the insurer with the protection racket. The insurer doesn’t create the threat of fire. They may warn you about it, vividly describe to you how much better off you’d be with insurance if your house burns down. But the protection racket has to actually set some people’s stuff on fire, or at least develop the credible ability to do so, in order to be effective at extracting protection money.
Confused as well by insurers who are protection rackets. The Mafia did provide some pluses as well as costs, and governments with emergency services will lock you up if you fail to pay the taxes that support them.
This is backwards. Tolerance of dumping negative utility allows extortion. The problem is the ability/willingness to cause harm, not the monetization.
I’m trying to identify the distinction between trade and blackmail ^_^
Unpack further, please. Are you trying to understand legal or colloquial implications of the words in the English language, or are you more concerned with clusterings of activities for some additional shared attributes?
As I understand it, the idea is that we want to design an AI that is difficult or impossible to blackmail, but which makes a good trading partner.
In other words there are a cluster of behaviors that we do NOT want our AI to have, which seem blackmailish to us, and a cluster of behaviors that we DO want it to have, which seem tradeish to us. So we are now trying to draw a line in conceptual space between them so that we can figure out how to program an AI appropriately.
You and Stuart seem to have different goals. You want to understand and prevent some behaviors (in which case, start by tabooing culturally-dense words like “blackmail”). He wants to understand linguistic or legal definitions (so tabooing the word is counterproductive).
? No, I have the same goals as kokotajlod.
“You want to understand and prevent some behaviors (in which case, start by tabooing culturally-dense words like “blackmail”)”
In a sense, that’s exactly what Stuart was doing all along. The whole point of this post was to come up with a rigorous definition of blackmail, i.e. to find a way to say what we wanted to say without using the word.
Now I’m really confused. Are you trying to define words, or trying to understand (and manipulate) behaviors? I’m hearing you say something like “I don’t know what blackmail is, but I want to make sure an AI doesn’t do it”. This must be a misunderstanding on my part.
I guess you might be trying to understand WHY some people don’t like blackmail, so you can decide whether you want to to guard against it, but even that seems pretty backward.
You make it sound like those two things are mutually exclusive. They aren’t. We are trying to define words so that we can understand and manipulate behavior.
“I don’t know what blackmail is, but I want to make sure an AI doesn’t do it.” Yes, exactly, as long as you interpret it in the way I explained it above.* What’s wrong with that? Isn’t that exactly what the AI safety project is, in general? “I don’t know what bad behaviors are, but I want to make sure the AI doesn’t do them.”
*”In other words there are a cluster of behaviors that we do NOT want our AI to have, which seem blackmailish to us, and a cluster of behaviors that we DO want it to have, which seem tradeish to us. So we are now trying to draw a line in conceptual space between them so that we can figure out how to program an AI appropriately.”
It’s a badly formulated question, likely to lead to confusion.
So, can you specify what this cluster is? Can you list the criteria by which a behaviour would be included in or excluded from this cluster? If you do this, you have defined blackmail.
“It’s a badly formulated question, likely to lead to confusion.” Why? That’s precisely what I’m denying.
“So, can you specify what this cluster is? Can you list the criteria by which a behaviour would be included in or excluded from this cluster? If you do this, you have defined blackmail.”
That’s precisely what I (Stuart really) am trying to do! I said so, you even quoted me saying so, and as I interpret him, Stuart said so too in the OP. I don’t care about the word blackmail except as a means to an end; I’m trying to come up with criteria by which to separate the bad behaviors from the good.
I’m honestly baffled at this whole conversation. What Stuart is doing seems the opposite of confused to me.
To avoid reinventing the wheel, I suggest looking at legal definitions of blackmail, as well as reading a couple of law research articles on the topic. Lawyers and judges had to deal with this issue for a very long time and they need to have criteria which produce a definite answer.
I think there’s also an element that if you know how to identify and resist blackmail yourself, not only do you win more, but also it’s easier for you to choose to not blackmail others.