I would have made it with Eliezer, who has a consequentialist morality but, on account of the consequences, has said he would not break an oath even for the sake of saving the world.
Is there a link to an online explanation of this? When are the consequences of breaking an oath worse than a destroyed world? What did “world” mean when he said it? Humans? Earth? Humans on Earth? Energy in the Multiverse?
But I only trust Alicorn and Eliezer because I’ve discussed morality with both of them in a situation where they had no incentive to lie
Given that you are still alive, posting, and connected within SIAI and LessWrong; and that they both probably expected that at the time; I don’t think any such situation is possible. I think you’re giving them shockingly little credit as rationalists, especially if you’ve read either Luminosity or Harry Potter and the Methods of Rationality.
Eliezer has proven, empirically, that his reputation is worth at least an amount on the order of the funding that SIAI has received to date—let’s say half of it. Every time he opens his mouth, he thus has an incentive of at least several hundred thousand dollars not to damage that reputation. If he did already state that he would not break an oath to save the world, and you offered him a million dollars to go on record as saying that he would lie or break an oath for pragmatic reasons, I’d be surprised if he took you up on it. (But I’d be up for it, if you have the money...)
I don’t believe that he’s lying. And I don’t believe that he’s telling the truth. I believe Eliezer may be operating on a level of rationality where we actually need to regard him as a rational agent. And he’s playing a game that he expects to go on for many iterations. That implies that, short of fMRI, knowing his intent is impossible.
It might be more rational for him to pretend to be less rational. Therefore, he possibly already has.
Is there a link to an online explanation of this? When are the consequences of breaking an oath worse than a destroyed world? What did “world” mean when he said it? Humans? Earth? Humans on Earth? Energy in the Multiverse?
Suppose someone comes to a rationalist Confessor and says: “You know, tomorrow I’m planning to wipe out the human species using this neat biotech concoction I cooked up in my lab.” What then? Should you break the seal of the confessional to save humanity?
It appears obvious to me that the issues here are just those of the one-shot Prisoner’s Dilemma, and I do not consider it obvious that you should defect on the one-shot PD if the other player cooperates in advance on the expectation that you will cooperate as well.
Thanks! I had read that, but had forgotten about it. Perhaps EY’s position makes more sense within timeless decision theory? Since it seems to be based on an absolute requirement for integrity of pre-commitment.
On the other hand, he did not express disapproval of sinking the ship to stop the German nuclear bomb. What if Haukelid had had to promise not to harm the ship, in order to get access to it?
Is there a link to an online explanation of this? When are the consequences of breaking an oath worse than a destroyed world? What did “world” mean when he said it? Humans? Earth? Humans on Earth? Energy in the Multiverse?
Given that you are still alive, posting, and connected within SIAI and LessWrong; and that they both probably expected that at the time; I don’t think any such situation is possible. I think you’re giving them shockingly little credit as rationalists, especially if you’ve read either Luminosity or Harry Potter and the Methods of Rationality.
Eliezer has proven, empirically, that his reputation is worth at least an amount on the order of the funding that SIAI has received to date—let’s say half of it. Every time he opens his mouth, he thus has an incentive of at least several hundred thousand dollars not to damage that reputation. If he did already state that he would not break an oath to save the world, and you offered him a million dollars to go on record as saying that he would lie or break an oath for pragmatic reasons, I’d be surprised if he took you up on it. (But I’d be up for it, if you have the money...)
I don’t believe that he’s lying. And I don’t believe that he’s telling the truth. I believe Eliezer may be operating on a level of rationality where we actually need to regard him as a rational agent. And he’s playing a game that he expects to go on for many iterations. That implies that, short of fMRI, knowing his intent is impossible.
It might be more rational for him to pretend to be less rational. Therefore, he possibly already has.
Prices or Bindings
Thanks! I had read that, but had forgotten about it. Perhaps EY’s position makes more sense within timeless decision theory? Since it seems to be based on an absolute requirement for integrity of pre-commitment.
On the other hand, he did not express disapproval of sinking the ship to stop the German nuclear bomb. What if Haukelid had had to promise not to harm the ship, in order to get access to it?