How is Orin going to be punished, exactly, if he’s a liar or agent provacateur in the pay of the foreign powers intending to wear down the King?
Well...
The King reached out and grabbed Orin’s shoulder, looking into his eyes with his own, and smiled wide.
My impression from that rather chilling sentence is that, if Orin is wrong, he’s going to be investigated extensively (and at leisure—we don’t have to make an urgent decision any more). If he proves to be a liar, not just honestly mistaken, then a 1000 coin fine is going to be the least of his worries.
That would be breaking the terms of the agreement: Orin pays 1000 coins on the installment plan if he’s wrong, end of story. If the king casually breaks his agreements with informants, well, now he has another problem...
If he’s wrong he pays 1000 coins. If he’s guilty of treason he gets executed (or whatever). Nothing in the story suggests to me that the king is giving some sort of immunity even if Orin was deliberately misleading.
This is correct. Betting, as a policy, helps distinguish between Orin(correct) and Orin(wrong), but is really only useful for eliminating Orin(spy) because it’s a novel method that the King expects spies to yet be unprepared for, and is easily investigated if circumvented.
Imagine, If Orin is wrong and yet mysteriously has all his debts re-paid and shop re-purchased shortly after being punished, some eyebrows would be raised.
Imagine, If Orin is wrong and yet mysteriously has all his debts re-paid and shop re-purchased shortly after being punished, some eyebrows would be raised.
I’m sure they would be, but alas, that will happen long after the kingdom has been conquered after being driven into financial ruin by listening to foreign informants who work for their enemies.
I’m confused. I’m sure it’s not possible to determine for certain whether Orin is guilty of treason, but whyever should it be impossible to know, say, beyond reasonable doubt?
I think ‘treason’ usually only applies to foreigners.
Quite the reverse, I think. E.g., definition 2a in the OED says
high treason or treason proper: Violation by a subject of his allegiance to his sovereign or to the state.
(Definition 1a is “The action of betraying; betrayal of the trust undertaken by or reposed in any one; breach of faith, treacherous action, treachery.” which is very broad and includes 2a as a special case. 1b is the very special case “treason of the clerks == trahison des clercs”.)
but whyever should it be impossible to know, say, beyond reasonable doubt?
How would you do that in practice? Even if you had a way to do that, how would you stop Orin from, say, going back to his patron and never leaving his native country again and enjoying the fruits of a grateful country’s gratitude? There’s just tons and tons of holes to this whole scenario which is why it was a bad idea if it wasn’t intended to be a commentary on existential risk.
Quite the reverse, I think. E.g., definition 2a in the OED says
Yes, my bad. I meant to write ‘only applies to citizens’. As a foreigner, Orin may be chargeable with things like espionage, but ‘treason’ makes little sense.
Search his house for incriminating correspondence. Talk to his bank manager and find out whether he’s had a surprising influx of money recently. (If so, then in court you can ask Orin to explain where it came from; if it was really a bribe from a foreign enemy, he’ll have to make up something that you may be able to refute.) Ask his neighbours whether unexpected people have been seen going in and out of his house. (If so, then maybe their descriptions match up to those of known foreign agents. Or their conversation might have been overheard.) All the same sorts of things you do when investigating any alleged crime. If a sufficient quantity of evidence of malfeasance piles up, you’re done.
In any case, we seem to have shifted from “obviously the king was granting Orin immunity from prosecution for treason” to “of course it might be difficult to convict him of treason if guilty”. Which, yes, it might. (So I should have said: “If he’s found guilty of treason he gets executed (or whatever)”; my apologies for the inexactitude.)
how would you stop Orin from, say, going back to his patron [...]
He’s already in custody.
it was a bad idea if it wasn’t intended to be a commentary on existential risk.
I thought it obviously was intended to be a commentary on existential risk.
[EDITED to add: but I now see that the author has specifically said it wasn’t. Oh well.]
Orin [...] never leaving his native country again [...] As a foreigner, Orin may be chargeable
I’m confused again. In the story, Orin isn’t a foreigner, he’s a citizen of the Kingdom under discussion. At least, there’s every indication he is and none he isn’t that I can see.
[EDITED shortly after posting, to make it clearer that the point of the evidence-gathering would be the aggregate evidence, not that you necessarily look for a single smoking gun, and to clarify the nature of some of the possible evidence.]
That would be breaking the terms of the agreement: Orin pays 1000 coins on the installment plan if he’s wrong, end of story.
Not quite. If Orin turns out to be wrong but had no malicious intent, then yes, he just pays 1000 coins. On the other hand, if Orin deliberately mislead the king, I don’t recall the terms of the agreement including immunity against charges of treason.
How do you prove it in a few days? ‘Oh, no army appeared’ said Orin. ‘My friend must have been wrong or the date was pushed back. Still, as an honest man, I stand by our deal though it beggar me.’
Are you asking me how the plot can play out in a fictional story? :-D
Here’s one possibility—the king’s large and effective network of spies and informants will send the word that the Northern Kingdom executed a disinformation campaign against the king using a fellow named Orin...
Possibility is not good enough. And in any case, my proposed defeater can be implemented by exactly two people: a volunteer and a rich benefactor, and so it is vastly more likely to be undiscovered by spies & informants than an actual attack. The king is unsure his spy network will uncover every attack, so a fortiori, he is very unsure that my proposed scheme would be detected.
Frankly, I don’t see towards which point are you driving. This is a fable about, basically, an exercise in game theory. You don’t like the story? You think it misleads? If you were king you would have behaved differently?
This is a fable about, basically, an exercise in game theory. You don’t like the story? You think it misleads? If you were king you would have behaved differently?
I pointed out, I thought clearly, my problems in my original comment: this is not isomorphic to existential risk (as the author clearly intended it to be) and solves an easier problem badly.
Gwern, I happen to agree with most of what you’ve said, if this were written in regards to x-risks. It is in fact irrelevant to UFAI, but was mostly an exercise in a) practicing writing, and b) working through some intuitions in regards to betting/prediction markets. I wrote it for LW because I assumed it would be enjoyed, but not really learned from (hence Discussion, not Main). A re-write would explore more thoroughly and explicitly the difference between Orin being correct, a spy, or mistaken, and how his bet changes those probabilities.
I suppose it makes an ok-ish example of “people take their money more seriously than their beliefs, and betting helps fix that” Which I think is am important lesson in general.
b) working through some intuitions in regards to betting/prediction markets.
Why, then, did you choose an existential threat issue to base the entire story on when you know how many issues those bring up for prediction markets with regard to incentives and counterparty risk and the difficulty of ‘betting on the apocalypse’? You should have chosen an issue more clearly within prediction markets’ scope.
Well...
My impression from that rather chilling sentence is that, if Orin is wrong, he’s going to be investigated extensively (and at leisure—we don’t have to make an urgent decision any more). If he proves to be a liar, not just honestly mistaken, then a 1000 coin fine is going to be the least of his worries.
That would be breaking the terms of the agreement: Orin pays 1000 coins on the installment plan if he’s wrong, end of story. If the king casually breaks his agreements with informants, well, now he has another problem...
I don’t think that’s right.
If he’s wrong he pays 1000 coins. If he’s guilty of treason he gets executed (or whatever). Nothing in the story suggests to me that the king is giving some sort of immunity even if Orin was deliberately misleading.
This is correct. Betting, as a policy, helps distinguish between Orin(correct) and Orin(wrong), but is really only useful for eliminating Orin(spy) because it’s a novel method that the King expects spies to yet be unprepared for, and is easily investigated if circumvented.
Imagine, If Orin is wrong and yet mysteriously has all his debts re-paid and shop re-purchased shortly after being punished, some eyebrows would be raised.
I’m sure they would be, but alas, that will happen long after the kingdom has been conquered after being driven into financial ruin by listening to foreign informants who work for their enemies.
As I pointed out, that’s not possible to find. Also, I think ‘treason’ usually only applies to foreigners.
I’m confused. I’m sure it’s not possible to determine for certain whether Orin is guilty of treason, but whyever should it be impossible to know, say, beyond reasonable doubt?
Quite the reverse, I think. E.g., definition 2a in the OED says
(Definition 1a is “The action of betraying; betrayal of the trust undertaken by or reposed in any one; breach of faith, treacherous action, treachery.” which is very broad and includes 2a as a special case. 1b is the very special case “treason of the clerks == trahison des clercs”.)
How would you do that in practice? Even if you had a way to do that, how would you stop Orin from, say, going back to his patron and never leaving his native country again and enjoying the fruits of a grateful country’s gratitude? There’s just tons and tons of holes to this whole scenario which is why it was a bad idea if it wasn’t intended to be a commentary on existential risk.
Yes, my bad. I meant to write ‘only applies to citizens’. As a foreigner, Orin may be chargeable with things like espionage, but ‘treason’ makes little sense.
Search his house for incriminating correspondence. Talk to his bank manager and find out whether he’s had a surprising influx of money recently. (If so, then in court you can ask Orin to explain where it came from; if it was really a bribe from a foreign enemy, he’ll have to make up something that you may be able to refute.) Ask his neighbours whether unexpected people have been seen going in and out of his house. (If so, then maybe their descriptions match up to those of known foreign agents. Or their conversation might have been overheard.) All the same sorts of things you do when investigating any alleged crime. If a sufficient quantity of evidence of malfeasance piles up, you’re done.
In any case, we seem to have shifted from “obviously the king was granting Orin immunity from prosecution for treason” to “of course it might be difficult to convict him of treason if guilty”. Which, yes, it might. (So I should have said: “If he’s found guilty of treason he gets executed (or whatever)”; my apologies for the inexactitude.)
He’s already in custody.
I thought it obviously was intended to be a commentary on existential risk.
[EDITED to add: but I now see that the author has specifically said it wasn’t. Oh well.]
I’m confused again. In the story, Orin isn’t a foreigner, he’s a citizen of the Kingdom under discussion. At least, there’s every indication he is and none he isn’t that I can see.
[EDITED shortly after posting, to make it clearer that the point of the evidence-gathering would be the aggregate evidence, not that you necessarily look for a single smoking gun, and to clarify the nature of some of the possible evidence.]
Not quite. If Orin turns out to be wrong but had no malicious intent, then yes, he just pays 1000 coins. On the other hand, if Orin deliberately mislead the king, I don’t recall the terms of the agreement including immunity against charges of treason.
If you can prove malicious intent, then why the need for the bet in the first place...?
You can’t prove it now, you may be able to prove it (or its absence) in a few days. If there is none, Orin just pays his 1000 coins.
The point of the bet was to properly incentivize Orin in the present.
How do you prove it in a few days? ‘Oh, no army appeared’ said Orin. ‘My friend must have been wrong or the date was pushed back. Still, as an honest man, I stand by our deal though it beggar me.’
Are you asking me how the plot can play out in a fictional story? :-D
Here’s one possibility—the king’s large and effective network of spies and informants will send the word that the Northern Kingdom executed a disinformation campaign against the king using a fellow named Orin...
Possibility is not good enough. And in any case, my proposed defeater can be implemented by exactly two people: a volunteer and a rich benefactor, and so it is vastly more likely to be undiscovered by spies & informants than an actual attack. The king is unsure his spy network will uncover every attack, so a fortiori, he is very unsure that my proposed scheme would be detected.
Not good enough for what?
Frankly, I don’t see towards which point are you driving. This is a fable about, basically, an exercise in game theory. You don’t like the story? You think it misleads? If you were king you would have behaved differently?
I pointed out, I thought clearly, my problems in my original comment: this is not isomorphic to existential risk (as the author clearly intended it to be) and solves an easier problem badly.
Maybe you should read it as a fable and not as a blueprint for dealing with the UFAI problem.
Why? As a fable it is boring and irrelevant, and clearly OP did not intend it to be taken the way you suggest taking it.
Gwern, I happen to agree with most of what you’ve said, if this were written in regards to x-risks. It is in fact irrelevant to UFAI, but was mostly an exercise in a) practicing writing, and b) working through some intuitions in regards to betting/prediction markets. I wrote it for LW because I assumed it would be enjoyed, but not really learned from (hence Discussion, not Main). A re-write would explore more thoroughly and explicitly the difference between Orin being correct, a spy, or mistaken, and how his bet changes those probabilities.
I suppose it makes an ok-ish example of “people take their money more seriously than their beliefs, and betting helps fix that” Which I think is am important lesson in general.
Why, then, did you choose an existential threat issue to base the entire story on when you know how many issues those bring up for prediction markets with regard to incentives and counterparty risk and the difficulty of ‘betting on the apocalypse’? You should have chosen an issue more clearly within prediction markets’ scope.