I agree with both of Daniel Kokotajlo’s points (both of which we also make in the paper in Sections IV.1 and IV.2): Certainly for humans it’s normal to not be able to randomize; and even if it was a primarily hypothetical situation without any obvious practical application, I’d still be interested in knowing how to deal with the absence of the ability to randomize.
Besides, as noted in my other comment insisting on the ability to randomize doesn’t get you that far (cf. Sections IV.1 and IV.4 on Ratificationism): even if you always have access to some nuclear decay noise channel, your choice of whether to consult that channel (or of whether to factor the noise into your decision) is still deterministic. So you can set up scenarios where if you are punished for randomizing. In the particular case of the Adversarial Offer, the seller might remove all money from both boxes if she predicts the buyer to randomize.
The reason why our main scenario just assumes that randomization isn’t possible is that our target of attack in this paper is primarily CDT, which is fine with not being allowed to randomize.
Care to elaborate on the every day thing? Aside from literal coins, your cell phone is perfectly capable of generating pseudorandom numbers, and I’m almost never without mine.
I guess whether your point stands depends on whether we are more concerned with abstract theory or practical decision making.
Here are some circumstances where you don’t have access to an unpredictable random number generator:
--You need to make a decision very quickly and so don’t have time to flip a coin
--Someone is watching you and will behave differently towards you if they see you make the decision via randomness, so consulting a coin isn’t a random choice between options but rather an additional option with its own set of payoffs
--Someone is logically entangled with you and if you randomize they will no longer be.
--You happen to be up against someone who is way smarter than you and can predict your coin / RNG / etc.
Admittedly, while in some sense these things happen literally every day to all of us, they typically don’t happen for important decisions.
But there are important decisions having to do with acausal trade that fit into this category, that either we our our AI successors will face one day.
And even if that wasn’t true, decision theory is decision THEORY. If one theory outperforms another in some class of cases, that’s a point in its favor, even if the class of cases is unusual.
EDIT: See Paul Christiano’s example below, it’s an excellent example because it takes Caspar’s paper and condenses it into a very down-to-earth, probably-has-actually-happened-to-someone-already example.
I’ve picked up my game theory entirely informally. But in real world terms, perhaps we’re imagining a situation where a randomization approach isn’t feasible for some other reason than a random number generator being unavailable.
This connects slightly with the debate over whether or not to administer untested COVID vaccine en masse. To pick randomly “feels scary” compared to picking “for a reason,” but to pick “for a reason” when there isn’t an actual evidence basis yet undermines the authority of regulators, so regulators don’t pick anything until they have a “good reason” to do so. Their political calculus, in short, makes them unable to use a randomization scheme.
So in terms of real world applicability, the constraint on a non-randomizing strategy seems potentially relevant, although the other aspects of this puzzle don’t map onto COVID vaccine selection specifically.
How often do you encounter a situation where an unpredictable randomization mechanism is unavailable?
I agree with both of Daniel Kokotajlo’s points (both of which we also make in the paper in Sections IV.1 and IV.2): Certainly for humans it’s normal to not be able to randomize; and even if it was a primarily hypothetical situation without any obvious practical application, I’d still be interested in knowing how to deal with the absence of the ability to randomize.
Besides, as noted in my other comment insisting on the ability to randomize doesn’t get you that far (cf. Sections IV.1 and IV.4 on Ratificationism): even if you always have access to some nuclear decay noise channel, your choice of whether to consult that channel (or of whether to factor the noise into your decision) is still deterministic. So you can set up scenarios where if you are punished for randomizing. In the particular case of the Adversarial Offer, the seller might remove all money from both boxes if she predicts the buyer to randomize.
The reason why our main scenario just assumes that randomization isn’t possible is that our target of attack in this paper is primarily CDT, which is fine with not being allowed to randomize.
Every day. But even if it was only something that happened in weird hypotheticals, my point would still stand.
Care to elaborate on the every day thing? Aside from literal coins, your cell phone is perfectly capable of generating pseudorandom numbers, and I’m almost never without mine.
I guess whether your point stands depends on whether we are more concerned with abstract theory or practical decision making.
Here are some circumstances where you don’t have access to an unpredictable random number generator:
--You need to make a decision very quickly and so don’t have time to flip a coin
--Someone is watching you and will behave differently towards you if they see you make the decision via randomness, so consulting a coin isn’t a random choice between options but rather an additional option with its own set of payoffs
--Someone is logically entangled with you and if you randomize they will no longer be.
--You happen to be up against someone who is way smarter than you and can predict your coin / RNG / etc.
Admittedly, while in some sense these things happen literally every day to all of us, they typically don’t happen for important decisions.
But there are important decisions having to do with acausal trade that fit into this category, that either we our our AI successors will face one day.
And even if that wasn’t true, decision theory is decision THEORY. If one theory outperforms another in some class of cases, that’s a point in its favor, even if the class of cases is unusual.
EDIT: See Paul Christiano’s example below, it’s an excellent example because it takes Caspar’s paper and condenses it into a very down-to-earth, probably-has-actually-happened-to-someone-already example.
I’ve picked up my game theory entirely informally. But in real world terms, perhaps we’re imagining a situation where a randomization approach isn’t feasible for some other reason than a random number generator being unavailable.
This connects slightly with the debate over whether or not to administer untested COVID vaccine en masse. To pick randomly “feels scary” compared to picking “for a reason,” but to pick “for a reason” when there isn’t an actual evidence basis yet undermines the authority of regulators, so regulators don’t pick anything until they have a “good reason” to do so. Their political calculus, in short, makes them unable to use a randomization scheme.
So in terms of real world applicability, the constraint on a non-randomizing strategy seems potentially relevant, although the other aspects of this puzzle don’t map onto COVID vaccine selection specifically.