Nice post! I found the diagrams particularly readable, it makes a lot of sense to me to have them in such a problem.
I’m not very well-read on this sort of work, so feel free to ignore any of the following.
The key question I have is the correctness of the section:
In a sense, ACDT can be seen as anterior to CDT. How do we know that causality exists, and the rules it runs on? From our experience in the world. If we lived in a world where the Newcomb problem or the predictors exist problem were commonplace, then we’d have a different view of causality.
It might seem gratuitous and wrong to draw extra links coming out of your decision node—but it was also gratuitous and wrong to cut all the links that go into your decision node. Drawing these extra arrows undoes some of the damage, in a way that a CDT agent can understand (they don’t understand things that cause their actions, but they do understand consequences of their actions).
I don’t quite see why the causality is this flexible and arbitrary. I haven’t read Causality, but think I get the gist.
It’s definitely convenient here to be uncertain about causality. But it would be similarly convenient to have uncertainty about the correct decision theory. A similar formulation could involve a meta-decision-algorithm that has tries different decision algorithms until one produces favorable outcomes. Personally I think I’d be easier to be convinced that acausal decision theory is correct than that a different causal structure is correct.
Semi-related, one aspect of Newcomb’s problem that has really confused me is the potential for Omega to have scenarios that favor incorrect beliefs. It would be arbitrary to imagine that Newcomb would offer $1,000 only if it could tell that one believes that “19 + 2 = 20”. One could solve that by imagining that the participant should have uncertainty about what “19 + 2″ is, trying out multiple options, and seeing which would produce the most favorable outcome.
Separately,
If it’s encountered the Newcomb problem before, and tried to one-box and two-box a few times, then it knows that the second graph gives more accurate predictions
To be clear, I’d assume that the agent would be smart enough to simulate this before actually having it done? The outcome seems decently apparent to me.
I don’t quite see why the causality is this flexible and arbitrary.
In stories and movies, people often find that the key tool/skill/knowledge they need to solve the problem, is something minor they picked up some time before.
The world could work like this, so that every minor thing you spent any time on would have a payoff at some point in the future. Call this a teleological world.
This world would have a different “causal” structure to our own, and we’d probably not conceive traditional CDT agents as likely in this world.
But it would be similarly convenient to have uncertainty about the correct decision theory.
Yes, this is really interesting for me. For example, if I have the Newcomb-like problem, but uncertain about the decision theory, I should one box, as in that case my expected payoff is higher (if I give equal probability to both outcomes of the Newcomb experiment.)
Nice post! I found the diagrams particularly readable, it makes a lot of sense to me to have them in such a problem.
I’m not very well-read on this sort of work, so feel free to ignore any of the following.
The key question I have is the correctness of the section:
I don’t quite see why the causality is this flexible and arbitrary. I haven’t read Causality, but think I get the gist.
It’s definitely convenient here to be uncertain about causality. But it would be similarly convenient to have uncertainty about the correct decision theory. A similar formulation could involve a meta-decision-algorithm that has tries different decision algorithms until one produces favorable outcomes. Personally I think I’d be easier to be convinced that acausal decision theory is correct than that a different causal structure is correct.
Semi-related, one aspect of Newcomb’s problem that has really confused me is the potential for Omega to have scenarios that favor incorrect beliefs. It would be arbitrary to imagine that Newcomb would offer $1,000 only if it could tell that one believes that “19 + 2 = 20”. One could solve that by imagining that the participant should have uncertainty about what “19 + 2″ is, trying out multiple options, and seeing which would produce the most favorable outcome.
Separately,
To be clear, I’d assume that the agent would be smart enough to simulate this before actually having it done? The outcome seems decently apparent to me.
In stories and movies, people often find that the key tool/skill/knowledge they need to solve the problem, is something minor they picked up some time before.
The world could work like this, so that every minor thing you spent any time on would have a payoff at some point in the future. Call this a teleological world.
This world would have a different “causal” structure to our own, and we’d probably not conceive traditional CDT agents as likely in this world.
Yes, this is really interesting for me. For example, if I have the Newcomb-like problem, but uncertain about the decision theory, I should one box, as in that case my expected payoff is higher (if I give equal probability to both outcomes of the Newcomb experiment.)