When modeling an agent that acts in a world <@that contains it@>(@@), there are different ways that we could represent what a “hypothesis about the world” should look like. (We’ll use <@infra-Bayesianism@>(@Infra-Bayesianism sequence@) to allow us to have hypotheses over environments that are “bigger” than the agent, in the sense of containing the agent.) In particular, hypotheses can vary along two axes:
1. **First-person vs. third-person:** In a first-person perspective, the agent is central. In a third-person perspective, we take a “birds-eye” view of the world, of which the agent is just one part.
2. **Static vs. dynamic:** In a dynamic perspective, the notion of time is explicitly present in the formalism. In a static perspective, we instead have beliefs directly about entire world-histories.
To get a tiny bit more concrete, let the world have states S and the agent have actions A and observations O. The agent can implement policies Π. I will use ΔX to denote a belief over X (this is a bit handwavy, but gets the right intuition, I think). Then the four views are:
1. First-person static: A hypothesis specifies how policies map to beliefs over observation-action sequences, that is, Π → Δ(O × A)*.
2. First-person dynamic: This is the typical POMDP framework, in which a hypothesis is a belief over initial states and transition dynamics, that is, ΔS and S × A → Δ(O × S).
3. Third-person static: A hypothesis specifies a belief over world histories, that is, Δ(S*).
4. Third-person dynamic: A hypothesis specifies a belief over initial states, and over the transition dynamics, that is, we have ΔS and S → ΔS. Notice that despite having “transitions”, actions do not play a role here.
Given a single “reality”, it is possible to move between these different views on reality, though in some cases this requires making assumptions on the starting view. For example, under regular Bayesianism, you can only move from third-person static to third-person dynamic if your belief over world histories Δ(S*) satisfies the Markov condition (future states are conditionally independent of past states given the present state); if you want to make this move even when the Markov condition isn’t satisfied, you have to expand your belief over initial states to be a belief over “initial” world histories.
You can then define various flavors of (a)causal influence by saying which types of states S you allow:
1. If a state s consists of a policy π and an world history (oa)* that is consistent with π, then the environment transitions can depend on your choice of π, leading to acausal influence. This is the sort of thing that would be needed to formalize Newcomb’s problem.
2. In contrast, if a state s consists only of an environment E that responds to actions but _doesn’t_ get to see the full policy, then the environment cannot depend on your policy, and there is only causal influence. You’re implicitly claiming that Newcomb’s problem cannot happen.
3. Finally, rather than have an environment E that (when combined with a policy π) generates a world history (oa*), you could have the state s directly be the world history (oa)*, _without_ including the policy π. This still precludes acausal influence. In normal Bayesianism, this would be equivalent to the previous case (since we could construct a belief over E that implies the given belief over (oa)*), but in the case of infra-Bayesianism it is not, for reasons I won’t go into. (Roughly speaking, the differences occur when you use a “belief” that isn’t just a claim about reality, but also a claim about which parts of reality you “care about”.) Since the existence of E isn’t required, but we do still preclude policy-dependent influence, the authors call this setup “pseudocausal”.
In all three versions, you can define translations between the four different views, such that following any path of translations will always give you the same final output (that is, translating from A to B to C has the same result as A to D to C). This property can be used to _define_ “acausal”, “causal”, and “pseudocausal” as applied to belief functions in infra-Bayesianism. (I’m not going to talk about what a belief function is; see the post for details.)
I’d say this is mostly accurate, but I’d amend number 3. There’s still a sort of non-causal influence going on in pseudocausal problems, you can easily formalize counterfactual mugging and XOR blackmail as pseudocausal problems (you need acausal specifically for transparent newcomb, not vanilla newcomb). But it’s specifically a sort of influence that’s like “reality will adjust itself so contradictions don’t happen, and there may be correlations between what happened in the past, or other branches, and what your action is now, so you can exploit this by acting to make bad outcomes inconsistent”. It’s purely action-based, in a way that manages to capture some but not all weird decision-theoretic scenarios.
In normal bayesianism, you do not have a pseudocausal-causal equivalence. Every ordinary environment is straight-up causal.
Finally, rather than have an environment E that (when combined with a policy π) generates a world history (oa)*, you could have the state s directly be the world history (oa)*, _without_ including the policy π. In normal Bayesianism, using (oa)* as states would be equivalent to using environments E as states (since we could construct a belief over E that implies the given belief over (oa)*), but in the case of infra-Bayesianism it is not. (Roughly speaking, the differences occur when you use a “belief” that isn’t just a claim about reality, but also a claim about which parts of reality you “care about”.) This ends up allowing some but not all flavors of acausal influence, and so the authors call this setup “pseudocausal”.
Re:
In normal bayesianism, you do not have a pseudocausal-causal equivalence. Every ordinary environment is straight-up causal.
What I meant was that if you define a Bayesian belief over world-histories (oa)*, that is equivalent to having a Bayesian belief over environments E, which I think you agree with. I’ve edited slightly to make this clearer.
Re: the dispute over normal bayesianism: For me, “environment” denotes “thingy that can freely interact with any policy in order to produce a probability distribution over histories”. This is a different type signature than a probability distribution over histories, which doesn’t have a degree of freedom corresponding to which policy you pick.
But for infra-bayes, we can associate a classical environment with the set of probability distributions over histories (for various possible choices of policy), and then the two distinct notions become the same sort of thing (set of probability distributions over histories, some of which can be made to be inconsistent by how you act), so you can compare them.
Planned summary for the Alignment Newsletter:
I’d say this is mostly accurate, but I’d amend number 3. There’s still a sort of non-causal influence going on in pseudocausal problems, you can easily formalize counterfactual mugging and XOR blackmail as pseudocausal problems (you need acausal specifically for transparent newcomb, not vanilla newcomb). But it’s specifically a sort of influence that’s like “reality will adjust itself so contradictions don’t happen, and there may be correlations between what happened in the past, or other branches, and what your action is now, so you can exploit this by acting to make bad outcomes inconsistent”. It’s purely action-based, in a way that manages to capture some but not all weird decision-theoretic scenarios.
In normal bayesianism, you do not have a pseudocausal-causal equivalence. Every ordinary environment is straight-up causal.
Thanks for checking! I’ve changed point 3 to:
Re:
What I meant was that if you define a Bayesian belief over world-histories (oa)*, that is equivalent to having a Bayesian belief over environments E, which I think you agree with. I’ve edited slightly to make this clearer.
Looks good.
Re: the dispute over normal bayesianism: For me, “environment” denotes “thingy that can freely interact with any policy in order to produce a probability distribution over histories”. This is a different type signature than a probability distribution over histories, which doesn’t have a degree of freedom corresponding to which policy you pick.
But for infra-bayes, we can associate a classical environment with the set of probability distributions over histories (for various possible choices of policy), and then the two distinct notions become the same sort of thing (set of probability distributions over histories, some of which can be made to be inconsistent by how you act), so you can compare them.
Ah right, that makes sense. That was a mistake on my part, my bad.