No Good Logical Conditional Probability

[Scott Garrabrant]

Fix a theory $T$ over a language $L$. A coherent probability function is one that satisfies laws of probability theory, each coherent probability function represents a probability distribution on complete logical extensions of $T$.

One of many equivalent definitions of coherence is that $P$ is coherent if $P\left(s_1\right)+P\left(s_2\right)+\dots +P\left(s_k\right)=1$ whenever $T$ can prove that exactly one of $s_1,\dots ,s_k$ is true.

Another very basic desirable property is that $P\left(s\right)=1$ only when $s$ is provable. There have been many proposals of specific coherent probability assignments that all satisfy this basic requirement. Many satisfy stronger requirements that give bounds on how far $P\left(s\right)$ is from 1 when $s$ is not provable.

In this post, I modify the framework slightly to instead talk about conditional probability. Consider a function $P$ which takes in a consistent theory $T$ and a sentence $s$, and outputs a number $P\left(s|T\right)\in [0,1]$, which represents the conditional probability of $s$ given $T$.

We say that $P$ is coherent if:

1. $P\left(s_1|T\right)+P\left(s_2|T\right)+\dots +P\left(s_k|T\right)=1$ whenever $T$ can prove that exactly one of $s_1,\dots ,s_k$ is true, and

2. $P(s\wedge r|T)=P\left(r|T\right)\cdot P(s|T\cup \{r\left\}\right).$

3. If $s$ proves every sentence in $T$, then $P(s|R\cup T)\ge P\left(s|R\right)$.

Theorem: There is no coherent conditional probability function $P$ such that $P\left(s|T\right)=1$ only when $T$ proves $s$.

Proof:

This proof will use the notation of log odds $\ell \left(p\right)={\log }_2\left(\frac{p}{1-p}\right)$ to make things simpler.

Let $P$ be a coherent conditional probability function. Fix a sentence $s$ which is neither provable nor disprovable from the empty theory. Construct an infinite sequences of theories as follows:

1. $T_0$ is the empty theory.

2. To construct $T_{n+1}$, choose a sentence $r_n$ such that neither $s\to r_n$ nor $s\to \neg r_n$ are provable in $T_n$. If $P(s\wedge r_n|T_n)\le P(s\wedge \neg r_n|T_n)$, then let $T_{n+1}=T_n\cup \{s\to r_n\}$. Otherwise, let $T_{n+1}=T_n\cup \{s\to \neg r_n\}$.

Fix an $n$, and without loss of generality, assume $P(s\wedge r_n|T_n)\le P(s\wedge \neg r_n|T_n)$. Since $P$ is coherent we have $P(s\wedge r|T_n)+P(s\wedge \neg r|T_n)=P\left(s|T_n\right).$ In particular, this means that $P(s\wedge r|T_n)\le \frac{1}{2}P\left(s|T_n\right)$.

Observe that $P(s\wedge (s\to r\left)|T_n\right)=P\left(s|T_{n+1}\right)P(s\to r|T_n)$, and $P(\neg s\wedge (s\to r\left)|T_n\right)=P\left(\neg s|T_{n+1}\right)P(s\to r|T_n)$. Therefore, $P(s\wedge r|T_n)/P\left(\neg s|T_n\right)=P\left(s|T_{n+1}\right)/P\left(\neg s|T_{n+1}\right)$, so $\frac{1}{2}P\left(s|T_n\right)/P\left(\neg s|T_n\right)\ge P\left(s|T_{n+1}\right)/P\left(\neg s|T_{n+1}\right)$.

In the language of log odds, this means that $\ell \left(P\right(s|T_n\left)\right)-1\ge \ell \left(P\right(s|T_{n+1}\left)\right)$.

Let $T_{\infty }$ be the union of all the $T_n$. Note that by the third condition of coherence, $\ell \left(P\right(\neg s|T_{\infty }\left)\right)\ge \ell \left(P\right(\neg s|T_n\left)\right)$ for all $n$, so $\ell \left(P\right(s|T_{\infty }\left)\right)\le \ell \left(P\right(s|T_n\left)\right)$ for all $n$

Consider $\ell \left(P\right(s|T_0\left)\right)$ and $\ell \left(P\right(s|T_{\infty }\left)\right)$. These numbers cannot both be finite, since $\ell \left(P\right(s|T_{\infty }\left)\right)\le \ell \left(P\right(s|T_n\left)\right)\le \ell \left(P\right(s|T_0\left)\right)-n$. Therefore, at least one of $P\left(s|T_0\right)$ and $P\left(s|T_{\infty }\right)$ must be 0 or 1. However neither $T_0$ nor $T_{\infty }$ prove or disprove $s$, so this means that $P$ assigns conditional probability 1 to some statement that cannot be proven.

Open Problem: Does this theorem still hold if we leave condition 3 out of the definition of coherence?

[Scott Garrabrant]

Charlie: Your proposal to remove condition 3 works when $T$ is a finite theory, but you cannot do that when $T$ is infinite. Indeed, we use 3 for the infinite extension form $T_n$ to $T_{\infty }$. I suspect that you cannot remove condition 3.

[Charlie Steiner]

Is the idea that the proof necessary to use 1 is of infinite length, and you want your logic to be finitary? Hm. This seems odd, because $P\left(s|T_{\infty }\right)$ is in some sense already a function with an infinitely long argument. How do you feel about using 2 in the form of $P\left(T_{\infty }|T_0\right)=P\left(T_n|T_0\right)\cdot \left[someprobability\right]$, therefore $P\left(T_{\infty }|T_0\right)\le P\left(T_n|T_0\right)$, which has the same amount of argument as $P\left(s|T_{\infty }\right)$? I’m confused about at least one thing.

Also, is there some reason you prefer not to reply using the button below the comment?

[Charlie Steiner]

It’s interesting that this is basically the opposite of the Gaifman condition—clearly there are conflicting intuitions about what makes a ‘good’ conditional logical probability.

On the open problem; In order to prove 3 from 2, all you need is that $P(s\wedge T|R)=P\left(s|R\right)$ when $s$ proves $T$ − 3 follows from 2 if you do that substitution, and then divide by $P\left(T|R\right)$, which is less than or equal 1 (this may assume an extra commonsense axiom that probabilities are positive).

Now consider applying rule 1 to $P(s\wedge T|R)$, T proven by s. R proves that only one of $s\wedge T,\neg s\wedge T,\neg s\wedge \neg T$ is true, and also proves that only one of $s,\neg s\wedge T,\neg s\wedge \neg T$ is true. Thus 3 is derivable from 1 and 2.

[János Kramár]

This is interesting! I would dispute, though, that a good logical conditional probability must be able to condition on arbitrary, likely-non-r.e. sets of sentences.

[Benya_Fallenstein]

Hm; we could add an uninterpreted predicate symbol $Q\left(n\right)$ to the language of arithmetic, and let $s\equiv Q\left(0\right)$ and $r_n\equiv Q\left(\overline{n+1}\right)$. Then, it seems like the only barrier to recursive enumerability of $T_{\infty }$ is that $P$’s opinions about $Q(\cdot )$ aren’t computable; this seems worrying in practice, since it seems certain that we would like logical uncertainty to be able to reason about the values of computations that use more resources than we use to compute our own probability estimates. But on the other hand, all of this makes this sound like an issue of self-reference, which is its own can of worms (once we have a computable process assigning probabilities to the value of computations, we can consider the sentence saying “I’m assigned probability $<\frac{1}{2}$” etc.).

[orthonormal]

Nice! Basically, it looks like you construct a theory by assembling an infinite quantity of what the prior takes as evidence about $s$, so that either the prior or the posterior has to take the most extreme odds on $s$. It’s pretty intuitive in that light, and so I’m not dismayed that the “0 and 1 are not probabilities” property can’t hold when conditioned on arbitrary theories.

Important typo: $P$ assigns conditional probability 1 to some statement that cannot be proven.