The SIA population update can be surprisingly small

[Stuart_Armstrong]

With many thanks to Damon Binder, and the spirited conversations that lead to this post, and to Anders Sandberg.

People often think that the self-indication assumption (SIA) implies a huge number of alien species, millions of times more than otherwise. Thought experiments like the presumptuous philosopher seem to suggest this.

But here I’ll show that, in many cases, updating on SIA doesn’t change the expected number of alien species much. It all depends on the prior, and there are many reasonable priors for which the SIA update does nothing more than double the probability of life in the universe^[1].

This can be the case even if the prior says that life is very unlikely! We can have a situation where we are astounded, flabbergasted, and disbelieving about our own existence—“how could we exist, how can this beeeeee?!?!?!?”—and still not update much—“well, life is still pretty unlikely elsewhere, I suppose”.

In the one situation where we have an empirical distribution, the “Dissolving the Fermi Paradox” paper, the effect of the SIA anthropics update is to multiply the expected civilization per planet by seven. Not seven orders of magnitude—just seven.

The formula

Let $\rho \in [0,1]$ be the probability of advanced space-faring life evolving on a given planet; for the moment, ignore issues of life expanding to other planets from their one point of origin. Let $f$ be the prior distribution of $\rho$, with mean $\mu$ and variance ${\sigma }^2$. This means that, if we visit another planet, our probability of finding life is $\mu$.

On this planet, we exist^[2]. Then if we update on our existence we get a new distribution $f^{\prime }$; this distribution will have mean ${\mu }^{\prime }$:

$${\mu }^{\prime }=\mu (1+\frac{{\sigma }^2}{{\mu }^2}).$$

To see a proof of this result, look at this footnote^[3].

Define $M_{\mu ,{\sigma }^2}=1+{\sigma }^2/{\mu }^2$ to be this multiplicative factor between $\mu$ and ${\mu }^{\prime }$; we’ll show that there are many reasonable situations where $M_{\mu ,{\sigma }^2}$ is surprisingly low: think $2$ to $100$, rather than in the millions or billions.

Beta distributions I

Let’s start with the most uninformative prior of all: a uniform prior over $[0,1]$. The expectation of $\rho$ is ${\int }_0^1\rho d\rho =1/2$, so, without any other information, we expect a planet to have life with $50\%$ probability. The variance is ${\sigma }^2=1/12$.

Thus if we update on our existence on Earth, we get the posterior $f^{\prime }\left(\rho \right)=2\rho$; the mean of this is $2/3$ (either direct calculation or using $M_{1/2,1/12}=1+4/12=4/3$).

Even though this change in expectation is multiplicatively small, it does seem that the uniform prior and the $f^{\prime }\left(\rho \right)$ are very different, with $f^{\prime }\left(\rho \right)$ heavily skewed to the right. But now consider what happens if we look at Mars and notice that it hasn’t got life. The probability of no life, given $\rho$, is $1-\rho$. Updating on this and renormalising gives a posterior $6\rho (1-\rho )$:

The expectation of $6\rho (1-\rho )$, symmetric around $1/2$, is of course $1/2$. Thus one extra observation (that Mars is dead) has undone, in expectation, all the anthropic impact of our own existence.

This is an example of a beta distribution for $\alpha =2$ and $\beta =2$ (yes, beta distributions have a parameter called $\beta$ and another one that’s $\alpha$; just deal with it). Indeed, the uniform prior is also a beta distribution (with $\alpha =\beta =1$) as is the anthropic updated version $2\rho$ (which has $\alpha =2$, $\beta =1$).

The update rule for beta distributions is that a positive observation (ie life) increases $\alpha$ by $1$, and a negative observation (a dead planet) increases $\beta$ by $1$. The mean of an updated beta distribution is a generalised version of Laplace’s law of succession: if our prior is a beta distribution with parameters $\alpha$ and $\beta$, and we’ve had $m$ positive observations and $n$ negative ones, then the mean of the posterior is:

$$\frac{\alpha +m}{\alpha +\beta +m+n}.$$

Suppose now that we have observed $n$ dead planets, but no life, and that we haven’t done an anthropic update yet, then we have a probability of life of $\alpha /(\alpha +\beta +n)$. Upon adding the anthropic update, this shifts to $(\alpha +1)/(\alpha +\beta +n+1)$, meaning that the multiplicative factor is at most $(\alpha +1)/\alpha$. If we started with the uniform prior with its $\alpha =1$, this multiplies the probability of life by at most $2$. In a later section, we’ll look at $\alpha <1$.

High prior probability is not required for weak anthropic update

The uniform prior has $\alpha =\beta =1$ and starts at expectation $1/2$. But we can set $\alpha =1$ and a much higher $\beta$, which skews the distribution to the left; for example, for $\beta =2$, $3$, and $10$:

Even though these priors are skewed to the left, and have lower prior probabilities of life ($1/3$, $1/4$, and $1/11$), the anthropic update has a factor $M_{\mu ,{\sigma }^2}$ that is less than $2$.

Also note that if we scale the prior $f$ by a small $ϵ$, so replace $f\left(\rho \right)$ on the range $[0,1]$ with $f\left(\rho /ϵ\right)/ϵ$ on the range $[0,ϵ]$, then $\mu$ is multiplied by $ϵ$ and ${\sigma }^2$ is multiplied by ${ϵ}^2$. Thus $M_{\mu ,ϵ}$ is unchanged. Here, for example, is the uniform distribution, scaled down by $ϵ=1$, $ϵ=1/3$, and $ϵ=1/20$:

All of these will have the same $M_{\mu ,{\sigma }^2}$ (which is $4/3$, just as for the uniform distribution). And, of course, doing the same scaling with the various beta distributions we’ve seen up until now will also keep $M_{\mu ,{\sigma }^2}$ constant.

Thus there are a lot of distributions with very low $\mu$ (ie very low prior probability of life) but an $M_{\mu ,{\sigma }^2}$ that’s less than $2$ (ie the anthropic update is less than a doubling of the probability of life).

Beta distributions II and log-normals

The best-case scenario for $M_{\mu ,{\sigma }^2}$ is if $f$ assigns probability $1$ to $\rho =\mu$. In that case, ${\sigma }^2=0$ and $M=1$: the anthropic update changes nothing.

Conversely, the worse-case scenario for $M_{\mu ,{\sigma }^2}$ is if $f$ only allows $\rho =0$ and $\rho =1$. In that case, $f$ assigns probability $\mu$ to $1$ and $1-\mu$ to $0$, for a mean of $\mu$ and a variance of ${\sigma }^2=\mu -{\mu }^2$, and a multiplicative factor of $M_{\mu ,{\sigma }^2}=1/\mu$. In this case, after anthropic update, $f^{\prime }$ assigns certainty to $\rho =1$ (since any life at all, given this $f$, means life on all planets).

But there are also more reasonable priors with large $M_{\mu ,{\sigma }^2}$. We’ve already seen some, implicitly, above: the beta distributions with $\alpha <1$. In that case, $M_{\mu ,{\sigma }^2}$ is bounded by $(\alpha +1)/\alpha$. If $\alpha =3/4$ and $\beta =1$, for instance, this corresponds to the (unbounded) distribution $f\left(\rho \right)=\left(3/4\right){\rho }^{-1/4}$; the multiplicative factor is below $7/3$, which is slightly above $2$. But as $\alpha$ declines, the multiplicative factor can go up surprisingly fast; at $\alpha =1/2$ it is $3$, at $\alpha =1/4$ it is $5$:

In general, for $\alpha =1/n$, the multiplicative factor is bounded by $n+1$. This gets arbitrarily large as $\alpha \to 0$. Though $\alpha =0$ itself corresponds to the improper prior $f\left(\rho \right)=1/\rho$, whose integral diverges. On a log scale, this corresponds to the log-uniform distribution, which is roughly what you get if you assume “we need $N$ steps, each of probability $p$, to get life; let’s put a uniform prior over the possible $N$s”.

It’s not clear why one might want to choose $\alpha =1/{10}^{20}$ for a prior, but there is a class of prior that is much more natural: the log-normal distributions. These are random variables $X$ such that $\log \left(X\right)$ is normally distributed.

If we choose $\log \left(X\right)$ to have a mean that is highly negative (and a variance that isn’t too large), then we can mostly ignore the fact that $X$ takes values above $1$, and treat it as a prior distribution for $\rho$. The mean and variance of the log-normal distributions can be explicitly defined, thus giving the multiplications factor as:

$$M_{\mu ,{\sigma }^2}=\exp {\overline{\sigma }}^2.$$

Here, ${\overline{\sigma }}^2$ is the variance of the normal distribution $\log \left(X\right)$. This ${\overline{\sigma }}^2$ might be large, as it denotes (roughly) “we need $N$ steps, each of probability $p$, to get life; let’s put a uniform-ish prior over a range of possible $N$s”. Unlike $1/\rho$, this is a proper prior, and a plausible one; therefore there are plausible priors with very large $M_{\mu ,{\sigma }^2}$. The log normal is quite likely to appear, as it is the approximate limit of multiplying together a host of different independent parameters.

Multiplication law

Do you know what’s more likely to be useful than “the approximate limit of multiplying together a host of different independent parameters”? Actually multiplying together independent parameters.

The famous Drake equation is:

$$R_{\ast }\cdot f_p\cdot n_e\cdot f_l\cdot f_i\cdot f_c\cdot L.$$

Here $R^{\ast }$ is the number of stars in our galaxy, $f_p$ the fraction of those with planets, $n_e$ the number of planets that can support life per star that has planets, $f_l$ the fraction of those that develop life, $f_i$ the fraction of those that develop intelligent life, $f_c$ the fraction of those that release detectable signs of their existence, and $L$ is the length of time those civilizations endure as detectable.

Then the proportion of advanced civilizations per planet is $q{f}_l{f}_i$, where $q$ is the proportion of life-supporting planets among all planets. To compute the $M$ of this distribution, we have the highly useful result (the proof is in this footnote^[4]):

• Let $X_i$ be independent random variables with multiplicative factors $M_i$, and let $M$ be the multiplicative factor of $X=X_1\cdot X_2\cdot \dots \cdot X_n$. Then $M={\prod }_i{M}_i$ - the total $M$ is the product of the individual $M_i$.

The paper “dissolving the Fermi paradox” gives estimated distributions for all the terms in the Drake equation. The $q$, which doesn’t appear in that paper, is a constant, so has $M_q=1$. The $f_i$ has a log-uniform distribution from $0.001$ to $1$; the $M$ can be computed from the mean and variance of such distributions, so $M_{f_i}=\log \left(1/0.001\right)\frac{1-{0.001}^2}{2(1-0.001{)}^2}\approx 3.5$.

The $f_l$ term is more complicated; it is distributed like $g\left(X\right)=1-e^{-e^{X\cdot 50\log \left(10\right)}}$ where $X$ is a standard normal distribution. Fortunately, we can estimate its mean and variance without having to figure out its distribution, by numerical integration of $g\left(x\right)$ and $g\left(x^2\right)$ on the normal distribution. This gives $\mu \approx 0.5$, ${\sigma }^2\approx 0.25$ and $M\approx 2$. The overall the multiplicative effect of anthropic update is:

$$M_{\text{planet}}\approx 7.$$

What if we considered the proportion of advanced civilization per star, rather than per planet? Then we can drop the $q$ term and add in $f_p$ and $n_e$. Those are both estimated to be distributed as log-uniform on $[0.1,1]$; for a total $M$ of

$$M_{\text{star}}\approx 14.$$

Why is the $M$ higher for civilizations per star than civilizations per planet? That’s because when we update on our existence, we increase the proportion of civilizations per planet, but we also update the proportion of planets per star—both of these can make life more likely. The $M_{\text{star}}$ incorporates both effects, so is strictly higher than $M_{\text{planet}}$.

We can do the same by considering the number of civilizations per galaxy; then we have to incorporate $R_{\ast }$ as well. This is log-uniform on $[1,100]$, giving:

$$M_{\text{galaxy}}\approx 32.$$

What about if we include the Fermi observation (the fact that we don’t see anything in our galaxy)? The “dissolving the Fermi paradox” paper shows there are multiple different ways of including this update, depending on how we parse out “not seeing anything” and how easy it is for civilizations to expand.

I did a crude estimate here by taking the Fermi observation to mean “the proportion of civilizations per galaxy must be less than one”. Then I did a Monte-Carlo simulation, ignoring all results above $0$ on the log scale:

From this, I got an estimated mean of $0.027$, variance of $0.014$, and a total multiplier of:

$$M_{\text{galaxy, Fermi}}\approx 21.$$

With the Fermi observation and the anthropic update combined, we expect $0.56$ civilizations per galaxy.

Limitations of the multiplier

Low multiplier, strong effects

It’s important to note that the anthropic update can be very strong, without changing the expected population much. So a low $M_{\mu ,{\sigma }^2}$ doesn’t necessary mean a low impact.

Consider for instance the presumptuous philosopher, slightly modified to use planetary population densities. Thus theory $T_1$ predicts $\rho =1/{10}^{12}$ (one in a trillion) and $T_2$ predicts $\rho =1$; we put initial probabilities $1/2$ on both theories.

As Nick Bostrom noted, the SIA update pushes $T_2$ to being a trillion times more probable than $T_1$; a postiori, $T_2$ is roughly a certainty (the actual probability is ${10}^{12}/({10}^{12}+1)$).

However, the expected population goes from roughly $1/2$ (the average of $1/{10}^{12}$ and $1$) to roughly $1$ (since a postiori $T_2$ is almost certain). This gives a $M_{\mu ,{\sigma }^2}$ of roughly $2$. So, despite the strong update towards $T_2$, the actual population update is small—and, conversely, despite the actual population update being small, we have a strong update towards $T_2$.

Combining multiple theories

In the previous post, note that that both $T_1$ and $T_2$ were point estimates: they posit a constant $\rho$. So they have a variance of zero, and hence a $M_{\mu ,{\sigma }^2}$ of $1$. But $T_2$ has a much stronger anthropic update. Thus we can’t use their $M_{\mu ,{\sigma }^2}$ to compare the anthropic effects on different theories.

We also can’t relate the individual $M$s to that of a combined theory. As we’ve seen, $T_1$ and $T_2$ have $M$s of $1$, but the combined theory $\left(1/2\right)T_1+\left(1/2\right)T_2$ has an $M$ of roughly $2$. But we can play around with the relative initial weight of $T_1$ and $T_2$ to get other $M$s.

If we started with odds ${10}^{12}:1$ on $T_1$ vs $T_2$, then this has a mean $\rho$ of roughly ${10}^{-12}$; the anthropic update sends it to $1:1$ odds, with a mean of roughly $1/2$. So this combined theory has an $M$ of roughly ${10}^{12}/2$, half a trillion.

But, conversely, if we started with odds $1:{10}^{12}$ on $T_1$ vs $T_2$, then we have an initial mean of $\rho$ of roughly one; its anthropic update is odds of $1:{10}^{24}$, also with a mean of roughly one. So this combined theory has an $M$ of roughly $1$.

There is a weak relation between $M$ and the $M_i$ of the various $T_i$. Let $M_i$ be the multiplier of $T_i$ has a multiplier of $M_i$; we can reorder the $T_i$ so that $M_i\le M_j$ for $i\le j$. Let $T$ be a combined theory that assigns probability $p_i$ to $T_i$.

1. For all $\left\{p_i\right\}$, $M\ge {min}_i\left(M_i\right)$.

2. For all $ϵ$, there exists $\left\{p_i\right\}$ with all $p_i>0$, so that $M<{min}_i\left(M_1\right)+ϵ$.

So, the minimum value of the $M_i$ is a lower bound on $M$, and we can get arbitrarily close to that bound. See the proof in this footnote^[5].

---

1. ↩︎

   As we’ll see, the population update is small even in the presumptuous philosopher experiment itself.

2. ↩︎

   Citation partially needed: I’m ignoring Boltzmann brains and simulations and similar ideas.

3. ↩︎

   Given a fixed $\rho$, the probability of observing life on our own planet is exactly $\rho$. So Bayes’s theorem implies that $f^{\prime }\left(\rho \right)\propto \rho f\left(\rho \right)$. With the full normalisation, this is

   $$f^{\prime }\left(\rho \right)=\frac{\rho f\left(\rho \right)}{{\int }_0^1\rho f\left(\rho \right)d\rho }.$$

   If we want to get the mean ${\mu }^{\prime }$ of this distribution, we further multiply by $\rho$ and integrate:

   $${\mu }^{\prime }=E_{f^{\prime }}\left(\rho \right)={\int }_0^1\frac{{\rho }^{2}f\left(\rho \right)}{{\int }_0^1\rho f\left(\rho \right)d\rho }d\rho =\frac{{\int }_0^1{\rho }^{2}f\left(\rho \right)d\rho }{{\int }_0^1\rho f\left(\rho \right)d\rho }.$$

   Let’s multiply this by $1=1/1=\left({\int }_0^{1}f\right(\rho \left)d\rho \right)/\left({\int }_0^{1}f\right(\rho \left)d\rho \right)$ and regroup the terms:

   $${\mu }^{\prime }=\frac{{\int }_0^1{\rho }^{2}f\left(\rho \right)d\rho }{{\int }_0^{1}f\left(\rho \right)d\rho }\cdot \frac{{\int }_0^{1}f\left(\rho \right)d\rho }{{\int }_0^1\rho f\left(\rho \right)d\rho }.$$

   Thus ${\mu }^{\prime }=$ $E_f\left({\rho }^2\right)/E_f\left(\rho \right)=$ $({\sigma }^2+{\mu }^2)/\mu =$ $\mu (1+{\sigma }^2/{\mu }^2)$, using the fact that the variance is the expectation of ${\rho }^2$ minus the square of the expectation of $\rho$.

4. ↩︎

   I adapted the proof in this post.

   So, let $X_i$ be independent random variables with means ${\mu }_i$ and variances ${\sigma }_i^2$. Let $X={\prod }_i{X}_i$, which has mean $\mu$ and variance ${\sigma }^2$. Due to the independence of the $X_i$, the expectations of their products are the product of their expectations. Note that $X_i^2$ and $X_j^2$ are also independent if $i\ne j$. Then we have:

   $$\begin{array}{cc}{\prod }_i{M}_{{\mu }_i,{\sigma }_i^2}& ={\prod }_i(1+\frac{{\sigma }_i^2}{{\mu }_i^2})\\ & ={\prod }_i\left(\frac{{\mu }_i^2+{\sigma }_i^2}{{\mu }_i^2}\right)\\ & ={\prod }_i\left(\frac{E\left(X_i^2\right)}{{\mu }_i^2}\right)\\ & =\frac{{\prod }_i\left(E\right(X_i^2\left)\right)}{{\prod }_{i}E(X_i{)}^2}\\ & =\frac{E\left(X^2\right)}{E(X{)}^2}\\ & =\frac{{\mu }^2+{\sigma }^2}{{\mu }^2}\\ & =1+\frac{{\sigma }^2}{{\mu }^2}=M_{\mu ,{\sigma }^2}.\end{array}$$

5. ↩︎

   Let $\{f_i{\}}_{1\le i\le n}$ be probability distributions on $\rho$, with mean ${\mu }_i$, variance ${\sigma }_i^2$, expectation squared $s_i=E_{f_i}\left({\rho }^2\right)={\sigma }_i^2+{\mu }_i^2$, and $M_i=s_i/{\mu }_i^2$. Without loss of generality, reorder the $f_i$ so that $M_i\le M_j$ for $i<j$.

   Let $f$ be the probability distribution $f=p_1{f}_1+\dots p_n{f}_n$, with associated multiplier $M$. Without loss of generality, assume $M_i\le M_j$ for $i<j$. Then we’ll show that $M\ge M_1$.

   We’ll first show this in the special case where $n=2$ and $M_1=M_2$, then generalise to the general case, as is appropriate for a generalisation. If $s_1/{\mu }_1^2=M_1=M_2=s_2/{\mu }_2^2$, then, since all terms are non-negative, there exists an $\alpha$ such that $s_1={\alpha }^2{s}_2$ while ${\mu }_1=\alpha {\mu }_2$. Then for any given $p=p_1$, the $M$ of $f$ is:

   $$M\left(p\right)=\frac{p{s}_1+(1-p)s_2}{(p{\mu }_1+(1-p){\mu }_2{)}^2}=\frac{p{s}_1+(1-p){\alpha }^2{s}_1}{(p{\mu }_1+(1-p)\alpha {\mu }_1{)}^2}=M_1\frac{1\left(p\right)+{\alpha }^2(1-p)}{\left(1\right(p)+\alpha (1-p){)}^2}.$$

   The function $x\to x^2$ is convex, so, interpolating between the values $x=1$ and $x=\alpha$, we know that for all $0\le p\le 1$, the term $\left(1\right(p)+\alpha (1-p){)}^2$ must be lower than $1^2\left(p\right)+{\alpha }^2(1-p)$. Therefore $\left(1\right(p)+{\alpha }^2(1-p\left)\right)/\left(1\right(p)+\alpha (1-p){)}^2$ is at most $1$, and $M\left(p\right)\le M_1$. This shows the result for $n=2$ if $M_1=M_2$.

   Now assume that $M_2>M_1$, so that $s_1/{\mu }_1^2<s_2/{\mu }_2^2$. Then replace $s_2$ with $s_2^{\prime }$, which is lower than $s_2$, so that $s_1/{\mu }_1^2=s_2^{\prime }/{\mu }_2^2$. If we define $M^{\prime }\left(p\right)$ as the expression for $M\left(p\right)$ with $s_2′ substituting for $s_2$, we know that $M^{\prime }\left(p\right)\le M\left(p\right)$, since $s_2^{\prime }<s_2$. Then the previous result shows that $M^{\prime }\left(p\right)\ge M_1$, thus $M\left(p\right)\ge M_1$ too.

   To show the result for larger $n$, we’ll induct on $n$. For $n=1$ the result is a tautology, $M_1\le M_1$, and we’ve shown the result for $n=2$. Assume the result is true for $n-1$, and then notice that $f=p_1{f}_1+\dots p_n{f}_n$ can be re-written as $f=p_1{f}_1+(1-p_1)f^{\prime }$, where $f^{\prime }=(p_2^{\prime }{f}_2+\dots p_n^{\prime }{f}_n)$ for $p_i^{\prime }=p_n/(1-p_1)$. Then, by the induction hypothesis, if $M^{\prime }$ is the $M$ of $f^{\prime }$, then $M^{\prime }\ge M_2$. Then applying the result for $n=2$ between $f_1$ and $f^{\prime }$, gives $M\le min(M_1,M^{\prime })$. However, since $M_1\le M_2$ and $M^{\prime }\ge M_2$, we know that $min(M_1,M^{\prime })=M_1$, proving the general result.

   To show $M$ can get arbitrarily close to $M_1$, simply note that $M$ is continuous in the $\left\{p_i\right\}$, define $p_1=1-ϵ$, $p_i=ϵ/(n-1)$ for $i>1$, and let $ϵ$ tend to $0$.