Jan Christian Refsgaard comments on a visual explanation of Bayesian updating

[Jan Christian Refsgaard]

I am well aware that nobody asked for this, but here is the proof that the posterior is $Beta(\alpha +z,\beta -z+1)$ for the beta-bernoulli model.

We start with Bayes Theorem:

$$p(\theta \mid z)=\frac{p(z\mid \theta )p\left(\theta \right)}{p\left(z\right)}$$

Then we plug in the definition for the Bernoulli likelihood and Beta prior:

$$p(\theta \mid z)={\theta }^z(1-\theta {)}^{1-z}\times \frac{{\theta }^{\alpha -1}(1-\theta {)}^{\beta -1}}{B(\alpha ,\beta )}\times \frac{1}{p\left(z\right)}$$

Let’s collect the powers in the numerator, and things that does not depend on $\theta$ in the denominator

$$p(\theta \mid z)=\frac{{\theta }^{\alpha +z-1}(1-\theta {)}^{\beta -z}}{B(\alpha ,\beta )p\left(z\right)}$$

Here comes the conjugation shenanigans. If you squint, the top of the distribution looks like the top of a Beta distribution:

$$\begin{array}{cc}{\alpha }^{\prime }& =\alpha +z\\ {\beta }^{\prime }& =\beta -z+1\\ p(\theta \mid z)& =\frac{{\theta }^{{\alpha }^{\prime }-1}(1-\theta {)}^{{\beta }^{\prime }-1}}{B(\alpha ,\beta )p\left(z\right)}\end{array}$$

Let’s continue the shenanigans, since the numerator looks like the numerator of a beta distribution, we know that it would be a proper beta distribution if we changed the denominator like this:

$$\begin{array}{cc}p(\theta \mid z)& =\frac{{\theta }^{{\alpha }^{\prime }-1}(1-\theta {)}^{{\beta }^{\prime }-1}}{B({\alpha }^{\prime },{\beta }^{\prime })}\\ p(\theta \mid z)& =\frac{{\theta }^{\alpha +z-1}(1-\theta {)}^{\beta -z}}{B(\alpha +z,\beta -z+1)}\end{array}$$