\(\theta\) is not fixed, but a random variable. Thus, it is characterised by a prior \(\pi(\theta)\).

Accordingly, we now refer to the statistical model as \(f(x|\theta)\), instead of \(f(x; \theta)\), where \(x=(x_1,\dotsc,x_n)\).

\(\pi(\theta)\) and \(f(x|\theta)\) are related by Bayes rule

\[\pi(\theta|x) = \frac{f(x|\theta) \pi(\theta)}{\int_\theta f(x|\theta) \pi(\theta) d\theta}=k L(\theta)\pi(\theta)\]

The posterior \(\pi(\theta|x)\) has all the information.

Conjugate family

A class of prior distributions is a conjugate family for the class \(f(x|\theta)\) if the posterior distributions belong to the class of the prior distributions for any \(f(x|\theta)\).

References

Wasserman, L. (2013). All of statistics: A concise course in statistical inference. Springer Science & Business Media.

Young, G. A., & Smith, R. L. (2005). Essentials of statistical inference (Vol. 16). Cambridge University Press.