Parameter space \(\Theta\)

It represents the set of possible unkown states of nature.

Sample space \(\mathcal{X}\)

It is where the data \(x\) lies.

Family of probability distributions in \(\mathcal{X}\)

Often indexed by \(\theta\) \[f(x; \theta)\]

Action space \(\mathcal{A}\)

Set of all actions or decisions. We use \(a\) to refer to a particular action.

Loss function \(L\)

\(L: \Theta \times \mathcal{A} \rightarrow \mathbb{R}\)

If we choose the action \(a \in \mathcal{A}\) when the parameter is \(\theta \in \Theta\), we incur a loss \(L(\theta,a)\).

Decision rule \(d \in \mathcal{D}\)

\(d: \mathcal{X} \rightarrow \mathcal{A}\)

Risk function \(R\)

\(R(\theta,d) = E_{\theta} [L(\theta,d(X))] = \int L(\theta,d(X)) f(x; \theta) dx\)

Maximum risk

\(\overline{R}(\widehat{\theta}) = \sup_\theta R(\theta,\widehat{\theta})\)

Minimax rule

Bayes risk

\(r(f,\widehat{\theta}) = \int R(\theta,\widehat{\theta}) f(\theta) d\theta\) where \(f(\theta)\) is a prior for \(\theta\).

References

Wasserman, L. (2013). All of statistics: A concise course in statistical inference. Springer Science & Business Media.
Young, G. A., & Smith, R. L. (2005). Essentials of statistical inference (Vol. 16). Cambridge University Press.