Parameter space $$\Theta$$

It represents the set of possible unkown states of nature.

Sample space $$\mathcal{X}$$

It is where the data $$x$$ lies.

Family of probability distributions in $$\mathcal{X}$$

Often indexed by $$\theta$$ $f(x; \theta)$

Action space $$\mathcal{A}$$

Set of all actions or decisions. We use $$a$$ to refer to a particular action.

Loss function $$L$$

$$L: \Theta \times \mathcal{A} \rightarrow \mathbb{R}$$

If we choose the action $$a \in \mathcal{A}$$ when the parameter is $$\theta \in \Theta$$, we incur a loss $$L(\theta,a)$$.

Decision rule $$d \in \mathcal{D}$$

$$d: \mathcal{X} \rightarrow \mathcal{A}$$

Risk function $$R$$

$$R(\theta,d) = E_{\theta} [L(\theta,d(X))] = \int L(\theta,d(X)) f(x; \theta) dx$$

Maximum risk

$$\overline{R}(\widehat{\theta}) = \sup_\theta R(\theta,\widehat{\theta})$$

Bayes risk

$$r(f,\widehat{\theta}) = \int R(\theta,\widehat{\theta}) f(\theta) d\theta$$ where $$f(\theta)$$ is a prior for $$\theta$$.

References

Wasserman, L. (2013). All of statistics: A concise course in statistical inference. Springer Science & Business Media.

Young, G. A., & Smith, R. L. (2005). Essentials of statistical inference (Vol. 16). Cambridge University Press.