Definition of hypotheses

A hypothesis is a statement about a population parameter.

\[H_0: \theta \in \Theta_0 \text{ and } H_1: \theta \in \Theta_1\]

Often \(\Theta_1 = \Theta_0^C\).

Types of hypotheses

Simple hypothesis

\[H_0: \theta = \theta_0\]

Composite hypothesis

When the hypothesis is not simple.

Types of hypothesis test

Simple

\[H_0: \theta = \theta_0 \text{ and } H_1: \theta = \theta_1\]

Composite

One-side

\[H_0: \theta \leq \theta_0 \text{ and } H_1: \theta > \theta_0\]

or

\[H_0: \theta \geq \theta_0 \text{ and } H_1: \theta > \theta_0\]

Two-sides

\[H_0: \theta = \theta_0 \text{ and } H_1: \theta \neq \theta_0\]

That is, under \(H_1\), \(\theta = (\theta_1,\dotsc,\theta_m,\theta_{m+1},\dotsc,\theta_{m+p})\) and under \(H_0\), \(\theta = (\theta_1,\dotsc,\theta_m,\theta_{m+1}=\theta_{0,1},\dotsc,\theta_{m+p}=\theta_{0,p})\).

Action space

\(\mathcal{A} = \{ a_0, a_1\}\) where \(a_0\) represents choosing \(H_0\) and \(a_1\) represents choosing \(H_1\)

Decision rule

In the context of hypothesis testing, a decision rule is called hypothesis test and it is defined by the sample values \(x\) for which we accept or reject \(H_0\). The sample values for which we reject \(H_0\) is called the rejection region \(R\).

\(d(x) = \left\{ \begin{array}{ll} a_0 \ if \ x \not\in R \\ a_1 \ if \ x \in R \\ \end{array} \right.\)

The rejection region is often specified by establishing a criterium on a test statistic \(W(x)\) (function of the sample):

\[R=\{x: W(x)<c\}\]

A common method to find a test statistic is the likelihood ratio. To completely specify \(R\), we need \(c\) which can be obtained by setting the probabilities of making mistakes (for example fixing \(\alpha\)).

Example

\(d(x) = \left\{ \begin{array}{ll} a_0 \ if \ \overline{x} \leq 4.3 \\ a_1 \ if \ \overline{x} > 4.3 \\ \end{array} \right.\)

Evaluating tests (probabilities of making mistakes)

Type I error: \(H_0\) is true \((\theta \in \Theta_0)\) and \(a=a_1\).

Type II error: \(H_1\) is true \((\theta \in \Theta_1)\) and \(a=a_0\).

There are 4 interesting probabilities:

\(P(a_1 | H_0) = P (Type \,I\, error)\)

\(P(a_1 | H_1)\)

\(P(a_0 | H_0)\)

\(P(a_0 | H_1) = P (Type \, II \, error)\)

Power function of a test (power function of a rejection region)

\[\beta(\theta) = P_\theta (X \in R)\]

A good test should have \(\beta(\theta)\) near 0 when \(\theta \in \Theta_0\) and near 1 when \(\theta \in \Theta_1\).

Example 1

\(X \sim B(5,p)\)

\(H_0: p \leq 1/2\)

\(H_1: p > 1/2\)

\(R=\{k = 5 \}\)

\(\beta(p) = P_p(k \in R) = P_p(k = 5) = \binom{5}{5} p^5 (1-p)^0 = p^5\)

library(ggplot2)
p <- seq(0,1,.01)
beta <- p^5
dat <- data.frame(p,beta)
ggplot(dat,aes(x=p,y=beta))+geom_line()

The probability of Type II error is quite large.

\(R=\{k \geq 3 \}\)

\(\beta(p) = P_p(k \in R) = P_p(k = 3 \ or \ k = 4 \ or \ k =5 ) = \binom{5}{3} p^3 (1-p)^2 + \binom{5}{4} p^4 (1-p)^1 + \binom{5}{5} p^5(1-p)^0\)

p <- seq(0,1,.01)
beta <- 10*p^3*(1-p)^2 + 5*p^4*(1-p)+p^5
dat <- data.frame(p,beta)
ggplot(dat,aes(x=p,y=beta))+geom_line()

The Type II error decreases, but Type I increases.

Size \(\alpha\) of a test

\[sup_{\theta \in \Theta_0} \beta(\theta) = \alpha\]

In example 1, \(\alpha\)s are respectively

p <- .5
p^5
## [1] 0.03125
10*p^3*(1-p)^2 + 5*p^4*(1-p)+p^5
## [1] 0.5

Level \(\alpha\) of a test

\[sup_{\theta \in \Theta_0} \beta(\theta) \leq \alpha\]

Null hypothesis testing

\(H_0\) and \(H_1\) are treated asymmetrically as \(\alpha\) is fixed (often to .01 or .05) so that the Type I error is less than \(\alpha\).

UMP

A test in class \(\mathcal{C}\) with a power function \(\beta(\theta)\) is the uniformly most powerful (UMP) test of class \(\mathcal{C}\) if \(\beta(\theta) \geq \beta'(\theta)\) for all \(\theta \in \Theta_1\).

UMP level \(\alpha\)

\(\mathcal{C} = \text{ all level } \alpha \text{ tests}\).

p-value

A p-value \(p(X)\) is a test statistics that satisfies \[0 \leq p(X) \leq 1\]

It is valid if \[P_{\theta} \left( p \left( X \right) \leq \alpha\right) \leq \alpha\]

Loss and risk functions

0-1

\(L(\theta,a) = \left\{ \begin{array}{ll} 1 & \mbox{if } \theta \in \Theta_0 \text{ and } a=a_1\\ 1 & \mbox{if } \theta \in \Theta_1 \text{ and } a=a_0 \\ 0 & \text{otherwise} \\ \end{array} \right.\)

\(R(\theta,d) = \left\{ \begin{array}{ll} P_{\theta\ \in \Theta_0}\left(d(X)=a_1\right) =\beta(\theta) & \mbox{if } \theta \in \Theta_0 \\ P_{\theta\ \in \Theta_1}\left(d(X)=a_0\right) =1 -\beta(\theta) & \mbox{if } \theta \in \Theta_1 \\ \end{array} \right.\)

Demonstration

If \(\theta \in \Theta_0\)

\(R(\theta,d(X)) = \int_R L(\theta,d(X)) f(x; \theta) \, dx + \int_{R^C} L(\theta,d(X)) f(x; \theta) \, dx = \int_R L(\theta,a_1) f(x; \theta) \, dx + \int_{R^C} L(\theta,a_0) f(x; \theta) \, dx= \\ = \int_R 1 \, f(x; \theta) \, dx + \int_{R^C} 0 \, f(x; \theta) \, dx = \int_R f(x; \theta) \, dx = P_{\theta\ \in \Theta_0}(d(X)=a_1)\)

If \(\theta \in \Theta_1\)

\(R(\theta,d(X)) = \int_R L(\theta,d(X)) f(x; \theta) \, dx + \int_{R^C} L(\theta,d(X)) f(x; \theta) \, dx = \int_R L(\theta,a_1) f(x; \theta) \, dx + \int_{R^C} L(\theta,a_0) f(x; \theta) \, dx= \\ = \int_R 0 \, f(x; \theta) \, dx + \int_{R^C} 1 \, f(x; \theta) \, dx = \int_{R^C} f(x; \theta) \, dx = P_{\theta\ \in \Theta_1}(d(X)=a_0)\)

Example
p <- seq(0,1,.01)
risk <- ifelse(p<.5, p^5, 1 - p^5) 
dat <- data.frame(p,risk)
ggplot(dat,aes(x=p,y=risk))+geom_line()

Generalised 0-1

\(L(\theta,a) = \left\{ \begin{array}{ll} c_I & \mbox{if } \theta \in \Theta_0 \text{ and } a=a_1\\ c_{II} & \mbox{if } \theta \in \Theta_1 \text{ and } a=a_0 \\ 0 & \text{otherwise} \\ \end{array} \right.\)

\(R(\theta,d) = \left\{ \begin{array}{ll} P_{\theta\ \in \Theta_0}\left(d(X)=a_1\right) =c_I \beta(\theta) & \mbox{if } \theta \in \Theta_0 \\ P_{\theta\ \in \Theta_1}\left(d(X)=a_0\right) =c_{II} \left( 1 -\beta(\theta) \right) & \mbox{if } \theta \in \Theta_1 \\ \end{array} \right.\)

More general

\(L(\theta,a) = \left\{ \begin{array}{ll} -V_{00} & \mbox{if } \theta \in \Theta_0 \text{ and } a=a_0\\ V_{01} & \mbox{if } \theta \in \Theta_0 \text{ and } a=a_1\\ -V_{11} & \mbox{if } \theta \in \Theta_1 \text{ and } a=a_1\\ V_{10} & \mbox{if } \theta \in \Theta_1 \text{ and } a=a_0\\ \end{array} \right.\)

\(R(\theta,a) = \left\{ \begin{array}{ll} V_{10} P_{\theta\ \in \Theta_0}\left(d(X)=a_1\right) - V_{00} P_{\theta\ \in \Theta_0}\left(d(X)=a_0\right)& \mbox{if } \theta \in \Theta_0 \\ V_{01} P_{\theta\ \in \Theta_1}\left(d(X)=a_0\right) - V_{11} P_{\theta\ \in \Theta_1}\left(d(X)=a_1\right) & \mbox{if } \theta \in \Theta_1 \\ \end{array} \right.\)

References

Casella, G., & Berger, R. L. (2002). Statistical inference (Vol. 2). Duxbury Pacific Grove, CA.

Wasserman, L. (2013). All of statistics: A concise course in statistical inference. Springer Science & Business Media.

Young, G. A., & Smith, R. L. (2005). Essentials of statistical inference (Vol. 16). Cambridge University Press.