A \(1 - \alpha\) confidence interval for a parameter \(\theta\) is an interval \(C_n = (a,b)\) where \(a=a(X_1,\dotsc,X_n)\) and \(b=b(X_1,\dotsc,X_n)\) are functions of the data such that \(P(\theta \in C_n) = 1 - \alpha\).

The action space is \(\mathcal{A}=\Theta\).

Loss function

\(L(\theta,a) = \left\{ \begin{array}{ll} 0 & \mbox{if } |\theta - a| \leq \delta \\ 1 & \mbox{if } |\theta - a| > \delta \\ \end{array} \right.\)

Interpretation

A confidence interval is not a probability statement about \(\theta\), as \(\theta\) is a fixed quantity, not a random variable.

A \(1 - \alpha\) confidence interval means that \(1 - \alpha\) times when you construct confidence interval it will contain \(\theta\).

Confidence interval for \(\mu\)

If \(\widehat{\theta_n}\) is normally distributed \(N(\mu,se^2)\) with \(se^2=\sigma^2/n\)

For example, if \(X_1,\dotsc,X_n\) are normally distributed or the sample is large (central limit theorem).

Knowing \(\sigma^2\)

\[C_n = (\widehat{\theta_n} - z_{1-\alpha/2} \, se, \widehat{\theta_n} + z_{1-\alpha/2}\, se)\]

  • Demonstration

\[P \left( \widehat{\theta_n} - z_{1-\alpha/2} \,se < \theta < \widehat{\theta_n} + z_{1-\alpha/2} \,se \right) = P \left( -z_{1-\alpha/2} < \frac{\widehat{\theta_n} - \theta}{se} < z_{1-\alpha/2} \right)=P\left(- z_{1-\alpha/2} < Z < z_{1-\alpha/2} \right)=1-\alpha\]

Not knowing \(\sigma^2\)

\[C_n = (\widehat{\theta_n} - t_{1-\alpha/2} \, \widehat{se}, \widehat{\theta_n} + t_{1-\alpha/2}\, \widehat{se})\]

with \(\widehat{se}^2=S^2_n / n\)

  • Demonstration

\[P \left( \widehat{\theta_n} - t_{1-\alpha/2} \,\widehat{se} < \theta < \widehat{\theta_n} + t_{1-\alpha/2} \, \widehat{se} \right) = P \left( -t_{1-\alpha/2} < \frac{\widehat{\theta_n} - \theta}{\widehat{se}} < t_{1-\alpha/2} \right)=P\left( -t_{1-\alpha/2} < T < t_{1-\alpha/2} \right)=1-\alpha\]

  • Example
library(dplyr)
data(sleep)
before <- sleep %>% filter(group ==1) 
after <- sleep %>% filter(group ==2)
difference<- after$extra - before$extra

Manual calculation

mean(difference) + c(-1,1) * qt(.975,length(difference)-1) * sd(difference)/sqrt(length(difference)) # sd uses (n-1) in the denominator
## [1] 0.7001142 2.4598858

Automatic calculation (two methods)

t.test(difference)$conf.int
## [1] 0.7001142 2.4598858
## attr(,"conf.level")
## [1] 0.95
library(Hmisc)
smean.cl.normal(difference)
##      Mean     Lower     Upper 
## 1.5800000 0.7001142 2.4598858

Confidence interval for \(\sigma^2\)

If \(X_1,\dotsc,X_n\) are normally distributed

\[C_n = \left( \frac{\left( n-1 \right) \, S^2}{\chi^2_{n-1, 1 - \alpha /2}}, \frac{\left( n-1 \right) \, S^2}{\chi^2_{n-1, \alpha /2 }} \right) \]

  • Demonstration

\[ P \left( \frac{\left( n-1 \right) \, S^2}{\chi^2_{n-1, 1 - \alpha /2}} < \sigma^2 < \frac{\left( n-1 \right) \, S^2}{\chi^2_{n-1, \alpha /2 }} \right) = P \left( \chi^2_{n-1, \alpha /2} < \frac{\left( n - 1 \right) \, S^2 }{\sigma^2} < \chi^2_{n-1, 1 - \alpha /2 } \right) = 1 - \alpha\]

given that it can be demonstrated that the statistic \(\frac{\left( n -1 \right) \, S^2}{\sigma^2}\) is distributed \(\chi^2_{n-1}\).