A $$1 - \alpha$$ confidence interval for a parameter $$\theta$$ is an interval $$C_n = (a,b)$$ where $$a=a(X_1,\dotsc,X_n)$$ and $$b=b(X_1,\dotsc,X_n)$$ are functions of the data such that $$P(\theta \in C_n) = 1 - \alpha$$.

The action space is $$\mathcal{A}=\Theta$$.

Loss function

$$L(\theta,a) = \left\{ \begin{array}{ll} 0 & \mbox{if } |\theta - a| \leq \delta \\ 1 & \mbox{if } |\theta - a| > \delta \\ \end{array} \right.$$

Interpretation

A confidence interval is not a probability statement about $$\theta$$, as $$\theta$$ is a fixed quantity, not a random variable.

A $$1 - \alpha$$ confidence interval means that $$1 - \alpha$$ times when you construct confidence interval it will contain $$\theta$$.

Confidence interval for $$\mu$$

If $$\widehat{\theta_n}$$ is normally distributed $$N(\mu,se^2)$$ with $$se^2=\sigma^2/n$$

For example, if $$X_1,\dotsc,X_n$$ are normally distributed or the sample is large (central limit theorem).

Knowing $$\sigma^2$$

$C_n = (\widehat{\theta_n} - z_{1-\alpha/2} \, se, \widehat{\theta_n} + z_{1-\alpha/2}\, se)$

• Demonstration

$P \left( \widehat{\theta_n} - z_{1-\alpha/2} \,se < \theta < \widehat{\theta_n} + z_{1-\alpha/2} \,se \right) = P \left( -z_{1-\alpha/2} < \frac{\widehat{\theta_n} - \theta}{se} < z_{1-\alpha/2} \right)=P\left(- z_{1-\alpha/2} < Z < z_{1-\alpha/2} \right)=1-\alpha$

Not knowing $$\sigma^2$$

$C_n = (\widehat{\theta_n} - t_{1-\alpha/2} \, \widehat{se}, \widehat{\theta_n} + t_{1-\alpha/2}\, \widehat{se})$

with $$\widehat{se}^2=S^2_n / n$$

• Demonstration

$P \left( \widehat{\theta_n} - t_{1-\alpha/2} \,\widehat{se} < \theta < \widehat{\theta_n} + t_{1-\alpha/2} \, \widehat{se} \right) = P \left( -t_{1-\alpha/2} < \frac{\widehat{\theta_n} - \theta}{\widehat{se}} < t_{1-\alpha/2} \right)=P\left( -t_{1-\alpha/2} < T < t_{1-\alpha/2} \right)=1-\alpha$

• Example
library(dplyr)
data(sleep)
before <- sleep %>% filter(group ==1)
after <- sleep %>% filter(group ==2)
difference<- after$extra - before$extra

Manual calculation

mean(difference) + c(-1,1) * qt(.975,length(difference)-1) * sd(difference)/sqrt(length(difference)) # sd uses (n-1) in the denominator
## [1] 0.7001142 2.4598858

Automatic calculation (two methods)

t.test(difference)\$conf.int
## [1] 0.7001142 2.4598858
## attr(,"conf.level")
## [1] 0.95
library(Hmisc)
smean.cl.normal(difference)
##      Mean     Lower     Upper
## 1.5800000 0.7001142 2.4598858

Confidence interval for $$\sigma^2$$

If $$X_1,\dotsc,X_n$$ are normally distributed

$C_n = \left( \frac{\left( n-1 \right) \, S^2}{\chi^2_{n-1, 1 - \alpha /2}}, \frac{\left( n-1 \right) \, S^2}{\chi^2_{n-1, \alpha /2 }} \right)$

• Demonstration

$P \left( \frac{\left( n-1 \right) \, S^2}{\chi^2_{n-1, 1 - \alpha /2}} < \sigma^2 < \frac{\left( n-1 \right) \, S^2}{\chi^2_{n-1, \alpha /2 }} \right) = P \left( \chi^2_{n-1, \alpha /2} < \frac{\left( n - 1 \right) \, S^2 }{\sigma^2} < \chi^2_{n-1, 1 - \alpha /2 } \right) = 1 - \alpha$

given that it can be demonstrated that the statistic $$\frac{\left( n -1 \right) \, S^2}{\sigma^2}$$ is distributed $$\chi^2_{n-1}$$.