Definition of the regression function \(r(x)\)

\(r(x)=E[Y|X=x]\) where conditional expectation is defined here.

The regression for discrete \(Y\) is called classification.

Regression analysis

Regression analysis consists in estimating \(r(x)\).

If we define \(\epsilon\) as \(\epsilon = Y -r(x)\), then the regression model can be expressed as

\[Y=r(x)+\epsilon\]

with \(E[\epsilon]=0\).

\(r(x)\) corresponds to the deterministic part.

References

Wasserman, L. All of statistics (2004). Springer Science & Business Media.