AP Statistics 5.3-5.4: Linear Regression and Least-Squares

Interpolation

Prediction for an explanatory-variable value within the observed range of the sample data. Interpolation is generally more reliable than extrapolation.

Extrapolation

Prediction for an explanatory-variable value outside the observed range of the sample data. It can be unreliable because the linear pattern may not continue beyond the data.

Why should a regression equation not automatically be used for an $x$-value outside the observed range?

The equation can produce a numerical answer, but there is no evidence that the linear relationship continues outside the sampled domain. Extrapolated predictions may be unrealistic or impossible.

How do you use a least-squares regression equation $\hat{y}=a+bx$ to predict a response?

Substitute the specified explanatory-variable value into the equation. The result $\hat{y}$ estimates the mean response for observations with that value of $x$.

Using $\hat{y}=72.5+2.8x$, what is the predicted response when $x=5$?

Substitute $x=5$: $\hat{y}=72.5+2.8(5)=86.5$.

What does a least-squares regression line estimate?

It estimates the conditional mean of the response variable $y$ for each value of the explanatory variable $x$. The fitted line is calculated from the sample and serves as an estimate of the population relationship.

Simple linear regression

A regression model with one explanatory variable and one response variable, commonly written as $y=\beta_0+\beta_1x+\varepsilon$.

Least-squares regression

A method that chooses regression parameters to minimize the sum of squared residuals, where a residual is the observed response minus the response predicted by the model.

Residual

For an observation, the residual is $e=y-\hat{y}$: the observed response minus the value predicted by the regression line. Residual patterns can reveal that a linear model is inappropriate.

What does the slope in a simple linear regression model represent?

The slope estimates the change in the mean response associated with a one-unit increase in the explanatory variable. It describes a linear association, not necessarily a causal effect.

How should the slope $b$ in $\hat{y}=a+bx$ be interpreted?

For each increase of one unit in $x$, the predicted value of $y$ changes by $b$ units on average. The units of the slope are response units per explanatory-variable unit.

How should the $y$-intercept $a$ in $\hat{y}=a+bx$ be interpreted?

The intercept is the predicted value of $y$ when $x=0$. It is meaningful only if $x=0$ is plausible and relevant to the context; otherwise, it may have little practical interpretation.

What is the difference between an observed response $y$ and a predicted response $\hat{y}$?

The observed response $y$ is the actual measured value, while $\hat{y}$ is the value predicted by the regression line for the corresponding $x$. Their difference is the residual: $e=y-\hat{y}$.

What does a residual plot that shows random scatter around zero indicate?

It supports the use of a linear model because the residuals do not show a systematic pattern. The residuals should be roughly balanced above and below zero with approximately constant spread.

What does a curved pattern in a residual plot indicate?

It indicates that the relationship is not adequately modeled by a straight line. A nonlinear model or transformation may be more appropriate.

What does a fan-shaped pattern in a residual plot indicate?

It indicates that the variability of the response changes as $x$ changes, so the constant-spread condition for a linear model is not satisfied.

What is the coefficient of determination $r^2$?

The coefficient of determination is the proportion of the variability in the response variable $y$ that is explained by the linear regression model using $x$.

How should $r^2$ be interpreted in context?

An $r^2$ value of, for example, $0.64$ means that 64% of the variation in the response variable is explained by the linear relationship with the explanatory variable. The remaining variation is unexplained by the model.

What is the relationship between $r$ and $r^2$ in a linear regression?

$r^2$ is the square of the correlation coefficient $r$, so it is always between 0 and 1. Squaring removes the sign, meaning $r^2$ describes the strength of the linear relationship but not its direction.

What is an influential point in linear regression?

An influential point is an observation whose removal substantially changes the least-squares regression line or a related statistic. Points with extreme $x$-values often have high leverage and can be influential.

How is an outlier different from an influential point in regression?

An outlier is unusual in the response direction, often having a large residual. An influential point has a substantial effect on the fitted line when removed. A point can be an outlier, influential, both, or neither.

What happens to a least-squares regression line when an influential point is removed?

The slope, intercept, predictions, and possibly $r^2$ can change substantially. Therefore, influential points should be investigated for data errors and considered when interpreting the model.

What special point does every least-squares regression line pass through?

The least-squares regression line passes through the point $(\bar{x},\bar{y})$, where $\bar{x}$ and $\bar{y}$ are the sample means of the explanatory and response variables.

What is the sum of the residuals from a least-squares regression line?

The residuals sum to zero, so their mean is also zero, provided the regression line includes an intercept.