SAT Math 10: Two-Variable Data — Scatterplots and Linear Models

Scatter plot

A graph that uses Cartesian coordinates to display paired values of two variables. Each point represents one observation, with one variable on the horizontal axis and the other on the vertical axis.

How should paired data be represented on a scatter plot?

For each observation, plot the point $(x,y)$ so that the corresponding values of the two variables remain paired. The horizontal coordinate represents $x$, and the vertical coordinate represents $y$.

Which variable is conventionally placed on each axis when one variable predicts or explains the other?

The independent or explanatory variable $x$ is conventionally placed on the horizontal axis. The dependent or response variable $y$ is placed on the vertical axis.

How can a scatter plot indicate a positive association?

A positive association occurs when larger values of $x$ tend to be paired with larger values of $y$, and smaller values of $x$ with smaller values of $y$. The point pattern generally rises from left to right.

How can a scatter plot indicate a negative association?

A negative association occurs when larger values of $x$ tend to be paired with smaller values of $y$. The point pattern generally falls from left to right.

What does it mean if a scatter plot shows no clear direction?

There is no evident association in which high or low values of one variable consistently correspond to particular values of the other. The variables may have little or no association, or their relationship may be nonlinear.

How is the strength of an association judged from a scatter plot?

Strength describes how closely the points follow a consistent pattern, such as a line or curve. For a linear association, points closer to a straight line indicate a stronger relationship.

Why should the overall pattern and deviations from it both be examined in a scatter plot?

The overall pattern reveals the direction, form, and strength of the association, while deviations may reveal outliers, clusters, or other unusual behavior that affects interpretation and predictions.

Why does a perfect horizontal pattern not represent a linear relationship between $x$ and $y$?

A horizontal line means that the predicted value of $y$ does not change as $x$ changes. Thus, despite the points fitting a line perfectly, there is no linear association between the variables.

What kinds of patterns besides linear relationships can appear in scatter plots?

Points may follow curved, power, exponential, or other systematic patterns. A straight-line model is appropriate only when the scatter plot reasonably supports a linear form.

Line of best fit

A line that summarizes a roughly linear pattern in paired data and can be used to estimate the response variable from the explanatory variable. The least-squares line is the standard calculated line of best fit.

When is it reasonable to calculate a linear regression line?

First inspect the scatter plot and verify that a linear pattern is plausible. A regression line is especially appropriate when variation in the explanatory variable can reasonably help predict the response variable.

Least-squares regression line

The line that minimizes the sum of the squared vertical residuals between observed values and predicted values. Its equation is written as $\hat{y}=a+bx$.

What does $\hat{y}$ represent in the regression equation $\hat{y}=a+bx$?

$\hat{y}$, read “y-hat,” is the predicted or estimated value of $y$ produced by the regression line for a specified value of $x$. It generally differs from the observed value $y$.

Residual

The residual for observation $i$ is the difference between the observed response and the predicted response: $\epsilon_i=y_i-\hat{y}_i$. It is the signed vertical distance from the data point to the regression line.

What does the sign of a residual indicate?

If $\epsilon_i>0$, the point lies above the regression line and the line underpredicts $y$. If $\epsilon_i<0$, the point lies below the line and the line overpredicts $y$.

What does the magnitude $|\epsilon_i|$ measure?

It measures the vertical distance between an observed data point and its predicted point on the regression line. Larger absolute residuals indicate less accurate predictions for those observations.

Sum of squared errors (SSE)

The sum of the squared residuals, $\mathrm{SSE}=\sum_{i=1}^{n}\epsilon_i^2=\sum_{i=1}^{n}(y_i-\hat{y}_i)^2$. Least-squares regression chooses the line that makes SSE as small as possible.

Why are residuals squared when finding a least-squares line?

Squaring prevents positive and negative residuals from canceling and gives greater weight to larger deviations. The line with the smallest total squared vertical error is selected.

What is the formula for the intercept of the least-squares regression line?

For $\hat{y}=a+bx$, the intercept is $a=\bar{y}-b\bar{x}$, where $\bar{x}$ and $\bar{y}$ are the sample means.

What is the formula for the slope of the least-squares regression line using deviations from the means?

The slope is $b=\frac{\sum(x-\bar{x})(y-\bar{y})}{\sum(x-\bar{x})^2}$. It describes the average change in predicted $y$ for a one-unit increase in $x$.

What point must every least-squares regression line pass through?

The line always passes through the point $(\bar{x},\bar{y})$, called the point of means.

How can the regression slope be expressed using the correlation coefficient and standard deviations?

The slope is $b=r\left(\frac{s_y}{s_x}\right)$, where $r$ is the correlation coefficient and $s_x$ and $s_y$ are the standard deviations of $x$ and $y$.

How should the slope $b$ of a regression line be interpreted?

For each one-unit increase in the independent variable $x$, the predicted value of the dependent variable $y$ changes by $b$ units on average. The interpretation must include the variables and their units.

What does the intercept $a$ mean in $\hat{y}=a+bx$?

It is the predicted value of $y$ when $x=0$. Its practical meaning may be limited if $x=0$ is outside the observed or scientifically meaningful range.

Why should a regression line generally not be used for extrapolation?

Predictions are most defensible for $x$-values within the range represented by the sample data. Predictions outside that range may be unreliable because the linear pattern may not continue.

What is the correlation coefficient $r$?

The correlation coefficient is a numerical measure of the direction and strength of a linear association between two quantitative variables. It is unitless and satisfies $-1\le r\le 1$.

What does the sign of the correlation coefficient indicate?

A positive $r$ indicates that $y$ tends to increase as $x$ increases, while a negative $r$ indicates that $y$ tends to decrease as $x$ increases. The sign of $r$ matches the sign of the regression slope.

What does the magnitude of $r$ indicate?

Values of $r$ close to $1$ or $-1$ indicate a strong linear association. Values near $0$ indicate a weak or absent linear association, though a nonlinear pattern can also produce $r$ near zero.

What do $r=1$ and $r=-1$ mean?

$r=1$ indicates a perfect positive linear association, and $r=-1$ indicates a perfect negative linear association. In either case, all data points lie exactly on a straight line.

Why must a scatter plot be examined even when the correlation coefficient is known?

The same value of $r$ can arise from different data patterns, and $r$ measures only linear association. A scatter plot can reveal curvature, clusters, outliers, or a horizontal pattern that correlation alone may hide.

Can a correlation coefficient of zero prove that two variables are unrelated?

No. It indicates little or no linear association, but the variables may still have a strong nonlinear relationship.

Coefficient of determination $r^2$

The coefficient of determination is the square of the correlation coefficient: $r^2$. Expressed as a percentage, it represents the proportion of variation in the response variable explained by the linear regression on the explanatory variable.

What information is lost when $r$ is squared to obtain $r^2$?

$r^2$ does not indicate the direction of the association because both positive and negative values of $r$ produce the same $r^2$. The sign must be obtained from $r$ or the regression slope.

How should $1-r^2$ be interpreted in a regression context?

Expressed as a percentage, $1-r^2$ is the proportion of variation in the response variable not explained by the linear regression model. This unexplained variation appears as scatter around the regression line.

A regression analysis gives $r=0.663$. What do $r$ and $r^2$ imply?

The positive value indicates a moderate positive linear association. Since $r^2\approx0.439$, about $44\%$ of the variation in the response is explained by the linear model, while about $56\%$ remains unexplained.

Does a strong correlation establish that one variable causes the other?

No. Correlation describes association, not causation. A third variable, confounding, selection effects, or coincidence may produce the observed relationship.

What is the difference between a response variable and an explanatory variable in regression?

The explanatory variable, often $x$, is used to explain or predict changes in the response variable, often $y$. The response variable is the outcome being estimated.

How can a calculator or statistical program be used to construct a scatter plot and regression line?

Enter paired $x$-values and $y$-values into corresponding data lists, graph the scatter plot, and then run a linear regression procedure to obtain $a$, $b$, $r$, and $r^2$. Graph $\hat{y}=a+bx$ on the same axes to assess the fit visually.