AP Statistics 4.7: Carrying Out a Test for the Difference Between Two Population Means

Matched or paired samples

A study design in which each pair contributes two related measurements, such as before-and-after measurements on the same subject or measurements from deliberately matched individuals. The analysis is performed on the differences within pairs.

What does a paired-samples test actually use from each pair?

It uses the direction and magnitude of the within-pair change, represented by one difference $D_i$. The separate absolute levels of the two measurements matter only insofar as they determine those differences.

When is a paired-samples $t$ test appropriate?

It is appropriate when two measurements are meaningfully linked within each pair, the pairs are obtained through random sampling or an appropriate random design, and the population of differences is approximately normal or the number of pairs is large enough for the sampling distribution of the mean difference to be approximately normal.

What assumptions are needed for the paired-samples $t$ procedure?

The pairs should be randomly sampled or randomly assigned as appropriate, the differences should be independent across pairs, and the distribution of differences should be approximately normal unless $n$ is sufficiently large. Normality applies to the differences, not necessarily to each original measurement list separately.

What does it mean for pairs to be independent across pairs?

The difference contributed by one pair should not influence the difference contributed by another pair. Measurements within a pair are allowed, and usually expected, to be dependent; independence is required between pairs.

How should the difference list be constructed for paired observations?

Match the two measurements belonging to the same subject or object, then subtract in the same order for every pair. For example, use $D_i = \text{after}_i - \text{before}_i$ for all $i$.

What does the sign of a paired difference mean?

Its interpretation depends on the subtraction order. If $D = \text{after} - \text{before}$, a negative mean difference indicates a decrease after treatment, whereas a positive mean difference indicates an increase.

What is the relationship between the mean of the paired differences and the two sample means?

If $D_i = Y_{i,2} - Y_{i,1}$, then $\bar{D} = \bar{Y}_2 - \bar{Y}_1$. Thus the observed mean change is the same whether it is calculated by averaging pairwise differences or subtracting the two sample means.

What population parameter is tested in a paired-samples $t$ test?

The parameter is the population mean difference, denoted $\mu_d$. The null and alternative hypotheses concern this single mean, so the procedure is mathematically a one-sample $t$ test applied to the differences.

What does a null mean difference of zero represent in a paired comparison?

It represents no average change or no average difference between the two conditions. Individual pairs may differ, but their population mean difference is hypothesized to be zero.

How should hypotheses be written for a paired test when a treatment is expected to reduce a measurement?

If the difference is defined as $D = \text{after} - \text{before}$, use $H_0: \mu_d \ge 0$ and $H_a: \mu_d < 0$. The alternative represents a lower average value after treatment.

How should hypotheses be written for a paired test when a treatment is expected to increase a measurement?

With $D = \text{after} - \text{before}$, use $H_0: \mu_d \le 0$ and $H_a: \mu_d > 0$. The alternative represents a positive average change.

How should hypotheses be written when the question asks whether two paired conditions differ in either direction?

Use a two-sided test, typically $H_0: \mu_d = 0$ versus $H_a: \mu_d \ne 0$. The direction of the observed mean difference does not determine the alternative after the test is chosen.

Paired-samples $t$ statistic

For $n$ paired differences, the test statistic is $t = \frac{\bar{x}_d - \mu_{d,0}}{s_d/\sqrt{n}}$, where $\bar{x}_d$ is the sample mean difference, $s_d$ is the sample standard deviation of the differences, and $\mu_{d,0}$ is the null-hypothesized mean difference.

How are the degrees of freedom determined for a paired-samples $t$ test?

The test uses $df = n - 1$, where $n$ is the number of matched pairs, equivalently the number of calculated differences.

How does sample size enter the standard error for a paired mean difference?

The standard error is $SE_{\bar{d}} = s_d/\sqrt{n}$. Increasing the number of pairs decreases the standard error, while greater variation among the differences increases it.

How is the p-value used in a paired-samples hypothesis test?

The p-value is the probability, assuming the null hypothesis is true, of obtaining a mean difference or test statistic at least as extreme as the observed one in the direction specified by the alternative. Reject $H_0$ when $p \le \alpha$; otherwise, do not reject $H_0$.

What is the correct conclusion after failing to reject the null hypothesis in a paired test?

State that the sample does not provide sufficient evidence for the claimed mean change or difference at the chosen significance level. Do not conclude that the null hypothesis has been proven true.

In a before-and-after pain study with $D = \text{after} - \text{before}$, what does $\bar{x}_d < 0$ suggest?

It suggests that pain scores decreased after the intervention on average. Statistical significance still requires comparing the resulting $t$ statistic or p-value with the chosen test criterion.

A paired study has eight pairs, with $\bar{x}_d = -3.13$, $s_d = 2.91$, and $\mu_{d,0}=0$. What is the approximate test statistic?

Using $t = \frac{\bar{x}_d - \mu_{d,0}}{s_d/\sqrt{n}}$, $t \approx \frac{-3.13}{2.91/\sqrt{8}} \approx -3.04$, with $df=7$.

How would a paired test interpret a result of $p=0.0095$ at $\alpha=0.05$ for a one-sided reduction hypothesis?

Because $0.0095 < 0.05$, reject $H_0$. The data provide sufficient evidence that the population mean difference is negative, supporting an average reduction.

Why can one extreme paired difference strongly affect the result?

The mean and standard deviation are calculated from the differences, so an outlier difference can shift $\bar{x}_d$, inflate $s_d$, or both. This can alter the test statistic and may also undermine the normality assumption.

Why can a paired test be useful when the sample size is small?

Pairing removes much of the subject-to-subject variation, so the differences may have a smaller spread than the original measurements. However, with a small number of pairs, the normality of the differences and the influence of outliers require particular attention.

Why should repeated measurements from the same subject not usually be analyzed as independent samples?

Measurements from the same subject are generally correlated, so treating them as independent can misrepresent the variability of their difference. A paired analysis accounts for this dependence directly.

When is an independent-samples test preferable to a paired test?

Use an independent-samples procedure when observations in the two groups are genuinely unrelated and there is no meaningful one-to-one matching. Artificially pairing unrelated observations can introduce arbitrary dependence and distort the analysis.

What is an independent two-sample $t$ test used to compare?

It tests whether two independent populations have different means. The parameter of interest is typically $\mu_1-\mu_2$, the difference between the two population means.

What conditions are required for an independent two-sample $t$ test?

The two samples should be independent random samples or come from a suitable randomized design, the observations within each sample should be independent, and each population should be approximately normal or both sample sizes should be sufficiently large. Strong skewness and outliers are especially concerning for small samples.

What is the standard error for the difference between two independent sample means?

For independent samples, the estimated standard error is $SE_{\bar{x}_1-\bar{x}_2}=\sqrt{\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}}$, where $s_1$ and $s_2$ are the sample standard deviations and $n_1$ and $n_2$ are the sample sizes.

Independent two-sample $t$ statistic

The test statistic is $t=\frac{(\bar{x}_1-\bar{x}_2)-\mu_0}{\sqrt{\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}}}$. Usually $\mu_0=0$, representing no difference between the population means.

How are degrees of freedom found for an independent two-sample $t$ test?

Use the calculator's two-sample $t$ procedure to determine the degrees of freedom, typically using Welch's approximation. If equal population variances are specifically assumed and a pooled procedure is used, $df=n_1+n_2-2$.

How should hypotheses be written for an independent two-sample test?

For no difference versus a difference in either direction, use $H_0:\mu_1-\mu_2=0$ and $H_a:\mu_1-\mu_2\ne0$. For a directional claim, use $H_a:\mu_1-\mu_2>0$ or $H_a:\mu_1-\mu_2<0$, matching the order of subtraction.

How should the conclusion of a two-sample $t$ test be stated in context?

State the decision about $H_0$, then describe what the evidence indicates about the difference in population means in the original context. Include the significance level and avoid claiming that a statistically significant association proves causation unless the study design supports a causal conclusion.

How do you choose between a paired and an independent two-sample $t$ test?

Use a paired test when each observation in one condition is linked to exactly one observation in the other condition and analyze the within-pair differences. Use an independent two-sample test when the observations come from unrelated groups with no meaningful matching.

What is the main distinction between a paired-samples $t$ test and a paired $Z$ test?

A paired $t$ test estimates the standard deviation of the differences from the sample and uses a $t$ distribution with $n-1$ degrees of freedom. A paired $Z$ test assumes the relevant population variance is known and uses the standard normal distribution.