Classical tests · Other
Welch's Two-Sample Test
Welch's test compares two means without imposing equal population variances.
A mean difference with separate variance estimates
For independent groups, Welch's statistic is (Welch, 1947):
Its degrees of freedom use the Welch-Satterthwaite approximation.
A calculation
| Group | Mean | Variance | Size |
|---|---|---|---|
| 1 | 10 | 4 | 20 |
| 2 | 8 | 9 | 15 |
The standard error is $\sqrt{4/20+9/15}=0.894$. The statistic is $2/0.894=2.236$. The approximate degrees of freedom are 23.0.
Assumptions and implementation
The groups must be independent. Each group mean needs enough observations for the approximation. Strong skewness or heavy tails can motivate randomization or robust alternatives.
Software can default to the pooled equal-variance test. Side-by-side distributions reveal each mean, spread, and standard-error contribution.
Interpretation and reporting
The observed mean difference is $10-8=2$. Welch's denominator combines the two variance contributions without pooling them. The resulting statistic is $2.236$, with about $23.0$ reference degrees of freedom.
State the subtraction order so the sign has a clear meaning. Report both group means, standard deviations, sample sizes, the mean difference, and its confidence interval. Keep the fractional degrees of freedom because rounding it to a group size changes the approximation.
Welch's method handles unequal variances. It does not protect a mean comparison from severe outliers or dependence between observations. Paired data require a paired analysis, while clustered samples require covariance that reflects their grouping.
Define the sampling unit before applying the test. Repeated measurements from one person or firm are dependent. Analyze within-unit differences or use a model that represents that pairing. Welch's variance adjustment cannot recover covariance that the calculation omitted.
Reproducible implementation
Define the two independent groups and remove missing outcomes within each group. Compute means and unbiased sample variances. Insert these values into the standard-error and degrees-of-freedom formulas.
Compare the manual statistic with a software call that explicitly disables equal-variance pooling. Confirm the group order used by the command. Reversing it should change the statistic's sign and preserve its magnitude.
Plot each group distribution and inspect extreme values. For small, highly skewed samples, add a justified randomization or robust-location analysis. Save the group coding and complete-case counts with the result (Welch, 1947).
Further reading
Welch derives the unequal-variance comparison (Welch, 1947). Satterthwaite supplies the degrees-of-freedom approximation (Satterthwaite, 1946).
Source status
- Welch 1947 and Satterthwaite 1946 remain manual-access primary sources.
Sources and further reading
- Bernard L. Welch. 1947. “The Generalization of Student's Problem When Several Different Population Variances Are Involved.” Biometrika 34(1/2): 28--35. doi:10.2307/2332510.
- Franklin E. Satterthwaite. 1946. “An Approximate Distribution of Estimates of Variance Components.” Biometrics Bulletin 2(6): 110--114. doi:10.2307/3002019.
About this benchmark task
- Status
- In the benchmark
- Identifier
ttest_welch- Family
- Other
- Software
- Stata, R, Python
- Source
- Benchmark task set
Task statement
Compare y between group 1 and group 0 using Welch's unequal-variance two-sample t-test. Define the t statistic as mean(y | group=1) minus mean(y | group=0), divided by the Welch standard error. Use the Welch-Satterthwaite approximation for the degrees of freedom. Report rows t, df, and p_value.
Required output
t, df, p_value