Linear regression
One-way clustered OLS
Clustered covariance estimation allows arbitrary error dependence inside declared groups.
The task
Task statement
Estimate y_panel on x1 and x2 with an intercept. Use cluster-robust standard errors clustered on cluster. Report rows const, x1, and x2.
Data
Columns include id, x1, x2, y_iv, x1_iv, z, cluster, y_panel, exposure, and y_count.
- Files
- input.csv
Required output
Scored estimates: const, x1, x2.
The standard errors of const, x1, x2 are also scored, because the main decision in this task is how to compute them.
Reference output
| Term | Reference estimate | Reference standard error | Sample size |
|---|---|---|---|
const | 0.60968147 | 0.12186504 | 2,000 |
x1 | 0.69663586 | 0.029502274 | 2,000 |
x2 | -0.18003081 | 0.027871776 | 2,000 |
A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger, and every scored standard error lies within 0.5 percent of the reference or of an accepted alternative convention. Sample sizes are not scored.
Task success
| Agency condition | Claude Sonnet 4.6 | GPT-5.6 Luna |
|---|---|---|
| Chatbot | 100.0% | 96.7% |
| Self-repair | 100.0% | 100.0% |
| Constrained agent | 100.0% | 98.3% |
- All runs
- 99.2% of 360 runs
- By prompt condition
- Zero-shot 98.3%, One-shot 100.0%
- By software
- Python 100.0%, R 98.3%, Stata 99.2%
Independence between groups
Repeated observations from one firm, school, or person can share shocks. Cluster-robust inference treats whole groups as independent units (Cameron and Miller, 2015).
For cluster $g$, collect its score as $s_g=X_g'\hat u_g$. The covariance meat is a sum of cluster score outer products:
Why within-cluster cancellation matters
Consider one slope and two observations per cluster.
| Cluster | Observation scores | Cluster sum | Squared cluster sum |
|---|---|---|---|
| A | $2,2$ | 4 | 16 |
| B | $3,-3$ | 0 | 0 |
| C | $-1,-2$ | -3 | 9 |
Observation-level squaring would miss the covariance between scores inside each group. Cluster B illustrates cancellation after scores are aggregated.
Assumptions and implementation
Clusters must be independent across groups under the standard large-$G$ argument. The clustering dimension should follow the treatment assignment or error dependence process. Few clusters require additional care.
Software differs in leverage corrections, degrees of freedom, and whether absorbed effects count toward the parameter total. Observations share dependence links inside groups, with no assumed links across groups.
What clustered uncertainty changes
Clustering leaves the OLS coefficient and residuals unchanged. It changes how score contributions enter the covariance matrix. Observations inside a cluster may move together, so the cluster score is summed before its outer product is formed. Cancellation within a cluster can lower a variance estimate, while aligned residuals can raise it. The number of independent clusters governs the effective information for inference.
Report the coefficient, clustered standard error, confidence interval, cluster variable, and cluster count. Name any finite-sample correction. A small standard error does not establish a causal effect. The regression design and identifying assumptions determine that interpretation.
Choosing clusters and checking the calculation
Choose the cluster level from the dependence or assignment process. If treatment is assigned to schools, clustering individual pupils only by classroom can omit school-level dependence. Panel regressions commonly cluster by unit because errors can correlate across time within each unit. Multiway clustering can address separate dependence along two dimensions when the sample supports it.
Verify that each observation has one valid cluster identifier. Compare the software output with cluster score sums and the declared correction factor. With few clusters, asymptotic normal approximations can be inaccurate. Use an appropriate small-sample reference distribution or a wild cluster bootstrap, and report the procedure. Highly unbalanced cluster sizes and high-leverage clusters deserve separate inspection. Removing a large cluster as a casual robustness check changes the target population, so explain any leave-one-cluster analysis. (Cameron and Miller, 2015)
Further reading
Cameron and Miller give an applied guide to cluster-robust inference (Cameron and Miller, 2015). Liang and Zeger develop the estimating-equation foundation (Liang and Zeger, 1986).
Sources and further reading
- A. Colin Cameron, Douglas L. Miller. 2015. “A Practitioner's Guide to Cluster-Robust Inference.” Journal of Human Resources 50(2): 317–372. doi:10.3368/jhr.50.2.317.
- Kung-Yee Liang, Scott L. Zeger. 1986. “Longitudinal Data Analysis Using Generalized Linear Models.” Biometrika 73(1): 13–22. doi:10.1093/biomet/73.1.13.
About this benchmark task
- Method group
- Linear regression
- Software
- Stata, R, and Python
- Runs
- 360 runs, ten in each combination of model, software, prompt, and agency condition