Linear regression
Two-way clustered OLS
Two-way clustering handles dependence along two nonnested dimensions through an inclusion-exclusion covariance.
The task
Task statement
Estimate y_twocluster on x with an intercept. Use two-way cluster-robust standard errors clustered on firm and year using the Cameron-Gelbach-Miller formula. Report rows const and x.
Data
Panel of 200 firms by 10 years. Columns include firm, year, x, y_twocluster, cohort, treat_post, y_did, z_sel, w, selected, and y_sel.
- Files
- input.csv
Required output
Scored estimates: const, x.
The standard errors of const, x are also scored, because the main decision in this task is how to compute them.
Reference output
| Term | Reference estimate | Reference standard error | Sample size |
|---|---|---|---|
const | 1.017456 | 0.15347234 | 2,000 |
x | 0.5531899 | 0.016647984 | 2,000 |
A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger, and every scored standard error lies within 0.5 percent of the reference or of an accepted alternative convention. Sample sizes are not scored.
Task success
| Agency condition | Claude Sonnet 4.6 | GPT-5.6 Luna |
|---|---|---|
| Chatbot | 73.3% | 61.7% |
| Self-repair | 80.0% | 78.3% |
| Constrained agent | 91.7% | 93.3% |
- All runs
- 79.7% of 360 runs
- By prompt condition
- Zero-shot 85.6%, One-shot 73.9%
- By software
- Python 90.8%, R 90.8%, Stata 57.5%
Dependence along rows and columns
Panel errors can share firm shocks across years and year shocks across firms. Cameron, Gelbach, and Miller combine three one-way covariance estimates (Cameron et al., 2011).
The subtraction removes the intersection contribution counted in both first terms.
A covariance map
Consider four observations in a two-by-two panel.
| Year 1 | Year 2 | |
|---|---|---|
| Firm A | $u_{A1}$ | $u_{A2}$ |
| Firm B | $u_{B1}$ | $u_{B2}$ |
The firm term allows covariance within each row. The year term allows covariance within each column. The intersection term subtracts one duplicated copy of each cell's own variance.
Assumptions and implementation
The two dimensions must define the intended dependence structure. Standard consistency requires the relevant cluster counts to grow. Persistent common time shocks can need stronger methods than basic two-way clustering.
Finite-sample factors may use a separate cluster count for each component. Software can also use a heteroskedasticity-robust intersection term when each intersection contains one observation. Row, column, and cell panels make the inclusion-exclusion calculation visible.
Interpretation and reporting
Two-way clustering leaves the regression coefficient unchanged. It combines uncertainty from dependence within firms and within years. The intersection subtraction prevents each observation's own contribution from appearing twice.
Report both cluster counts and the count of nonempty firm-year cells. State the finite-sample correction for every covariance component. A single standard-error label is incomplete when software defaults can produce different component scales.
The method assumes that the two labels capture the relevant dependence. Few firms or few years can make conventional inference inaccurate. Strong aggregate dynamics may require methods that model time dependence beyond a basic year cluster.
Reproducible implementation
Fit the model once on a fixed sample and retain observation-level residual scores. Aggregate those scores by firm, by year, and by ordered firm-year pair.
Build three sandwich covariances with their documented correction factors. Add the firm and year matrices, then subtract the intersection matrix. Check the final matrix for symmetry and admissible diagonal elements.
Verify that row reordering does not alter results. If every intersection is a singleton, compare its component with the matching heteroskedasticity-robust covariance. Also reproduce each one-way covariance separately. These component checks locate label errors and mismatched software conventions (Cameron et al., 2011).
Further reading
Cameron, Gelbach, and Miller derive multiway clustered covariance estimation (Cameron et al., 2011). Petersen studies firm and time dependence in finance panels (Petersen, 2009).
Sources and further reading
- A. Colin Cameron, Jonah B. Gelbach, Douglas L. Miller. 2011. “Robust Inference with Multiway Clustering.” Journal of Business and Economic Statistics 29(2): 238–249. doi:10.1198/jbes.2010.07136.
- Mitchell A. Petersen. 2009. “Estimating Standard Errors in Finance Panel Data Sets: Comparing Approaches.” Review of Financial Studies 22(1): 435–480. doi:10.1093/rfs/hhn053.
About this benchmark task
- Method group
- Linear regression
- Software
- Stata, R, and Python
- Runs
- 360 runs, ten in each combination of model, software, prompt, and agency condition