Instrumental variables
Two-stage least squares
Instrumental variables isolate variation in an endogenous regressor that comes from an excluded instrument.
The task
Task statement
Estimate y_iv on endogenous x1_iv with an intercept, using z as the excluded instrument. Use a proper 2SLS routine with HC1 robust standard errors. Report rows const and x1_iv.
Data
Columns include id, x1, x2, y_iv, x1_iv, z, cluster, y_panel, exposure, and y_count.
- Files
- input.csv
Required output
Scored estimates: const, x1_iv.
The standard errors of x1_iv are also scored, because the main decision in this task is how to compute them.
Reference output
| Term | Reference estimate | Reference standard error | Sample size |
|---|---|---|---|
const | 0.96962015 | not scored | 2,000 |
x1_iv | 0.79166834 | 0.043202028 | 2,000 |
A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger, and every scored standard error lies within 0.1 percent of the reference or of an accepted alternative convention. Sample sizes are not scored.
Task success
| Agency condition | Claude Sonnet 4.6 | GPT-5.6 Luna |
|---|---|---|
| Chatbot | 85.0% | 90.0% |
| Self-repair | 91.7% | 96.7% |
| Constrained agent | 100.0% | 98.3% |
- All runs
- 93.6% of 360 runs
- By prompt condition
- Zero-shot 91.7%, One-shot 95.6%
- By software
- Python 83.3%, R 99.2%, Stata 98.3%
Replacing endogenous variation
An instrument $z$ must predict the endogenous regressor $x$ and remain excluded from the structural error. Two-stage least squares projects $x$ onto the instrument space (Angrist and Pischke, 2009).
A Wald-ratio example
With one binary instrument and one endogenous regressor, the slope can be written as a ratio of group differences.
| Instrument group | Mean $x$ | Mean $y$ |
|---|---|---|
| $z=0$ | 2 | 5 |
| $z=1$ | 5 | 11 |
The reduced-form difference is $6$. The first-stage difference is $3$. The IV estimate is $6/3=2$.
Assumptions
Relevance requires a nonzero first stage. Exogeneity requires the instrument to be orthogonal to the structural error. Exclusion requires no outcome path outside the endogenous variable. In causal settings, monotonicity supports a local average treatment effect interpretation.
Weak instruments can make conventional normal approximations unreliable. Overidentification does not prove instrument validity.
Robust covariance
Heteroskedasticity-robust IV covariance uses structural residuals and projected regressors. HC0 and HC1 differ by a finite-sample scale. Software also differs in whether it applies this scale by default.
The first-stage projection splits $x$ into instrument-predicted and residual components. Only the predicted component identifies the second-stage coefficient.
What the IV coefficient represents
Two-stage least squares uses the part of $x$ predicted by the excluded instruments. In the just-identified binary case, the coefficient equals the reduced-form outcome effect divided by the first-stage treatment effect. Its causal meaning requires instrument relevance, exclusion, independence, and a suitable treatment-response framework. With heterogeneous effects and monotonic treatment selection, the estimate can describe a local average treatment effect for compliers.
Report the first-stage coefficient, reduced form, 2SLS estimate, and robust confidence interval. State which regressors are endogenous and which instruments are excluded. A large 2SLS coefficient can arise from a small first stage, so the denominator needs equal attention.
Identification and robust inference
Construct the instrument matrix with every included exogenous regressor plus the excluded instruments. Verify matrix rank and use structural residuals in the heteroskedasticity-robust covariance. The first-stage fitted values define the coefficient calculation. Regressing the outcome directly on fitted values with ordinary second-stage standard errors gives incorrect uncertainty.
Inspect first-stage strength and report a weak-instrument diagnostic suited to the model. When strength is limited, use Anderson-Rubin or other weak-IV-robust confidence sets (Andrews et al., 2019). Robust covariance handles heteroskedasticity. It does not address weak identification, invalid exclusion, or clustered errors. Cluster the score contributions when instrument and outcome shocks share group-level dependence. Discuss the substantive path from each instrument to the outcome because overidentification tests cannot verify exclusion by themselves.
Further reading
Angrist and Pischke give an applied treatment of IV and local effects (Angrist and Pischke, 2009). Andrews, Stock, and Sun review weak-instrument inference (Andrews et al., 2019).
Sources and further reading
- Joshua D. Angrist, Jorn-Steffen Pischke. 2009. Mostly Harmless Econometrics: An Empiricist's Companion. Princeton University Press. Source.
- Isaiah Andrews, James H. Stock, Liyang Sun. 2019. “Weak Instruments in Instrumental Variables Regression: Theory and Practice.” Annual Review of Economics 11: 727–753. doi:10.1146/annurev-economics-080218-025643.
About this benchmark task
- Method group
- Instrumental variables
- Software
- Stata, R, and Python
- Runs
- 360 runs, ten in each combination of model, software, prompt, and agency condition