Instrumental variables
Shift-share IV
Shift-share IV with exposure-robust inference.
The task
Task statement
Estimate the effect of regional employment growth x on wage growth y using external industry demand shocks. input.csv has one row per region-industry pair. The columns are region_id, industry_id, share, shock, region_weight, x, and y. The baseline employment shares sum to one within each region. shock is constant within industry. region_weight, x, and y are repeated within region. Use all regions and industries. Normalize the regional weights once, using one observation per region: e_r = region_weight_r / sum_r region_weight_r. Construct the regional shift-share instrument z_r = sum_n share_rn * shock_n. Estimate weighted 2SLS of y_r on an intercept and x_r, instrumenting x_r with z_r and weighting by e_r. Identification comes from independent industry shocks. Obtain exposure-robust uncertainty through the Borusyak-Hull-Jaravel shock-level aggregation, with the following conventions. First subtract each variable's e_r-weighted regional mean from y_r, x_r, and z_r. Denote the residuals y0_r, x0_r, and z0_r. For each industry n, construct s_n = sum_r e_r*share_rn and aggregate each residual as vbar_n = sum_r e_r*share_rn*v0_r / s_n. Fit the shock-level weighted IV regression of ybar_n on an intercept and xbar_n, instrumenting xbar_n with shock_n and weighting by s_n. Its slope equals the regional 2SLS slope. Treat industries as the independent observations. Use heteroskedasticity-robust HC1 standard errors from this shock-level regression, with variance multiplier N/(N-2), where N is the number of industries. Use structural IV residuals and analytic weights, without clustering. To assess instrument relevance, also estimate the regional first-stage slope of x_r on an intercept and z_r using e_r weights. Obtain its exposure-robust standard error from the equivalent shock-level IV regression of xbar_n on an intercept and zbar_n, instrumenting zbar_n with shock_n and weighting by s_n. Use the same HC1 convention. This shock-level regression has zbar_n as its endogenous regressor. To describe how widely the identifying exposure is distributed, compute effective_shocks = 1/sum_n s_n^2. Report three rows: effect (the regional 2SLS slope with its shock-level HC1 SE), first_stage (the regional first-stage slope with its shock-level HC1 SE), and effective_shocks (with se=null). For every row, report n_obs as the number of industries used in the shock-level calculation.
Data
One row per region-industry pair: region_id, industry_id, baseline share, external industry shock, regional analysis weight, regional employment growth x, and regional wage growth y.
- Files
- input.csv
- Seed
- 2026091503
Required output
Scored estimates: effect, first_stage, effective_shocks.
The standard errors of effect, first_stage are also scored, because the main decision in this task is how to compute them.
Reference output
| Term | Reference estimate | Reference standard error | Sample size |
|---|---|---|---|
effect | 0.48924509 | 0.49199709 | 48 |
first_stage | 1.3127174 | 0.25749161 | 48 |
effective_shocks | 32.024626 | not scored | 48 |
A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger, and every scored standard error lies within 0.001 percent of the reference or of an accepted alternative convention. Sample sizes are not scored.
Task success
| Agency condition | Claude Sonnet 4.6 | GPT-5.6 Luna |
|---|---|---|
| Chatbot | 60.0% | 60.0% |
| Self-repair | 93.3% | 65.0% |
| Constrained agent | 91.7% | 66.7% |
- All runs
- 72.8% of 360 runs
- By prompt condition
- Zero-shot 63.9%, One-shot 81.7%
- By software
- Python 85.0%, R 83.3%, Stata 50.0%
About this benchmark task
- Method group
- Instrumental variables
- Software
- Stata, R, and Python
- Runs
- 360 runs, ten in each combination of model, software, prompt, and agency condition