AI Agents & Econometric Coding

Instrumental variables

Shift-share IV

Shift-share IV with exposure-robust inference.

The task

Task statement

Estimate the effect of regional employment growth x on wage growth y using external industry demand shocks.
input.csv has one row per region-industry pair. The columns are region_id, industry_id, share, shock, region_weight, x, and y.
The baseline employment shares sum to one within each region. shock is constant within industry. region_weight, x, and y are repeated within region.
Use all regions and industries. Normalize the regional weights once, using one observation per region: e_r = region_weight_r / sum_r region_weight_r.
Construct the regional shift-share instrument z_r = sum_n share_rn * shock_n. Estimate weighted 2SLS of y_r on an intercept and x_r, instrumenting x_r with z_r and weighting by e_r.

Identification comes from independent industry shocks. Obtain exposure-robust uncertainty through the Borusyak-Hull-Jaravel shock-level aggregation, with the following conventions.
First subtract each variable's e_r-weighted regional mean from y_r, x_r, and z_r. Denote the residuals y0_r, x0_r, and z0_r.
For each industry n, construct s_n = sum_r e_r*share_rn and aggregate each residual as vbar_n = sum_r e_r*share_rn*v0_r / s_n.
Fit the shock-level weighted IV regression of ybar_n on an intercept and xbar_n, instrumenting xbar_n with shock_n and weighting by s_n. Its slope equals the regional 2SLS slope.
Treat industries as the independent observations. Use heteroskedasticity-robust HC1 standard errors from this shock-level regression, with variance multiplier N/(N-2), where N is the number of industries. Use structural IV residuals and analytic weights, without clustering.

To assess instrument relevance, also estimate the regional first-stage slope of x_r on an intercept and z_r using e_r weights. Obtain its exposure-robust standard error from the equivalent shock-level IV regression of xbar_n on an intercept and zbar_n, instrumenting zbar_n with shock_n and weighting by s_n. Use the same HC1 convention. This shock-level regression has zbar_n as its endogenous regressor.
To describe how widely the identifying exposure is distributed, compute effective_shocks = 1/sum_n s_n^2.

Report three rows: effect (the regional 2SLS slope with its shock-level HC1 SE), first_stage (the regional first-stage slope with its shock-level HC1 SE), and effective_shocks (with se=null).
For every row, report n_obs as the number of industries used in the shock-level calculation.

Data

One row per region-industry pair: region_id, industry_id, baseline share, external industry shock, regional analysis weight, regional employment growth x, and regional wage growth y.

Files
input.csv
Seed
2026091503

Required output

Scored estimates: effect, first_stage, effective_shocks.

The standard errors of effect, first_stage are also scored, because the main decision in this task is how to compute them.

Reference output

TermReference estimateReference standard errorSample size
effect0.489245090.4919970948
first_stage1.31271740.2574916148
effective_shocks32.024626not scored48

A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger, and every scored standard error lies within 0.001 percent of the reference or of an accepted alternative convention. Sample sizes are not scored.

Task success

Agency conditionClaude Sonnet 4.6GPT-5.6 Luna
Chatbot60.0%60.0%
Self-repair93.3%65.0%
Constrained agent91.7%66.7%
All runs
72.8% of 360 runs
By prompt condition
Zero-shot 63.9%, One-shot 81.7%
By software
Python 85.0%, R 83.3%, Stata 50.0%

About this benchmark task

Method group
Instrumental variables
Software
Stata, R, and Python
Runs
360 runs, ten in each combination of model, software, prompt, and agency condition

Open every recorded run for this task.