AI Agents & Econometric Coding

Instrumental variables

Multi-vintage exposure IV

Trade-exposure IV with different employment vintages.

The task

Task statement

Use every CSV supplied for this task. Join trade.code to crosswalk.code. Missing area-industry employment cells mean zero. There are four industries. National industry employment is the sum across all supplied areas, within period and vintage. For each area-period, construct x=sum_j[(base area-industry employment/base total_employment)*(us_change/national base industry employment)/(price_index/100)/1000*(10/years)]. Construct z by replacing base with early and us_change with other_change everywhere. Trade changes are nominal dollars over the stated interval. The resulting exposures are thousands of real dollars per worker per decade. Outcome change_pp is already a decadal percentage-point change. Estimate weighted 2SLS on all 72 area-period observations: change_pp on an intercept, period, control, and x. Instrument x with z. Include all three exogenous columns as instruments. Use base-vintage population as the regression weight. Report the x coefficient, x for area 0 in each period, and z for area 0 in period 0. The task defines its denominator and interval conventions explicitly. Each code maps to exactly one industry. Within each period, each industry has one trade-code row. Price_index and years are constant within period. All national industry employment denominators are positive. Set n_obs to the number of area-period estimation rows.

Data

crosswalk.csv: code = Trade-code key for the industry crosswalk; industry = Industry identifier trade.csv: period = Calendar period or interval identifier as defined in the task; code = Trade-code key for the industry crosswalk; us_change = Nominal import change for the focal country in dollars; other_change = Nominal import change in the instrument countries, in dollars; price_index = Price deflator, with base value 100; years = Interval length in years areas.csv: area = Synthetic area identifier; period = Calendar period or interval identifier as defined in the task; vintage = Employment measurement date: base or early; total_employment = Total employment, including industries outside the listed manufacturing sectors; population = Base-vintage population used as the regression weight employment.csv: area = Synthetic area identifier; period = Calendar period or interval identifier as defined in the task; vintage = Employment measurement date: base or early; industry = Industry identifier; employment = Area-industry employment; omitted cells contain zero outcomes.csv: area = Synthetic area identifier; period = Calendar period or interval identifier as defined in the task; control = Supplied exogenous control; change_pp = Outcome change already expressed in percentage points per decade

Files
input.csv, crosswalk.csv, trade.csv, areas.csv, employment.csv, outcomes.csv

Required output

Scored estimates: iv_slope, exposure_area0_period0, exposure_area0_period1, instrument_area0_period0.

Reference output

TermReference estimateSample size
iv_slope-1.40342272
exposure_area0_period026.9341572
exposure_area0_period137.78319372
instrument_area0_period029.94430372

A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger. Sample sizes are not scored.

Task success

Agency conditionClaude Sonnet 4.6GPT-5.6 Luna
Chatbot75.0%30.0%
Self-repair100.0%61.7%
Constrained agent98.3%75.0%
All runs
73.3% of 360 runs
By prompt condition
Zero-shot 72.8%, One-shot 73.9%
By software
Python 80.0%, R 81.7%, Stata 58.3%

Sources

Sources

About this benchmark task

Method group
Instrumental variables
Software
Stata, R, and Python
Runs
360 runs, ten in each combination of model, software, prompt, and agency condition

Open every recorded run for this task.