Instrumental variables
Multi-vintage exposure IV
Trade-exposure IV with different employment vintages.
The task
Task statement
Use every CSV supplied for this task. Join trade.code to crosswalk.code. Missing area-industry employment cells mean zero. There are four industries. National industry employment is the sum across all supplied areas, within period and vintage. For each area-period, construct x=sum_j[(base area-industry employment/base total_employment)*(us_change/national base industry employment)/(price_index/100)/1000*(10/years)]. Construct z by replacing base with early and us_change with other_change everywhere. Trade changes are nominal dollars over the stated interval. The resulting exposures are thousands of real dollars per worker per decade. Outcome change_pp is already a decadal percentage-point change. Estimate weighted 2SLS on all 72 area-period observations: change_pp on an intercept, period, control, and x. Instrument x with z. Include all three exogenous columns as instruments. Use base-vintage population as the regression weight. Report the x coefficient, x for area 0 in each period, and z for area 0 in period 0. The task defines its denominator and interval conventions explicitly. Each code maps to exactly one industry. Within each period, each industry has one trade-code row. Price_index and years are constant within period. All national industry employment denominators are positive. Set n_obs to the number of area-period estimation rows.
Data
crosswalk.csv: code = Trade-code key for the industry crosswalk; industry = Industry identifier trade.csv: period = Calendar period or interval identifier as defined in the task; code = Trade-code key for the industry crosswalk; us_change = Nominal import change for the focal country in dollars; other_change = Nominal import change in the instrument countries, in dollars; price_index = Price deflator, with base value 100; years = Interval length in years areas.csv: area = Synthetic area identifier; period = Calendar period or interval identifier as defined in the task; vintage = Employment measurement date: base or early; total_employment = Total employment, including industries outside the listed manufacturing sectors; population = Base-vintage population used as the regression weight employment.csv: area = Synthetic area identifier; period = Calendar period or interval identifier as defined in the task; vintage = Employment measurement date: base or early; industry = Industry identifier; employment = Area-industry employment; omitted cells contain zero outcomes.csv: area = Synthetic area identifier; period = Calendar period or interval identifier as defined in the task; control = Supplied exogenous control; change_pp = Outcome change already expressed in percentage points per decade
- Files
- input.csv, crosswalk.csv, trade.csv, areas.csv, employment.csv, outcomes.csv
Required output
Scored estimates: iv_slope, exposure_area0_period0, exposure_area0_period1, instrument_area0_period0.
Reference output
| Term | Reference estimate | Sample size |
|---|---|---|
iv_slope | -1.403422 | 72 |
exposure_area0_period0 | 26.93415 | 72 |
exposure_area0_period1 | 37.783193 | 72 |
instrument_area0_period0 | 29.944303 | 72 |
A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger. Sample sizes are not scored.
Task success
| Agency condition | Claude Sonnet 4.6 | GPT-5.6 Luna |
|---|---|---|
| Chatbot | 75.0% | 30.0% |
| Self-repair | 100.0% | 61.7% |
| Constrained agent | 98.3% | 75.0% |
- All runs
- 73.3% of 360 runs
- By prompt condition
- Zero-shot 72.8%, One-shot 73.9%
- By software
- Python 80.0%, R 81.7%, Stata 58.3%
Sources
Sources
About this benchmark task
- Method group
- Instrumental variables
- Software
- Stata, R, and Python
- Runs
- 360 runs, ten in each combination of model, software, prompt, and agency condition