AI Agents & Econometric Coding

Instrumental variables

Weighted 2SLS spillover effects

Population-weighted spillovers among nonrecipients.

The task

Task statement

Use households.csv and census.csv. Keep all households with recipient=0. This includes ineligible households in both village treatment statuses. Fit weighted 2SLS with two endogenous columns dose*(1-eligible) and dose*eligible. Exogenous columns are 1-eligible, eligible, baseline*(1-eligible), and baseline*eligible. Instruments are these four columns plus instrument*(1-eligible) and instrument*eligible. Add no other intercept. Use weight as the regression weight. For each eligibility group, the spillover effect is its dose slope times the household-weighted mean dose among its retained observations. Report ineligible_effect and eligible_nonrecipient_effect. Average these two effects using nonrecipient_population from census.csv. Report population_effect. The counterfactual sets dose to zero for every retained household. Census totals determine only the final between-group weights. For every result, n_obs is the number of nonrecipient household rows used in estimation. The group effect equals the fitted dose slope times its weighted mean dose. Attribution to a zero-dose counterfactual is a stated synthetic convention.

Data

households.csv: household = Synthetic household identifier; eligible = Eligibility-group indicator, zero or one; treated = Own village treatment indicator; recipient = Indicator for actually receiving the transfer; instrument = Supplied excluded instrument for dose; dose = Supplied treatment exposure; baseline = Supplied baseline covariate; weight = Positive observation weight for estimation and within-group averages; outcome = Synthetic dependent variable in the stated regression census.csv: eligible = Eligibility-group indicator, zero or one; nonrecipient_population = Census count used only for final group aggregation

Files
input.csv, households.csv, census.csv

Required output

Scored estimates: ineligible_effect, eligible_nonrecipient_effect, population_effect.

Reference output

TermReference estimateSample size
ineligible_effect2.0530515300
eligible_nonrecipient_effect5.7819293300
population_effect2.649672300

A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger. Sample sizes are not scored.

Task success

Agency conditionClaude Sonnet 4.6GPT-5.6 Luna
Chatbot80.0%81.7%
Self-repair100.0%100.0%
Constrained agent100.0%100.0%
All runs
93.6% of 360 runs
By prompt condition
Zero-shot 94.4%, One-shot 92.8%
By software
Python 88.3%, R 97.5%, Stata 95.0%

Sources

Sources

About this benchmark task

Method group
Instrumental variables
Software
Stata, R, and Python
Runs
360 runs, ten in each combination of model, software, prompt, and agency condition

Open every recorded run for this task.