AI Agents & Econometric Coding

Regression discontinuity

Fuzzy regression discontinuity

Training eligibility and participation effects.

The task

Task statement

A training program becomes more accessible when an applicant's eligibility score reaches zero. The file contains id, score, participated (0 or 1), and earnings. Estimate the effect of participation at the eligibility cutoff using fuzzy regression discontinuity.

Use cutoff zero and bandwidth 1 on each side. Include observations with abs(score)<1. Use triangular weights 1-abs(score). Fit separate local-linear slopes on the two sides, with intercepts. Use the same observations and weights for earnings and participation. Report the earnings discontinuity as reduced_form, the participation discontinuity as first_stage, and their ratio as treatment_effect. Each discontinuity is the right limit minus the left limit.

Report conventional heteroskedasticity-robust HC0 standard errors with no degrees-of-freedom correction. Treat the bandwidth and weights as fixed. For treatment_effect, use the delta method and the joint covariance of the two discontinuity estimates from the same observations. Account for their covariance. Do not apply bias correction. Report n_obs as the total number of observations inside the bandwidth for every row.

Data

id, score centered at eligibility cutoff, participated, earnings. Simulated applicant data.

Files
input.csv
Seed
2026091501

Required output

Scored estimates: treatment_effect, reduced_form, first_stage.

The standard errors of treatment_effect are also scored, because the main decision in this task is how to compute them.

Reference output

TermReference estimateReference standard errorSample size
treatment_effect1.94107330.6895292598
reduced_form0.85992217not scored598
first_stage0.44301376not scored598

A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger, and every scored standard error lies within 0.1 percent of the reference or of an accepted alternative convention. Sample sizes are not scored.

Task success

Agency conditionClaude Sonnet 4.6GPT-5.6 Luna
Chatbot81.7%70.0%
Self-repair93.3%90.0%
Constrained agent95.0%85.0%
All runs
85.8% of 360 runs
By prompt condition
Zero-shot 80.6%, One-shot 91.1%
By software
Python 90.8%, R 89.2%, Stata 77.5%

About this benchmark task

Method group
Regression discontinuity
Software
Stata, R, and Python
Runs
360 runs, ten in each combination of model, software, prompt, and agency condition

Open every recorded run for this task.