Regression discontinuity
Fuzzy regression discontinuity
Training eligibility and participation effects.
The task
Task statement
A training program becomes more accessible when an applicant's eligibility score reaches zero. The file contains id, score, participated (0 or 1), and earnings. Estimate the effect of participation at the eligibility cutoff using fuzzy regression discontinuity. Use cutoff zero and bandwidth 1 on each side. Include observations with abs(score)<1. Use triangular weights 1-abs(score). Fit separate local-linear slopes on the two sides, with intercepts. Use the same observations and weights for earnings and participation. Report the earnings discontinuity as reduced_form, the participation discontinuity as first_stage, and their ratio as treatment_effect. Each discontinuity is the right limit minus the left limit. Report conventional heteroskedasticity-robust HC0 standard errors with no degrees-of-freedom correction. Treat the bandwidth and weights as fixed. For treatment_effect, use the delta method and the joint covariance of the two discontinuity estimates from the same observations. Account for their covariance. Do not apply bias correction. Report n_obs as the total number of observations inside the bandwidth for every row.
Data
id, score centered at eligibility cutoff, participated, earnings. Simulated applicant data.
- Files
- input.csv
- Seed
- 2026091501
Required output
Scored estimates: treatment_effect, reduced_form, first_stage.
The standard errors of treatment_effect are also scored, because the main decision in this task is how to compute them.
Reference output
| Term | Reference estimate | Reference standard error | Sample size |
|---|---|---|---|
treatment_effect | 1.9410733 | 0.6895292 | 598 |
reduced_form | 0.85992217 | not scored | 598 |
first_stage | 0.44301376 | not scored | 598 |
A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger, and every scored standard error lies within 0.1 percent of the reference or of an accepted alternative convention. Sample sizes are not scored.
Task success
| Agency condition | Claude Sonnet 4.6 | GPT-5.6 Luna |
|---|---|---|
| Chatbot | 81.7% | 70.0% |
| Self-repair | 93.3% | 90.0% |
| Constrained agent | 95.0% | 85.0% |
- All runs
- 85.8% of 360 runs
- By prompt condition
- Zero-shot 80.6%, One-shot 91.1%
- By software
- Python 90.8%, R 89.2%, Stata 77.5%
About this benchmark task
- Method group
- Regression discontinuity
- Software
- Stata, R, and Python
- Runs
- 360 runs, ten in each combination of model, software, prompt, and agency condition