Instrumental variables
Anderson–Rubin confidence set
Weak-IV confidence-set inversion.
The task
Task statement
Estimate the 95 percent Anderson-Rubin confidence set for the coefficient beta of endog in y = intercept + beta*endog + gamma*x + error, using excluded instruments z1 and z2. For each candidate beta, regress y-beta*endog on an intercept, x, z1, and z2. Use the conventional homoskedastic joint F test of both excluded-instrument coefficients, with numerator degrees of freedom 2 and denominator n-4. Accept beta when F is at most the 95th percentile of that F distribution. Invert this test over the entire real line, without a bounded search grid. The data yield two finite boundary points. Report their smaller and larger values as boundary_low and boundary_high. Report accepts_zero as 1 if zero belongs to the confidence set, otherwise 0. Report unbounded as 1 if the set includes arbitrarily large positive or negative beta values, otherwise 0. Report ar_f_at_one as the test statistic at beta=1. Use se=null and the full sample size as n_obs for every row. Determine set membership from the test inequality. A disjoint set is possible.
Data
One shared fixed dataset with 240 rows. Columns: y, endog, x, z1, z2.
- Files
- input.csv
Required output
Scored estimates: boundary_low, boundary_high, accepts_zero, unbounded, ar_f_at_one.
Reference output
| Term | Reference estimate | Sample size |
|---|---|---|
boundary_low | 2.1120211 | 240 |
boundary_high | 5.1372287 | 240 |
accepts_zero | 1 | 240 |
unbounded | 1 | 240 |
ar_f_at_one | 0.44197014 | 240 |
A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger. Sample sizes are not scored.
Task success
| Agency condition | Claude Sonnet 4.6 | GPT-5.6 Luna |
|---|---|---|
| Chatbot | 78.3% | 73.3% |
| Self-repair | 81.7% | 80.0% |
| Constrained agent | 98.3% | 86.7% |
- All runs
- 83.1% of 360 runs
- By prompt condition
- Zero-shot 81.7%, One-shot 84.4%
- By software
- Python 95.0%, R 94.2%, Stata 60.0%
About this benchmark task
- Method group
- Instrumental variables
- Software
- Stata, R, and Python
- Runs
- 360 runs, ten in each combination of model, software, prompt, and agency condition