Discrete choice
Ordered logit
Ordered response probabilities.
The task
Task statement
Estimate an ordered logit model for y, whose categories are ordered 1, 2, 3. Use x and z as covariates. Estimate two free cutpoints and no separate intercept. Use maximum likelihood and conventional observed-information standard errors. Report slope rows x and z. Also report prob_high as the fitted probability of category 3 for x=1 and z=0, with se=null. Use all observations and report their number as n_obs for every row.
Data
One shared fixed dataset with 420 rows. Columns: id, y, x, z.
- Files
- input.csv
Required output
Scored estimates: x, z, prob_high.
Reference output
| Term | Reference estimate | Sample size |
|---|---|---|
x | 0.60884749 | 420 |
z | -0.39208537 | 420 |
prob_high | 0.48327683 | 420 |
A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger. Sample sizes are not scored.
Task success
| Agency condition | Claude Sonnet 4.6 | GPT-5.6 Luna |
|---|---|---|
| Chatbot | 78.3% | 93.3% |
| Self-repair | 95.0% | 98.3% |
| Constrained agent | 100.0% | 95.0% |
- All runs
- 93.3% of 360 runs
- By prompt condition
- Zero-shot 86.7%, One-shot 100.0%
- By software
- Python 95.0%, R 92.5%, Stata 92.5%
About this benchmark task
- Method group
- Discrete choice
- Software
- Stata, R, and Python
- Runs
- 360 runs, ten in each combination of model, software, prompt, and agency condition