Discrete choice
Conditional logit welfare
Conditional-logit policy welfare.
The task
Task statement
Read ./input.csv. Its columns are person (person identifier), alternative (alternative identifier: 0, 1, or 2), chosen (1 for the chosen alternative and 0 otherwise), cost (alternative cost), quality (alternative quality attribute), and weight (person weight). Each person has three alternatives 0, 1, and 2 and exactly one chosen alternative. Fit conditional logit by maximum likelihood with utility beta_cost*cost+beta_quality*quality+asc1*1(alternative==1)+asc2*1(alternative==2). Alternative 0 has zero intercept. Use all people and all their alternatives. Estimate without observation weights. Evaluate a policy that reduces each person's alternative-1 cost by 30 percent, with every quality and other cost unchanged. Keep fitted coefficients fixed. Report welfare_gain, the weighted mean across people of [logsumexp(policy utilities)-logsumexp(original utilities)]/(-beta_cost). Report share_change_alt1, the weighted mean change in alternative-1 choice probability. Use the provided person weight only for these two post-estimation averages. Report cost_coefficient. For every row, n_obs is the number of distinct people, not the number of alternatives. The supplied weight is constant within each person. Normalize each weighted average by the sum of person weights.
Data
Simulated CSV. Columns: person, alternative, chosen, cost, quality, weight
- Files
- input.csv
- Seed
- 2026091701
Required output
Scored estimates: welfare_gain, share_change_alt1, cost_coefficient.
Reference output
| Term | Reference estimate | Sample size |
|---|---|---|
welfare_gain | 0.2321329 | 550 |
share_change_alt1 | 0.10947154 | 550 |
cost_coefficient | -1.0261688 | 550 |
A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger. Sample sizes are not scored.
Task success
| Agency condition | Claude Sonnet 4.6 | GPT-5.6 Luna |
|---|---|---|
| Chatbot | 78.3% | 60.0% |
| Self-repair | 91.7% | 83.3% |
| Constrained agent | 100.0% | 96.7% |
- All runs
- 85.0% of 360 runs
- By prompt condition
- Zero-shot 88.3%, One-shot 81.7%
- By software
- Python 95.0%, R 94.2%, Stata 65.8%
Sources
Sources
About this benchmark task
- Method group
- Discrete choice
- Software
- Stata, R, and Python
- Runs
- 360 runs, ten in each combination of model, software, prompt, and agency condition