AI Agents & Econometric Coding

Discrete choice

Conditional logit welfare

Conditional-logit policy welfare.

The task

Task statement

Read ./input.csv. Its columns are person (person identifier), alternative (alternative identifier: 0, 1, or 2), chosen (1 for the chosen alternative and 0 otherwise), cost (alternative cost), quality (alternative quality attribute), and weight (person weight).

Each person has three alternatives 0, 1, and 2 and exactly one chosen alternative. Fit conditional logit by maximum likelihood with utility beta_cost*cost+beta_quality*quality+asc1*1(alternative==1)+asc2*1(alternative==2). Alternative 0 has zero intercept. Use all people and all their alternatives. Estimate without observation weights. Evaluate a policy that reduces each person's alternative-1 cost by 30 percent, with every quality and other cost unchanged. Keep fitted coefficients fixed. Report welfare_gain, the weighted mean across people of [logsumexp(policy utilities)-logsumexp(original utilities)]/(-beta_cost). Report share_change_alt1, the weighted mean change in alternative-1 choice probability. Use the provided person weight only for these two post-estimation averages. Report cost_coefficient. For every row, n_obs is the number of distinct people, not the number of alternatives. The supplied weight is constant within each person. Normalize each weighted average by the sum of person weights.

Data

Simulated CSV. Columns: person, alternative, chosen, cost, quality, weight

Files
input.csv
Seed
2026091701

Required output

Scored estimates: welfare_gain, share_change_alt1, cost_coefficient.

Reference output

TermReference estimateSample size
welfare_gain0.2321329550
share_change_alt10.10947154550
cost_coefficient-1.0261688550

A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger. Sample sizes are not scored.

Task success

Agency conditionClaude Sonnet 4.6GPT-5.6 Luna
Chatbot78.3%60.0%
Self-repair91.7%83.3%
Constrained agent100.0%96.7%
All runs
85.0% of 360 runs
By prompt condition
Zero-shot 88.3%, One-shot 81.7%
By software
Python 95.0%, R 94.2%, Stata 65.8%

Sources

Sources

About this benchmark task

Method group
Discrete choice
Software
Stata, R, and Python
Runs
360 runs, ten in each combination of model, software, prompt, and agency condition

Open every recorded run for this task.