AI Agents & Econometric Coding

Survival analysis

Discrete-time conditional logit

Exact conditional odds for monthly employment exits.

The task

Task statement

Workers belong to cohorts. Each cohort-month forms its own risk set. All workers enter before month one. Cohorts have independent baseline exit odds. training is a fixed binary covariate. end_month records either employment exit or right censoring. event=1 indicates employment exit. At each month, include workers whose end_month is at least that month. Evaluate all exits jointly before removing same-month censored workers. Remove both groups before the next month. No worker reenters.

Estimate Cox's discrete-time conditional odds model. Its coefficient beta multiplies training. For a risk set R with d observed exits and observed training sum S, the conditional log likelihood is beta*S-log(sum_A exp(beta*sum_{j in A} training_j)). The sum covers all distinct size-d subsets of R. Add contributions across cohorts and months with at least one exit. Use exact enumeration or an elementary symmetric polynomial recursion. Maximize over beta in [-4,4].

Return beta, exp(beta) as the conditional odds ratio, and the exact log likelihood without additional constants. At the fitted coefficient, return the conditional probability, given its exit count, that exactly the observed set of workers exits in cohort 0, month 1. This is the probability of those named workers, without multiplying by the number of sets with the same training sum. Also return its natural logarithm. The odds ratio is a conditional discrete-time quantity. For every result, n_obs is the number of input worker rows.

Data

input.csv: cohort = Independent cohort-specific risk set identifier; worker = Worker identifier within the cohort; training = Fixed binary training covariate; end_month = Month of employment exit or censoring; event = One for an exit; zero for right censoring

Files
input.csv

Required output

Scored estimates: training_coefficient, conditional_odds_ratio, log_likelihood, first_event_set_probability, first_event_set_log_probability.

Reference output

TermReference estimateSample size
training_coefficient1.3451113144
conditional_odds_ratio3.8386137144
log_likelihood-144.52236144
first_event_set_probability0.00038003953144
first_event_set_log_probability-7.8752353144

A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger. Sample sizes are not scored.

Task success

Agency conditionClaude Sonnet 4.6GPT-5.6 Luna
Chatbot85.0%51.7%
Self-repair100.0%73.3%
Constrained agent100.0%88.3%
All runs
83.1% of 360 runs
By prompt condition
Zero-shot 78.3%, One-shot 87.8%
By software
Python 93.3%, R 93.3%, Stata 62.5%

Sources

Sources

About this benchmark task

Method group
Survival analysis
Software
Stata, R, and Python
Runs
360 runs, ten in each combination of model, software, prompt, and agency condition

Open every recorded run for this task.