AI Agents & Econometric Coding

Survival analysis

Cox proportional hazards

The Cox model relates covariates to event hazards while leaving the baseline hazard unspecified.

The task

Task statement

Fit a Cox proportional-hazards model for (t_obs, event) on treat and x. Report log-hazard-ratio rows treat and x with model-based standard errors.

Data

Columns: subject, treat, x, t_obs, event.

Files
input.csv

Required output

Scored estimates: treat, x.

Reference output

TermReference estimateSample size
treat-0.498961452,000
x0.323126792,000

A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger. Sample sizes are not scored.

Task success

Agency conditionClaude Sonnet 4.6GPT-5.6 Luna
Chatbot66.7%91.7%
Self-repair100.0%95.0%
Constrained agent100.0%98.3%
All runs
91.9% of 360 runs
By prompt condition
Zero-shot 93.9%, One-shot 90.0%
By software
Python 76.7%, R 99.2%, Stata 100.0%

A multiplicative hazard model

The Cox model writes (Cox, 1972):

$$ h(t\mid x)=h_0(t)\exp(x'\beta). $$

The baseline hazard $h_0(t)$ can vary freely over time. Covariates multiply it by a constant hazard ratio.

A risk-set contribution

Suppose person A fails when A, B, and C remain at risk. Their indices are $0.7$, $0.2$, and $-0.1$.

Person$\exp(x'\beta)$Conditional failure share
A2.0140.486
B1.2210.295
C0.9050.219

The risk-set total is $2.014+1.221+0.905=4.140$. The contribution for A is $2.014/4.140=0.486$.

Assumptions and implementation

Proportional hazards requires the covariate hazard ratio to remain constant over time. Censoring must be conditionally independent of the event process. Time-varying effects can relax proportionality.

Tied event times require a convention such as Breslow or Efron. When event times are unique, these methods coincide. Software also differs in baseline-survival reporting.

A risk-set timeline makes each event denominator visible as people enter, fail, or leave observation.

Interpreting a hazard ratio

For a one-unit increase in $x_j$, $\exp(\beta_j)$ multiplies the instantaneous event rate among people still at risk. A hazard ratio of 1.5 means a 50 percent higher hazard at each time under proportional hazards. It does not mean a 50 percent higher event probability. Absolute survival differences also depend on the estimated baseline hazard and the time horizon.

Report hazard ratios with confidence intervals and define the event, time origin, and censoring rule. For a binary exposure, show adjusted survival curves at meaningful covariate values. These curves translate the relative hazard into probabilities that readers can inspect over time.

Risk-set construction and diagnostics

Build one record per risk interval and verify entry, exit, event, and censoring dates. Delayed entry requires risk sets that exclude people before they become observable. Choose and report the tie method. Breslow and Efron approximations can differ when many events share a recorded time.

The proportional-hazards assumption requires a stable covariate effect across event time. Examine scaled Schoenfeld residuals and time-by-covariate interactions. If an effect changes, report a time-varying coefficient or a clearly defined time split. Independent censoring must hold conditional on modeled variables. Informative dropout can bias the partial-likelihood estimate. Clustering and repeated events need covariance or model changes that match the dependence. Check influential score residuals because a small number of late events can determine an estimate. (Kalbfleisch and Prentice, 2002)

Further reading

Cox introduces partial likelihood for the proportional-hazards model (Cox, 1972). Kalbfleisch and Prentice develop duration-model estimation and diagnostics (Kalbfleisch and Prentice, 2002).

Sources and further reading

  1. D. R. Cox. 1972. “Regression Models and Life-Tables.” Journal of the Royal Statistical Society, Series B 34(2): 187–220. doi:10.1111/j.2517-6161.1972.tb00899.x.
  2. John D. Kalbfleisch, Ross L. Prentice. 2002. The Statistical Analysis of Failure Time Data. Wiley. doi:10.1002/9781118032985.

About this benchmark task

Method group
Survival analysis
Software
Stata, R, and Python
Runs
360 runs, ten in each combination of model, software, prompt, and agency condition

Open every recorded run for this task.