Survival analysis
Cox proportional hazards
The Cox model relates covariates to event hazards while leaving the baseline hazard unspecified.
The task
Task statement
Fit a Cox proportional-hazards model for (t_obs, event) on treat and x. Report log-hazard-ratio rows treat and x with model-based standard errors.
Data
Columns: subject, treat, x, t_obs, event.
- Files
- input.csv
Required output
Scored estimates: treat, x.
Reference output
| Term | Reference estimate | Sample size |
|---|---|---|
treat | -0.49896145 | 2,000 |
x | 0.32312679 | 2,000 |
A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger. Sample sizes are not scored.
Task success
| Agency condition | Claude Sonnet 4.6 | GPT-5.6 Luna |
|---|---|---|
| Chatbot | 66.7% | 91.7% |
| Self-repair | 100.0% | 95.0% |
| Constrained agent | 100.0% | 98.3% |
- All runs
- 91.9% of 360 runs
- By prompt condition
- Zero-shot 93.9%, One-shot 90.0%
- By software
- Python 76.7%, R 99.2%, Stata 100.0%
A multiplicative hazard model
The Cox model writes (Cox, 1972):
The baseline hazard $h_0(t)$ can vary freely over time. Covariates multiply it by a constant hazard ratio.
A risk-set contribution
Suppose person A fails when A, B, and C remain at risk. Their indices are $0.7$, $0.2$, and $-0.1$.
| Person | $\exp(x'\beta)$ | Conditional failure share |
|---|---|---|
| A | 2.014 | 0.486 |
| B | 1.221 | 0.295 |
| C | 0.905 | 0.219 |
The risk-set total is $2.014+1.221+0.905=4.140$. The contribution for A is $2.014/4.140=0.486$.
Assumptions and implementation
Proportional hazards requires the covariate hazard ratio to remain constant over time. Censoring must be conditionally independent of the event process. Time-varying effects can relax proportionality.
Tied event times require a convention such as Breslow or Efron. When event times are unique, these methods coincide. Software also differs in baseline-survival reporting.
A risk-set timeline makes each event denominator visible as people enter, fail, or leave observation.
Interpreting a hazard ratio
For a one-unit increase in $x_j$, $\exp(\beta_j)$ multiplies the instantaneous event rate among people still at risk. A hazard ratio of 1.5 means a 50 percent higher hazard at each time under proportional hazards. It does not mean a 50 percent higher event probability. Absolute survival differences also depend on the estimated baseline hazard and the time horizon.
Report hazard ratios with confidence intervals and define the event, time origin, and censoring rule. For a binary exposure, show adjusted survival curves at meaningful covariate values. These curves translate the relative hazard into probabilities that readers can inspect over time.
Risk-set construction and diagnostics
Build one record per risk interval and verify entry, exit, event, and censoring dates. Delayed entry requires risk sets that exclude people before they become observable. Choose and report the tie method. Breslow and Efron approximations can differ when many events share a recorded time.
The proportional-hazards assumption requires a stable covariate effect across event time. Examine scaled Schoenfeld residuals and time-by-covariate interactions. If an effect changes, report a time-varying coefficient or a clearly defined time split. Independent censoring must hold conditional on modeled variables. Informative dropout can bias the partial-likelihood estimate. Clustering and repeated events need covariance or model changes that match the dependence. Check influential score residuals because a small number of late events can determine an estimate. (Kalbfleisch and Prentice, 2002)
Further reading
Cox introduces partial likelihood for the proportional-hazards model (Cox, 1972). Kalbfleisch and Prentice develop duration-model estimation and diagnostics (Kalbfleisch and Prentice, 2002).
Sources and further reading
- D. R. Cox. 1972. “Regression Models and Life-Tables.” Journal of the Royal Statistical Society, Series B 34(2): 187–220. doi:10.1111/j.2517-6161.1972.tb00899.x.
- John D. Kalbfleisch, Ross L. Prentice. 2002. The Statistical Analysis of Failure Time Data. Wiley. doi:10.1002/9781118032985.
About this benchmark task
- Method group
- Survival analysis
- Software
- Stata, R, and Python
- Runs
- 360 runs, ten in each combination of model, software, prompt, and agency condition