Poisson and count models
Poisson hurdle effects
Poisson hurdle unconditional-mean effects.
The task
Task statement
Fit a Poisson hurdle model on all supplied rows. Model Pr(y>0|x,d) by logit with an intercept, x, and d. Conditional on y>0, fit a zero-truncated Poisson with log lambda=b0+b_x*x+b_d*d, using only positive observations. Its likelihood is the Poisson mass divided by 1-exp(-lambda). The unconditional mean is Pr(y>0)*lambda/(1-exp(-lambda)). Report mean_y, the equally weighted average unconditional fitted mean over all rows. Report ame_x, the average derivative of this unconditional mean with respect to continuous x, accounting for x in both fitted equations. Report effect_d, the average difference in this unconditional mean when d changes from zero to one in both equations. Keep x at its observed value. Also report prob_at_least_three, the equally weighted average fitted unconditional probability Pr(y>=3|x,d) over all supplied rows. This probability includes the hurdle probability and the zero-truncation adjustment. For all results, n_obs is the full dataset row count, including zeros.
Data
Simulated CSV. Columns: y, x, d
- Files
- input.csv
- Seed
- 2026091703
Required output
Scored estimates: mean_y, ame_x, effect_d, prob_at_least_three.
Reference output
| Term | Reference estimate | Sample size |
|---|---|---|
mean_y | 0.76508742 | 1,300 |
ame_x | 0.35551381 | 1,300 |
effect_d | 0.064043813 | 1,300 |
prob_at_least_three | 0.097777625 | 1,300 |
A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger. Sample sizes are not scored.
Task success
| Agency condition | Claude Sonnet 4.6 | GPT-5.6 Luna |
|---|---|---|
| Chatbot | 83.3% | 66.7% |
| Self-repair | 95.0% | 76.7% |
| Constrained agent | 98.3% | 81.7% |
- All runs
- 83.6% of 360 runs
- By prompt condition
- Zero-shot 82.8%, One-shot 84.4%
- By software
- Python 90.0%, R 95.8%, Stata 65.0%
About this benchmark task
- Method group
- Poisson and count models
- Software
- Stata, R, and Python
- Runs
- 360 runs, ten in each combination of model, software, prompt, and agency condition