Poisson and count models
PPML with separated zeros
Trade policy with separated zero flows.
The task
Task statement
Estimate Poisson pseudo maximum likelihood for trade. The index contains agreement, pair fixed effects, exporter-year fixed effects, and importer-year fixed effects. Use the supplied identifiers. All country pairs, including equal exporter and importer identifiers, are synthetic trading units. Detect every separated zero observation. A separation direction h satisfies Xh=0 on positive outcomes and Xh<=0 on zeros. A zero is separated if some such direction gives its index a strict negative value. Remove these observations. Retain ordinary zeros. Resolve collinear dummy columns without changing the policy span. Return the agreement coefficient and exp(coefficient)-1. Compute its sandwich standard error clustered by pair. Use bread inverse(X' diag(mu) X) and cluster scores sum_i X_i*(y_i-mu_i). Multiply the covariance by G/(G-1)*(N-1)/(N-K). N is the retained sample size, G the retained pair count, and K the retained design rank including all fixed effects. Return retained observations, separated observations, and retained zero observations. For every result, n_obs is N, the retained estimation sample size.
Data
input.csv: exporter = Synthetic exporter identifier; importer = Synthetic importer identifier; year = Calendar year identifier; pair = Exporter-importer pair identifier and covariance cluster; exporter_year = Exporter-year fixed-effect identifier; importer_year = Importer-year fixed-effect identifier; agreement = Trade-agreement indicator; trade = Nonnegative trade outcome
- Files
- input.csv
Required output
Scored estimates: agreement_coefficient, proportional_mean_change.
Reference output
| Term | Reference estimate | Sample size |
|---|---|---|
agreement_coefficient | 0.41323834 | 139 |
proportional_mean_change | 0.51170528 | 139 |
A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger. Sample sizes are not scored.
Task success
| Agency condition | Claude Sonnet 4.6 | GPT-5.6 Luna |
|---|---|---|
| Chatbot | 58.3% | 46.7% |
| Self-repair | 93.3% | 71.7% |
| Constrained agent | 100.0% | 96.7% |
- All runs
- 77.8% of 360 runs
- By prompt condition
- Zero-shot 70.0%, One-shot 85.6%
- By software
- Python 84.2%, R 81.7%, Stata 67.5%
Sources
Sources
About this benchmark task
- Method group
- Poisson and count models
- Software
- Stata, R, and Python
- Runs
- 360 runs, ten in each combination of model, software, prompt, and agency condition