AI Agents & Econometric Coding

Poisson and count models

PPML with separated zeros

Trade policy with separated zero flows.

The task

Task statement

Estimate Poisson pseudo maximum likelihood for trade. The index contains agreement, pair fixed effects, exporter-year fixed effects, and importer-year fixed effects. Use the supplied identifiers. All country pairs, including equal exporter and importer identifiers, are synthetic trading units. Detect every separated zero observation. A separation direction h satisfies Xh=0 on positive outcomes and Xh<=0 on zeros. A zero is separated if some such direction gives its index a strict negative value. Remove these observations. Retain ordinary zeros. Resolve collinear dummy columns without changing the policy span.

Return the agreement coefficient and exp(coefficient)-1. Compute its sandwich standard error clustered by pair. Use bread inverse(X' diag(mu) X) and cluster scores sum_i X_i*(y_i-mu_i). Multiply the covariance by G/(G-1)*(N-1)/(N-K). N is the retained sample size, G the retained pair count, and K the retained design rank including all fixed effects. Return retained observations, separated observations, and retained zero observations. For every result, n_obs is N, the retained estimation sample size.

Data

input.csv: exporter = Synthetic exporter identifier; importer = Synthetic importer identifier; year = Calendar year identifier; pair = Exporter-importer pair identifier and covariance cluster; exporter_year = Exporter-year fixed-effect identifier; importer_year = Importer-year fixed-effect identifier; agreement = Trade-agreement indicator; trade = Nonnegative trade outcome

Files
input.csv

Required output

Scored estimates: agreement_coefficient, proportional_mean_change.

Reference output

TermReference estimateSample size
agreement_coefficient0.41323834139
proportional_mean_change0.51170528139

A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger. Sample sizes are not scored.

Task success

Agency conditionClaude Sonnet 4.6GPT-5.6 Luna
Chatbot58.3%46.7%
Self-repair93.3%71.7%
Constrained agent100.0%96.7%
All runs
77.8% of 360 runs
By prompt condition
Zero-shot 70.0%, One-shot 85.6%
By software
Python 84.2%, R 81.7%, Stata 67.5%

Sources

Sources

About this benchmark task

Method group
Poisson and count models
Software
Stata, R, and Python
Runs
360 runs, ten in each combination of model, software, prompt, and agency condition

Open every recorded run for this task.