AI Agents & Econometric Coding

Decomposition

Oaxaca decomposition

Oaxaca wage-gap decomposition.

The task

Task statement

Use Python. Read ./input.csv. Available columns: id, y, x, z, group.

Decompose the difference in mean y between group 1 and group 0. Fit separate OLS regressions of y on an intercept, x, and z in each group. Use group 0 coefficients as the reference. Let Xbar_g include the intercept 1. Report gap=mean(y|group=1)-mean(y|group=0), explained=(Xbar_1-Xbar_0) times beta_0, and unexplained=Xbar_1 times (beta_1-beta_0). Also report explained_x=(mean(x|group=1)-mean(x|group=0))*beta_0_x. Include every row. Set se=null and n_obs to the full sample size for all rows. The components describe this regression decomposition and do not establish causal discrimination.

Report the requested point estimates. Standard errors are not requested. Use json.dump to write result.json with the following structure. Replace each <number> with the computed value. Report at least eight significant digits.
{
  "task_id": "broad_oaxaca",
  "platform": "python",
  "estimates": [
    {
      "term": "gap",
      "estimate": "<number>",
      "se": null,
      "n_obs": 360
    },
    {
      "term": "explained",
      "estimate": "<number>",
      "se": null,
      "n_obs": 360
    },
    {
      "term": "unexplained",
      "estimate": "<number>",
      "se": null,
      "n_obs": 360
    },
    {
      "term": "explained_x",
      "estimate": "<number>",
      "se": null,
      "n_obs": 360
    }
  ],
  "method_metadata": {},
  "diagnostics": {}
}

Data

One shared fixed dataset with 360 rows. Columns: id, y, x, z, group.

Files
input.csv

Required output

Scored estimates: gap, explained, unexplained, explained_x.

Reference output

TermReference estimateSample size
gap1.4355166360
explained0.57229798360
unexplained0.86321861360
explained_x0.51050288360

A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger. Sample sizes are not scored.

Task success

Agency conditionClaude Sonnet 4.6GPT-5.6 Luna
Chatbot100.0%86.7%
Self-repair100.0%100.0%
Constrained agent100.0%100.0%
All runs
97.8% of 360 runs
By prompt condition
Zero-shot 97.2%, One-shot 98.3%
By software
Python 98.3%, R 97.5%, Stata 97.5%

Sources

Sources

About this benchmark task

Method group
Decomposition
Software
Stata, R, and Python
Runs
360 runs, ten in each combination of model, software, prompt, and agency condition

Open every recorded run for this task.