AI Agents & Econometric Coding

Time series

Cumulative lag effects (HAC)

Stock sales and cumulative volatility changes.

The task

Task statement

The file contains consecutive trading observations with date, sales, and vix. Sales measure aggregate stock sales. Vix measures market volatility. Estimate the cumulative association between past changes in volatility and current log sales.

Sort by date. Define the volatility change as the first difference of natural log(vix) across consecutive trading observations. Regress natural log(sales) on an intercept and lags 1 through 15 of this change. Lags refer to trading observations, including across weekends. Use every observation with the complete set of lagged changes. Report the sum of the fifteen coefficients as cumulative_response. Also report coefficients lag_1 and lag_15.

Use Newey-West heteroskedasticity and autocorrelation robust covariance with maximum lag 15 and Bartlett weights 1-j/16 for lag j. Multiply the covariance by N/(N-16), where N is the regression sample size. Use the full coefficient covariance matrix for the cumulative-response standard error. Report standard errors and n_obs for all three rows.

Data

Exact CSV columns: date, sales, and vix. The columns sales and vix contain positive values. Simulated consecutive weekday trading observations.

Files
input.csv
Seed
2026091502

Required output

Scored estimates: cumulative_response, lag_1, lag_15.

The standard errors of cumulative_response are also scored, because the main decision in this task is how to compute them.

Reference output

TermReference estimateReference standard errorSample size
cumulative_response2.63665720.46595283704
lag_10.51459007not scored704
lag_15-0.2822273not scored704

A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger, and every scored standard error lies within 0.1 percent of the reference or of an accepted alternative convention. Sample sizes are not scored.

Task success

Agency conditionClaude Sonnet 4.6GPT-5.6 Luna
Chatbot68.3%63.3%
Self-repair86.7%80.0%
Constrained agent95.0%81.7%
All runs
79.2% of 360 runs
By prompt condition
Zero-shot 63.9%, One-shot 94.4%
By software
Python 97.5%, R 77.5%, Stata 62.5%

About this benchmark task

Method group
Time series
Software
Stata, R, and Python
Runs
360 runs, ten in each combination of model, software, prompt, and agency condition

Open every recorded run for this task.