Time series
Cumulative lag effects (HAC)
Stock sales and cumulative volatility changes.
The task
Task statement
The file contains consecutive trading observations with date, sales, and vix. Sales measure aggregate stock sales. Vix measures market volatility. Estimate the cumulative association between past changes in volatility and current log sales. Sort by date. Define the volatility change as the first difference of natural log(vix) across consecutive trading observations. Regress natural log(sales) on an intercept and lags 1 through 15 of this change. Lags refer to trading observations, including across weekends. Use every observation with the complete set of lagged changes. Report the sum of the fifteen coefficients as cumulative_response. Also report coefficients lag_1 and lag_15. Use Newey-West heteroskedasticity and autocorrelation robust covariance with maximum lag 15 and Bartlett weights 1-j/16 for lag j. Multiply the covariance by N/(N-16), where N is the regression sample size. Use the full coefficient covariance matrix for the cumulative-response standard error. Report standard errors and n_obs for all three rows.
Data
Exact CSV columns: date, sales, and vix. The columns sales and vix contain positive values. Simulated consecutive weekday trading observations.
- Files
- input.csv
- Seed
- 2026091502
Required output
Scored estimates: cumulative_response, lag_1, lag_15.
The standard errors of cumulative_response are also scored, because the main decision in this task is how to compute them.
Reference output
| Term | Reference estimate | Reference standard error | Sample size |
|---|---|---|---|
cumulative_response | 2.6366572 | 0.46595283 | 704 |
lag_1 | 0.51459007 | not scored | 704 |
lag_15 | -0.2822273 | not scored | 704 |
A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger, and every scored standard error lies within 0.1 percent of the reference or of an accepted alternative convention. Sample sizes are not scored.
Task success
| Agency condition | Claude Sonnet 4.6 | GPT-5.6 Luna |
|---|---|---|
| Chatbot | 68.3% | 63.3% |
| Self-repair | 86.7% | 80.0% |
| Constrained agent | 95.0% | 81.7% |
- All runs
- 79.2% of 360 runs
- By prompt condition
- Zero-shot 63.9%, One-shot 94.4%
- By software
- Python 97.5%, R 77.5%, Stata 62.5%
About this benchmark task
- Method group
- Time series
- Software
- Stata, R, and Python
- Runs
- 360 runs, ten in each combination of model, software, prompt, and agency condition