Time series
Rolling AR(1) forecasts
The task
Task statement
Sort input.csv by time. Construct one-step-ahead rolling OLS AR(1) forecasts for times f=101,...,250. For each forecast f, use exactly the 100 observed levels y_(f-100),...,y_(f-1). Fit y_s=a+b*y_(s-1) with an intercept on s=f-99,...,f-1, giving exactly 99 regression pairs. Forecast y_f as a_hat+b_hat*y_(f-1). Refit separately at every forecast date using observed values. Score RMSE=sqrt(sum_{f=101}^{250}(y_f-forecast_f)^2/150). Report terms rmse, forecast_101, forecast_150, forecast_200, and forecast_250. Set se=null for every term. For rmse set n_obs=150. For each forecast set n_obs=99, the number of regression pairs used to estimate that forecast's model.Data
250 ordered observations with columns time (1 through 250) and y.
- Files
- input.csv
- Seed
- 2026091412
Required output
Scored estimates: rmse, forecast_101, forecast_150, forecast_200, forecast_250.
Reference output
| Term | Reference estimate | Sample size |
|---|---|---|
rmse | 1.0551116 | 150 |
forecast_101 | 0.05064737 | 99 |
forecast_150 | -0.84958902 | 99 |
forecast_200 | 0.71711729 | 99 |
forecast_250 | -0.83319457 | 99 |
A run succeeds when every scored estimate lies within 1 percent or 0.005 of its reference value, whichever is larger. Sample sizes are not scored.
Task success
| Agency condition | Claude Sonnet 4.6 | GPT-5.6 Luna |
|---|---|---|
| Chatbot | 81.7% | 81.7% |
| Self-repair | 88.3% | 91.7% |
| Constrained agent | 100.0% | 98.3% |
- All runs
- 90.3% of 360 runs
- By prompt condition
- Zero-shot 81.7%, One-shot 98.9%
- By software
- Python 85.8%, R 92.5%, Stata 92.5%
About this benchmark task
- Method group
- Time series
- Software
- Stata, R, and Python
- Runs
- 360 runs, ten in each combination of model, software, prompt, and agency condition