Panel and foundation-model forecasting
The forecast-panel workflow compares models across related series: MLForecast with LightGBM or scikit-learn histogram gradient boosting, NeuralForecast with NHITS, and zero-shot foundation models (FMs) through Darts. A dependency-light seasonal baseline is always available. These capabilities are available from version 0.3.0:
python -m pip install "ts-agents[ml,neural,foundation,viz]==0.4.1"
ts-agents workflow show forecast-panel --jsonFrom a source checkout, use uv sync --extra ml --extra neural --extra foundation --extra viz and prefix commands with uv run. The foundation extra switched from chronos-forecasting to Darts in 0.3.0; recreate older foundation environments when upgrading.
The base profile remains dependency-light. In 0.4.0, recommended drops UI and agent-framework dependencies while retaining classical analysis and plotting. The all profile includes every optional analysis backend. All imports of the optional libraries are lazy. workflow show forecast-panel reports status available on every install, because seasonal_naive always works; each optional family (ml_backends, neural_backends, foundation_models, plots) appears under availability.optional_features with its missing dependencies and extra. Requesting an unavailable method fails before any forecasting artifacts are written (only a failed run_manifest.json is recorded), with an install hint such as Install `ts-agents[foundation]`. (exit code 3).
To try the workflow on bundled data, export the M4 Monthly mini-panel. The source stores integer time indexes, so the exported month-start dates are a synthetic alignment: every train split ends at --train-end (default 2015-06-01) and the holdout covers the next 18 months.
uv run ts-agents data export-panel m4-monthly-mini --split train --out panel.csv --json
uv run ts-agents workflow run forecast-panel --input panel.csv --freq MS \
--horizon 18 --season-length 12 --methods seasonal_naive --skip-plots --jsonKeep the holdout (--split holdout) outside the workflow input and use it only to score the saved forecast.csv.
Input and execution
Single-series foundation-model adapters require a one-dimensional history and reject context values outside the finite Float32 range before loading weights. In forecast-series, each foundation model’s model_spec.json identifies its actual forecast_phase and horizon. Nonwinning models record validation only; a winning model records its future configuration plus validation_spec.
Supply a CSV/TSV/parquet/JSON/JSONL file, tabular JSON, or stdin. The input has exactly three columns, unique_id, ds, and y; rename them with --id-col, --time-col, and --value-col. A single series is a panel with one constant ID. Each series must be regular at the explicit frequency and share an end date; earlier start dates may differ. Gaps, missing values, duplicate keys, nonfinite targets, timezone-aware timestamps and extra covariates are rejected. Normalize timestamps to a documented timezone convention before passing the data.
uv run ts-agents workflow run forecast-panel \
--input panel.csv --freq h --horizon 24 \
--season-length 168 --lags 1,24,168 \
--methods seasonal_naive,lightgbm,nhits \
--n-windows 3 --n-estimators 200 --max-steps 1000 \
--accelerator cpu --output-dir outputs/electricity-validation --jsonThe hourly example needs an hourly panel such as prepared electricity data; use the monthly frequency and seasonal settings from the bundled example for m4-monthly-mini.
Use --methods lightgbm,histgbm for the ML profile or --methods nhits for neural-only runs. LightGBM and histogram GBM receive lag, shifted rolling-mean and known calendar features. NHITS trains across the whole panel with a training-only internal validation tail for early stopping. --input-size controls its context. --accelerator gpu requests one GPU explicitly. Release qualification covers CPU; a separate bounded electricity-demand dogfood exercised NHITS on one local RTX 5090, not every model/GPU configuration. Native models retain their backend’s dependency requirements when reloaded.
Every backtest origin fits a fresh model on the preceding history; no model is fit on later panel targets. Origins are spaced by --step-size (default: horizon). There is also a final fit for future forecasts. --n-estimators and --max-steps cap each fit, not total runtime. Choose origin counts with this cost in mind. Resume reruns the compatible computation; it does not load a training checkpoint. The existing jobs surface provides background execution, logs, and supervised local cancellation.
Evaluation and artifacts
Validation MAE/RMSE and training-only seasonal MASE are recorded overall, per series and per horizon. The ranking uses pooled validation RMSE, which can favor large-scale series. Zero seasonal scales have undefined MASE and are counted explicitly. Overlapping origins count different forecasts of the same target. Use these scores for selection and keep a final test outside the input. Future forecasts in forecast.csv are not scored against invented targets. metrics.json also records options.context_length and, when FMs are requested, options.resolved_context_length (per model) and foundation_models (one model spec per FM).
| Artifact | Contents |
|---|---|
metrics.json |
Metrics, settings, quality flags, package versions, per-fit runtime |
backtest_predictions.csv |
Series, origin, horizon step, actual, prediction, training MASE scale |
forecast.csv |
Future forecasts from every requested model |
models/<method>/ |
Native persisted model files; baseline stores history; FMs store only model_spec.json |
backtest.png |
Last validation origin for up to four series |
report.md |
Comparison summary, validation boundaries and limitations |
run_manifest.json |
Run identity, input fingerprint, options and artifact paths |
The report-generation skill guides an agent in assembling a comprehensive report from these artifacts without retraining; it is not a general report rebuild command. Dataset preparation, daily holdout evaluation with frozen models, resource monitoring, and richer report plots required custom code in the first electricity dogfood. Keep such workarounds explicit until supported interfaces exist. To reuse models in Python, use MLForecast.load(".../models/lightgbm") or NeuralForecast.load(".../models/nhits"), then the backend’s predict method. Load only model files you trust: native model formats can deserialize Python objects. This workflow currently offers point forecasts, not calibrated intervals.
Foundation models (Darts, zero-shot)
Install ts-agents[foundation] (darts[torch] plus torch). Darts is used as the interoperability layer; ts-agents pins a small catalog of checkpoints rather than acting as a model hub. The same method names work in forecast-panel, forecast-series, the forecast_foundation* agent tools and the foundation-smoke autoresearch loop.
| Method | Darts class | Checkpoint @ pinned revision | Licence | Weights |
|---|---|---|---|---|
chronos2_small (default) |
Chronos2Model |
autogluon/chronos-2-small @ ddec013 |
Apache-2.0 | ~112 MB |
chronos2 |
Chronos2Model |
amazon/chronos-2 @ 29ec376 |
Apache-2.0 | ~478 MB |
timesfm2p5 |
TimesFM2p5Model |
google/timesfm-2.5-200m-pytorch @ 1d95242 |
Apache-2.0 | ~925 MB |
patchtst_fm |
PatchTSTFMModel |
ibm-granite/granite-timeseries-patchtst-fm-r1 @ 151f9c6 |
Apache-2.0 | ~1032 MB |
Full revisions are in workflow show forecast-panel --json (capabilities.foundation_model_checkpoints) and in every run’s metrics.json.
uv run ts-agents workflow run forecast-panel --input panel.csv --freq MS \
--horizon 18 --season-length 12 --n-windows 2 \
--methods seasonal_naive,chronos2_small --skip-plots --json- Zero-shot: nothing is trained. At every validation origin each FM conditions only on that origin’s history, so no later target leaks into a forecast. The final forecast conditions on the most recent history.
- Context:
--context-lengthsets how many trailing points each series contributes (default 512). Chronos-2 allows up to 8192 input points independently of its output horizon. TimesFM 2.5 requires context plus output chunk to fit 16384 points; PatchTST-FM uses a combined limit of 8192. An explicit value that does not fit fails validation (exit code 2). Horizons above 1024 are rejected. Shorter series use all available history.--input-sizestays NHITS-only. - Model notes: TimesFM 2.5 switches to its longer projection head above 128 steps. PatchTST-FM was trained for a 64-step horizon and is used autoregressively beyond that, which can weaken accuracy.
- Downloads: only the requested checkpoints are fetched, into
HF_HOME, on first use. Network-less sandboxes need a pre-populated cache andHF_HUB_OFFLINE=1. A checkpoint that is neither cached nor downloadable fails withFoundationModelUnavailableError(“weights for … are not cached and could not be downloaded”), not with a missing-package error. The JSON envelope reports it asbackend_unavailable(exit code 5,retryable: true) with the remediation inhint. SeeSANDBOX.mdfor sandbox-specific notes. - Artifacts:
models/<fm>/holds onlymodel_spec.json(Darts class, checkpoint, revision, Darts version, resolved context, accelerator,weights_saved: false). Weights are never copied into run directories. - Resources: each requested FM stays loaded for the whole run and is released at the end; running all four together needs several GB of RAM. The
forecast_foundation*agent tools and direct library calls outside a run reuse a bounded one-model cache; another configuration evicts the old model.--accelerator gpuis used only when requested. - Interpretation: public benchmarks such as M4 may overlap FM pretraining data, so FM scores on them can be optimistic. FMs produce point forecasts only, and they never add quality flags; caveats appear in
report.md.
MLForecast documentation, NHITS documentation and the Darts foundation-model documentation describe the underlying interfaces. MLForecast documentation and NHITS documentation describe the underlying training interfaces.
Requested input columns must exist; missing custom names fail even when the default columns exist. Default lag offsets are deduplicated for a season length of one. Explicit empty method lists are rejected. Sandbox exports preserve model subdirectories.
Foundation inference is serialized within a process without changing global stdout or logger levels. Repeated agent tool calls retain at most one loaded model; workflows release their scoped models on completion. Series comparisons reserve one output chunk for the larger validation/future horizon and record that chunk plus each requested forecast horizon. Daytona tool runs install their required extras unless overridden. The CSV panel tool reads files on the host and returns durable host artifact paths.