Skip to main content

Panel and foundation-model forecasting

The forecast-panel workflow compares models across related series: MLForecast with LightGBM or scikit-learn histogram gradient boosting, NeuralForecast with NHITS, and zero-shot foundation models (FMs) through Darts. A dependency-light seasonal baseline is always available. These capabilities are available from version 0.3.0:

python -m pip install "ts-agents[ml,neural,foundation,viz]==0.4.1"
ts-agents workflow show forecast-panel --json

From a source checkout, use uv sync --extra ml --extra neural --extra foundation --extra viz and prefix commands with uv run. The foundation extra switched from chronos-forecasting to Darts in 0.3.0; recreate older foundation environments when upgrading.

The base profile remains dependency-light. In 0.4.0, recommended drops UI and agent-framework dependencies while retaining classical analysis and plotting. The all profile includes every optional analysis backend. All imports of the optional libraries are lazy. workflow show forecast-panel reports status available on every install, because seasonal_naive always works; each optional family (ml_backends, neural_backends, foundation_models, plots) appears under availability.optional_features with its missing dependencies and extra. Requesting an unavailable method fails before any forecasting artifacts are written (only a failed run_manifest.json is recorded), with an install hint such as Install `ts-agents[foundation]`. (exit code 3).

To try the workflow on bundled data, export the M4 Monthly mini-panel. The source stores integer time indexes, so the exported month-start dates are a synthetic alignment: every train split ends at --train-end (default 2015-06-01) and the holdout covers the next 18 months.

uv run ts-agents data export-panel m4-monthly-mini --split train --out panel.csv --json
uv run ts-agents workflow run forecast-panel --input panel.csv --freq MS \
  --horizon 18 --season-length 12 --methods seasonal_naive --skip-plots --json

Keep the holdout (--split holdout) outside the workflow input and use it only to score the saved forecast.csv.

Input and execution

Single-series foundation-model adapters require a one-dimensional history and reject context values outside the finite Float32 range before loading weights. In forecast-series, each foundation model’s model_spec.json identifies its actual forecast_phase and horizon. Nonwinning models record validation only; a winning model records its future configuration plus validation_spec.

Supply a CSV/TSV/parquet/JSON/JSONL file, tabular JSON, or stdin. The input has exactly three columns, unique_id, ds, and y; rename them with --id-col, --time-col, and --value-col. A single series is a panel with one constant ID. Each series must be regular at the explicit frequency and share an end date; earlier start dates may differ. Gaps, missing values, duplicate keys, nonfinite targets, timezone-aware timestamps and extra covariates are rejected. Normalize timestamps to a documented timezone convention before passing the data.

uv run ts-agents workflow run forecast-panel \
  --input panel.csv --freq h --horizon 24 \
  --season-length 168 --lags 1,24,168 \
  --methods seasonal_naive,lightgbm,nhits \
  --n-windows 3 --n-estimators 200 --max-steps 1000 \
  --accelerator cpu --output-dir outputs/electricity-validation --json

The hourly example needs an hourly panel such as prepared electricity data; use the monthly frequency and seasonal settings from the bundled example for m4-monthly-mini.

Use --methods lightgbm,histgbm for the ML profile or --methods nhits for neural-only runs. LightGBM and histogram GBM receive lag, shifted rolling-mean and known calendar features. NHITS trains across the whole panel with a training-only internal validation tail for early stopping. --input-size controls its context. --accelerator gpu requests one GPU explicitly. Release qualification covers CPU; a separate bounded electricity-demand dogfood exercised NHITS on one local RTX 5090, not every model/GPU configuration. Native models retain their backend’s dependency requirements when reloaded.

Every backtest origin fits a fresh model on the preceding history; no model is fit on later panel targets. Origins are spaced by --step-size (default: horizon). There is also a final fit for future forecasts. --n-estimators and --max-steps cap each fit, not total runtime. Choose origin counts with this cost in mind. Resume reruns the compatible computation; it does not load a training checkpoint. The existing jobs surface provides background execution, logs, and supervised local cancellation.

Evaluation and artifacts

Validation MAE/RMSE and training-only seasonal MASE are recorded overall, per series and per horizon. The ranking uses pooled validation RMSE, which can favor large-scale series. Zero seasonal scales have undefined MASE and are counted explicitly. Overlapping origins count different forecasts of the same target. Use these scores for selection and keep a final test outside the input. Future forecasts in forecast.csv are not scored against invented targets. metrics.json also records options.context_length and, when FMs are requested, options.resolved_context_length (per model) and foundation_models (one model spec per FM).

Artifact Contents
metrics.json Metrics, settings, quality flags, package versions, per-fit runtime
backtest_predictions.csv Series, origin, horizon step, actual, prediction, training MASE scale
forecast.csv Future forecasts from every requested model
models/<method>/ Native persisted model files; baseline stores history; FMs store only model_spec.json
backtest.png Last validation origin for up to four series
report.md Comparison summary, validation boundaries and limitations
run_manifest.json Run identity, input fingerprint, options and artifact paths

The report-generation skill guides an agent in assembling a comprehensive report from these artifacts without retraining; it is not a general report rebuild command. Dataset preparation, daily holdout evaluation with frozen models, resource monitoring, and richer report plots required custom code in the first electricity dogfood. Keep such workarounds explicit until supported interfaces exist. To reuse models in Python, use MLForecast.load(".../models/lightgbm") or NeuralForecast.load(".../models/nhits"), then the backend’s predict method. Load only model files you trust: native model formats can deserialize Python objects. This workflow currently offers point forecasts, not calibrated intervals.

Foundation models (Darts, zero-shot)

Install ts-agents[foundation] (darts[torch] plus torch). Darts is used as the interoperability layer; ts-agents pins a small catalog of checkpoints rather than acting as a model hub. The same method names work in forecast-panel, forecast-series, the forecast_foundation* agent tools and the foundation-smoke autoresearch loop.

Method Darts class Checkpoint @ pinned revision Licence Weights
chronos2_small (default) Chronos2Model autogluon/chronos-2-small @ ddec013 Apache-2.0 ~112 MB
chronos2 Chronos2Model amazon/chronos-2 @ 29ec376 Apache-2.0 ~478 MB
timesfm2p5 TimesFM2p5Model google/timesfm-2.5-200m-pytorch @ 1d95242 Apache-2.0 ~925 MB
patchtst_fm PatchTSTFMModel ibm-granite/granite-timeseries-patchtst-fm-r1 @ 151f9c6 Apache-2.0 ~1032 MB

Full revisions are in workflow show forecast-panel --json (capabilities.foundation_model_checkpoints) and in every run’s metrics.json.

uv run ts-agents workflow run forecast-panel --input panel.csv --freq MS \
  --horizon 18 --season-length 12 --n-windows 2 \
  --methods seasonal_naive,chronos2_small --skip-plots --json
  • Zero-shot: nothing is trained. At every validation origin each FM conditions only on that origin’s history, so no later target leaks into a forecast. The final forecast conditions on the most recent history.
  • Context: --context-length sets how many trailing points each series contributes (default 512). Chronos-2 allows up to 8192 input points independently of its output horizon. TimesFM 2.5 requires context plus output chunk to fit 16384 points; PatchTST-FM uses a combined limit of 8192. An explicit value that does not fit fails validation (exit code 2). Horizons above 1024 are rejected. Shorter series use all available history. --input-size stays NHITS-only.
  • Model notes: TimesFM 2.5 switches to its longer projection head above 128 steps. PatchTST-FM was trained for a 64-step horizon and is used autoregressively beyond that, which can weaken accuracy.
  • Downloads: only the requested checkpoints are fetched, into HF_HOME, on first use. Network-less sandboxes need a pre-populated cache and HF_HUB_OFFLINE=1. A checkpoint that is neither cached nor downloadable fails with FoundationModelUnavailableError (“weights for … are not cached and could not be downloaded”), not with a missing-package error. The JSON envelope reports it as backend_unavailable (exit code 5, retryable: true) with the remediation in hint. See SANDBOX.md for sandbox-specific notes.
  • Artifacts: models/<fm>/ holds only model_spec.json (Darts class, checkpoint, revision, Darts version, resolved context, accelerator, weights_saved: false). Weights are never copied into run directories.
  • Resources: each requested FM stays loaded for the whole run and is released at the end; running all four together needs several GB of RAM. The forecast_foundation* agent tools and direct library calls outside a run reuse a bounded one-model cache; another configuration evicts the old model. --accelerator gpu is used only when requested.
  • Interpretation: public benchmarks such as M4 may overlap FM pretraining data, so FM scores on them can be optimistic. FMs produce point forecasts only, and they never add quality flags; caveats appear in report.md.

MLForecast documentation, NHITS documentation and the Darts foundation-model documentation describe the underlying interfaces. MLForecast documentation and NHITS documentation describe the underlying training interfaces.

Requested input columns must exist; missing custom names fail even when the default columns exist. Default lag offsets are deduplicated for a season length of one. Explicit empty method lists are rejected. Sandbox exports preserve model subdirectories.

Foundation inference is serialized within a process without changing global stdout or logger levels. Repeated agent tool calls retain at most one loaded model; workflows release their scoped models on completion. Series comparisons reserve one output chunk for the larger validation/future horizon and record that chunk plus each requested forecast horizon. Daytona tool runs install their required extras unless overridden. The CSV panel tool reads files on the host and returns durable host artifact paths.