Building Data Science Agents in the Real World

CLI-first time-series automation for enterprise and research AI.

Farrukh Nauman

2026-05-13

What is ts-agents?

CLI + workflows + skills + sandboxes for reproducible time-series automation.

workflow

Reproducible end-to-end runs

tool

Direct access to analysis functions

skills

Domain runbooks for agent harnesses

sandbox

Isolation for messy dependencies

Why This Matters

Why automate the work?

  • Routine analysis consumes expert time: clean data, run baselines, make plots, write reports
  • One-off runs hide choices in notebooks, transcripts, and local folders
  • Teams need more diagnostics than humans can review line by line
  • Automation makes the routine path cheap, repeatable, and ready for expert review

Why make it a CLI?

  • Commands are a stable contract for people, agents, CI, schedulers, and notebooks
  • A run can be replayed with the same inputs, flags, environment, and output directory
  • JSON envelopes and file artifacts are easier to audit than chat transcripts
  • Different orchestrators can call the same workflow without changing the analysis code

Architecture

Agent or script

Codex / Claude / LangChain / CI

plans and runs

ts-agents

workflow · tool · skills · sandbox

writes

Artifacts

JSON · PNG · CSV · Markdown · manifests

The CLI is intentionally the stable spine. Agents, scripts, notebooks, and CI can all call the same contract.

Demo 1: Activity Recognition

Terminal demo running the activity recognition workflow and listing generated artifacts

What happens

  • Generate or load a labeled sensor stream
  • Sweep candidate window sizes
  • Evaluate a windowed classifier
  • Write plots and a short report

window_scores.png

confusion_matrix.png

report.md

Demo 2: Forecasting

Terminal demo running the forecasting workflow and listing generated artifacts

What happens

  • Compare baseline forecasting methods
  • Rank by holdout metrics
  • Save the selected forecast
  • Produce report-ready artifacts

forecast_comparison.png

forecast.csv

report.md

Reports Are First-Class

Workflow reports

Each first-class workflow writes a short Markdown report alongside structured outputs.

ts-agents workflow run forecast-series --output-dir outputs/forecast

report.md

run_manifest.json

forecast.csv

Stakeholder reports

A report-generation skill can turn a run manifest and artifacts into a Quarto or PDF report.

ts-agents skills show report-generation

quarto render outputs/reports/forecast.qmd

The interaction can be conversational, scheduled, or scripted. The deliverable is the artifact bundle.

SKILL.md + Mature CLIs

Start with mature CLI agents

They already provide repo search, command execution, logs, diffs, permissions, background jobs, and resumable sessions.

Inject domain expertise with skills

SKILL.md files encode tool order, cheap baselines, artifact checks, report expectations, and done criteria.

Custom agents are still possible

Build a dedicated harness when you need proprietary integrations, strict policies, non-developer workflows, or offline/low-latency deployment.

The practical path is incremental: stable CLI first, skills next, custom harness only when the use case justifies it.

Lesson: Tool Selection Needs Priors

Pain point

When many similar tools exist, agents often jump to expensive or fragile methods before checking cheap baselines.

How ts-agents helps

Tool metadata, bundles, and skills make the intended order explicit: cheap baselines, robust defaults, then advanced methods.

This matters for forecasting, classification, anomaly detection, and any workflow with multiple plausible algorithms.

Lesson: Long-Running Loops Break Chat

Pain point

Minutes-to-hours jobs need progress, logs, retry behavior, review checkpoints, and resumable state.

How ts-agents helps

Run-scoped output directories, manifests, artifact refs, cost metadata, and sandbox hooks make jobs batch-friendly.

The roadmap is to push this further: progress reporting, checkpoint recovery, and richer run history on top of the current background-job and run-catalog surfaces.

Lesson: Dependencies Need Sandboxes

Pain point

Time-series stacks are fragmented: optional libraries, compiled dependencies, version conflicts, and cloud/local differences.

How ts-agents helps

Keep imports lazy, detect install profiles, and use sandbox backends for isolation, reproducibility, and heavier workloads.

Sandboxes are not magic: data gravity, cold starts, cost governance, and infrastructure debugging still matter.

What Works Now

Stable surfaces

capabilities, workflow show, strict JSON envelopes, typed artifacts

Demo-ready workflows

Inspection, forecasting, activity recognition

Agent-friendly outputs

Run manifests, non-clobbering output dirs, artifact refs

Integration paths

External framework tool wrappers, packaged skills, mature CLI agents

Use Cases: Industry and Research

Process industry

Supplement simulation forecasts, scan sensor fields, flag anomalies, and summarize operating regimes.

Live or batched data

Run scheduled diagnostics, forecast refreshes, quality checks, and report generation over streaming or daily batch data.

Research teams

Compare experiments with repeatable commands, preserve artifact trails, and turn repeated analyses into reusable skills.

Custom harnesses

Use ts-agents as one callable subsystem inside a larger orchestrator with governance, data access, and review.

Roadmap: Autoresearch Loop

Orchestrator
plans · routes · reviews
Data
digital twins · telemetry · simulations
Tools (ts-agents)
CLIs · MCP
Evaluation
rubrics · metrics · reviewer signal
Loop: Orchestrator → Data → Tools (ts-agents) → Evaluation → back to Orchestrator
  • Run simulation campaigns and large analysis batches without inspecting every intermediate file.
  • Feed outcomes, artifacts, failed tool choices, and review decisions back into better orchestrator policies.

Autoresearch vs Policy Gradient

Policy gradient

  • The orchestrator samples actions
  • A reward function scores behavior
  • Optimizer updates orchestrator weights

Learns inside the policy.

Autoresearch

  • The LLM reads metrics, figures, reports, and traces
  • It proposes a hypothesis and next experiment
  • It can revise plans, skills, tools, and training data

Learns through artifacts and workflow changes.

Simple examples

Policy gradient: sample workflow choices, reward lower validation RMSE, then update orchestrator weights toward actions that picked better methods.

Autoresearch: Iteration 0 finds Theta wins on metrics, but the figure shows drift and the report flags short validation. Iteration 1 runs decomposition, widens baselines, and updates the forecasting skill.

Autoresearch can improve the orchestrator, but also the surrounding skill files, tool metadata, eval rubrics, and reusable workflow defaults.

Personalized Agents: OpenClaw / Hermes + ts-agents

What the agent remembers

  • Preferred analysis order: inspect first, cheap baselines next, advanced methods only when justified
  • Domain defaults: variables, time scales, quality checks, missing-data tolerance, validation style
  • Communication preferences: executive summary, research appendix, plots to prioritize, caveats to call out

Where ts-agents fits

  • OpenClaw / Hermes-style agents provide persistent memory, scheduling, and personalization
  • ts-agents turns remembered preferences into repeatable workflow commands and artifact checks
  • Enterprise form factor: governed personal/team agents calling vetted CLIs in sandboxes with approvals, logs, and artifact stores

The personal agent learns the user’s data-science playbook; ts-agents keeps each run auditable and reproducible.

Takeaways

  1. Use the CLI as the durable contract across agents, scripts, and CI.

  2. Treat artifacts as the product: plots, JSON, CSV, reports, and manifests.

  3. Pair skills with memory so personalized agents choose methods, checks, and reporting style consistently.

Closing thought: ts-agents is strongest as a specialist subsystem inside a larger governed agent workflow.

Thanks!

Open source repo

github.com/fnauman/ts-agents

Docs, examples, skills, and demo scripts are in the repository.