SDK and CLI integration

Docs

Install the published Python metrics collector, attach each training run to a stable folder id when needed, then inspect experiments, metrics, and annotations from your workspace or the command line.

API Keys Experiments

1. Install collector

Add the PyPI collector to your training project.

uv add syvain-metrics-collector

2. Create API key

Create an organization key before the run starts.

API Keys

3. Collect metrics

Wrap training in experiment.run() and send metric calls.

SDK example

4. Observe metrics

Use the dashboard charts or inspect runs with syvain-metrics.

Dashboard CLI

Python SDK

Training instrumentation

PyPI collector instrumentation for training jobs.

syvain-metrics-collectorExperiment tracker and metrics collector for training, evaluation, and analysis jobs.

uv add syvain-metrics-collector

Package docs

from syvain_metrics_collector import Collector

collector = Collector(api_key="ak_org_...")

experiment = collector.experiment(
    slug="mamba-run-001",
    description="Baseline mamba training run",
    folder_id="00000000-0000-0000-0000-000000000000",  # Optional dashboard folder id.
    meta={
        "model": "mamba",
        "dataset": "internal-v1",
        "git_sha": "abc123",
    },
)

with experiment.run():
    for step in range(1_000):
        loss = 1.0 / (step + 1)
        experiment.metric("train_loss", loss, step=step)
        experiment.metric(
            "gpu_utilization",
            78.0,
            step=step,
            metadata={"device": "gpu:0"},
        )

    experiment.annotation(
        "saved checkpoint at step 999",
        metadata={"path": "checkpoints/run-001/step-999.pt"},
    )

CLI

Command line inspection

Installed syvain-metrics commands for folders, lookup, metrics, and annotations.

syvain-metricsPublished npm package that installs the syvain-metrics command.

npm install -g syvain-metrics

Package docs

# Use the latest locally installed syvain-metrics CLI.
syvain-metrics --help
syvain-metrics auth --help
syvain-metrics folders --help
syvain-metrics experiments --help

# Authenticate once. Paste the organization API key when prompted.
syvain-metrics auth login
syvain-metrics auth status

# Create or find the folder id to pass into collector.experiment(...).
syvain-metrics folders create --name dendro-mamba
syvain-metrics folders list --path /dendro-mamba
syvain-metrics folders get --id 00000000-0000-0000-0000-000000000000

# Find an experiment.
syvain-metrics experiments list --limit 20
syvain-metrics experiments get --slug dendro-mamba-run-001
syvain-metrics experiments move --id exp_... --folder-id folder_...

# Inspect metrics and metadata through query endpoints.
syvain-metrics experiments metrics --id exp_... --metric-name loss
syvain-metrics experiments query metric-series \
  --series '{"experimentId":"exp_...","metricName":"loss","metadataFilter":[{"key":"split","value":"train"}]}'
syvain-metrics experiments render metric-series \
  --series '{"experimentId":"exp_...","metricName":"loss","splitKeys":["data_split","curriculum_split"]}' \
  --output metrics.png
syvain-metrics experiments query metric-catalog --experiment-ids exp_...

# Attach and inspect annotations.
syvain-metrics experiments annotate --id exp_... \
  --annotation "saved checkpoint at step 999" \
  --metadata-json '{"path":"checkpoints/run-001/step-999.pt"}'
syvain-metrics experiments annotations --id exp_... --limit 50

API reference

Core HTTP surface

Generated from the same OpenAPI document served by this app. The dashboard, SDK, and CLI use this organization-scoped API surface.

syvain-metrics-api-clientFully typed Python package for the REST API.

uv add syvain-metrics-api-client

Package docs

OpenAPI JSON

Loading API reference.