RundialDocs

Appearance

Log a training run ​

Record metrics and configuration for a training run.

Start Rundial and create an API key with the Quickstart. Keep the stack running and set RUNDIAL_API_KEY.

Start and log a run ​

python
import rundial as rd

with rd.init(
    workspace="default-workspace",
    project="default-project",
    name="training-baseline",
    config={"learning_rate": 0.0003, "batch_size": 32},
) as run:
    for step in range(100):
        run.log({"train/loss": 1 / (step + 1)}, step=step)

Use metric names such as train/loss and validation/loss to distinguish series. Pass step to put values at a specific training step. Open the run's Metrics tab to see the charts.

rd.init optionUse
nameSet the run's display name.
configRecord a mapping of parameters.
kindSet the run kind; the default is training.
modeUse online (default), offline, or disabled. Offline saves data locally; disabled turns logging off. Disabled mode requires omitting workspace and unsetting RUNDIAL_WORKSPACE.
run_id and resumeSupply a run ID and choose never (default), allow, or must when starting or resuming a run.

Use with to finish the run when the block exits. If you create a run without with, call run.finish() at the end. Call run.flush() when you need to attempt delivery before the end of the run; check its return value for success.

Environment variables ​

VariableUse
RUNDIAL_API_KEYAPI key used when api_key is omitted.
RUNDIAL_WORKSPACEWorkspace used when workspace is omitted.
RUNDIAL_PROJECTProject used when project is omitted.
RUNDIAL_RANKProcess rank used for distributed logging when distributed_rank is omitted.

For an API on another host, pass endpoint to rd.init.

Distributed training ​

The default distributed="rank0" logs on rank 0. Set distributed="all" to log on every rank. Without distributed_rank, the SDK reads the rank from RUNDIAL_RANK, RANK, LOCAL_RANK, SLURM_PROCID, OMPI_COMM_WORLD_RANK or PMI_RANK.

When the API is unreachable ​

If the API goes down during a run, keep logging. The SDK sends the stored points when the API is back. When the local spool reaches its size or age limit, the oldest stored points are dropped; a full queue or a disk error can also drop points. To check, print run.diagnostics() after the run: the dropped_points_* counts are 0 when nothing was lost, and last_error shows the most recent failure. To log with no API at all, pass mode="offline" and the project ID (proj_default-project for the quickstart project) as project, then run your next online run from the same directory to send the data.

Next steps ​