Logger and Signals¶
Weightslab logger behavior is similar in spirit to TensorBoard: it tracks scalar evolution and per-sample context.
What gets logged¶
Scalar signals (losses, metrics)
Per-sample signal vectors
Per-step model signals (gradient/weight norms, activation statistics)
Optional predictions/targets for deeper analysis
Two kinds of signal¶
Signals divide by what a value is about, and that decides which verb records it:
Keyed by |
Verb |
Example |
|---|---|---|
Sample |
|
The classification loss of one image. |
Annotation |
|
The IoU of one bounding box. |
Group |
|
A contrastive loss over an image pair. |
Step |
|
The gradient norm of layer 5 at step 900. |
The first three write onto dataframe rows; the sample grid can then be sorted
and filtered by them. The fourth does not — a gradient norm belongs to the
optimization step that produced it, not to any of the samples in the batch, so
it is plotted as a curve and nothing else. Recording it with save_signals
would mean broadcasting one number across a whole batch of ids and polluting
every one of those samples’ history with a value that was never about them.
Default plot order¶
The plots board groups curves by signal-name prefix, in this order:
Your experiment’s signals — losses, metrics, and the whole-model
metrics/global/*norms. These are what the board is for, so they stay at the top.Per-layer model signals — everything under
metrics/layer/(see track_model_signals).Resource monitors — everything under
resource/(CPU, memory, disk, network, GPU and process telemetry).
The grouping exists because arrival order stops being usable once model signals
are on: track_model_signals can emit dozens of metrics/layer/* curves in
a single step (74 for the Fashion-MNIST example) and resource monitoring is
enabled by default, so an unordered board buries the loss curve under
telemetry. Note that metrics/global/* deliberately sits in the first
group — a whole-model gradient norm is read next to the loss, not scrolled past
70 per-layer curves.
This is only a default. Dragging a card puts it exactly where you drop it and
that arrangement is remembered, per browser; signals that appear later (a
metrics/layer/* curve showing up once training starts) are filed into their
group without disturbing anything you have already arranged.
Start services¶
import weightslab as wl
wl.serve(serving_cli=True, serving_grpc=True)
Wrap losses and metrics as signals¶
The simplest way to produce signals is to wrap a loss or metric with
wl.watch_or_edit. It hooks the object’s forward (losses) or compute
(torchmetrics) method, so every call computes, logs, and persists
per-sample values automatically — no manual save_signals needed.
import torch.nn as nn
import weightslab as wl
train_loss = wl.watch_or_edit(
nn.CrossEntropyLoss(reduction="none"),
flag="loss",
signal_name="train/loss",
per_sample=True,
log=True,
)
val_loss = wl.watch_or_edit(
nn.CrossEntropyLoss(reduction="none"),
flag="loss",
signal_name="val/loss",
per_sample=True,
log=True,
)
with wl.guard_training_context:
loss_per_sample = train_loss(train_logits, train_targets, batch_ids=train_ids)
with wl.guard_testing_context:
_ = val_loss(val_logits, val_targets, batch_ids=val_ids)
wl.save_signals(
signals={"train/confidence": conf_per_sample},
batch_ids=train_ids,
preds_raw=train_logits,
preds=train_preds_processed,
targets=train_targets,
log=True,
)
Signal-shape classification¶
Per-sample trajectories can be classified into categorical tags (for example monotonic / plateaued / forgotten) and used by Studio, agent flows, and reports. See Signal Trajectory Classification for the concept and User Functions Reference for classifier customization APIs.
Standalone logger-only integration (UI + CLI ready)¶
A complete, runnable MNIST script where the only wrapped objects are the loss and the metric. The model, the optimizer and the loaders are plain PyTorch, and no configuration is registered, yet the run produces real train/eval curves, a persisted history export, and a CLI/UI report.
Bundled example: weightslab/examples/PyTorch/wl-standalone-logger/main.py
weightslab start example --logger # run it (MNIST downloads on first run)
weightslab cli # attach a terminal, in another shell
weightslab start # open Weights Studio, in a third shell
def main(argv=None) -> int:
args = parse_args(argv)
# The logger level needs an experiment directory for history/report output but
# NOT the config level: WEIGHTSLAB_ROOT_LOG_DIR is the documented way to give
# an otherwise unconfigured run one.
log_dir = resolve_log_dir(args.log_dir, "standalone_logger")
os.environ["WEIGHTSLAB_ROOT_LOG_DIR"] = str(log_dir)
import weightslab as wl
device = "cuda" if torch.cuda.is_available() else "cpu"
train_loader, eval_loader = build_loaders(
args.data_root, args.batch_size, args.max_samples)
model = SmallCNN().to(device) # not wrapped
optimizer = optim.Adam(model.parameters(), lr=args.lr) # not wrapped
# --- the only WeightsLab registrations in this file ------------------------
train_loss = wl.watch_or_edit(
nn.CrossEntropyLoss(reduction="none"),
flag="loss",
signal_name="train/loss",
log=True,
)
eval_loss = wl.watch_or_edit(
nn.CrossEntropyLoss(reduction="none"),
flag="loss",
signal_name="eval/loss",
log=True,
)
eval_acc = wl.watch_or_edit(
Accuracy(task="multiclass", num_classes=10).to(device),
flag="metric",
signal_name="eval/accuracy",
log=True,
)
# --------------------------------------------------------------------------
wl.serve(serving_grpc=not args.no_grpc, serving_cli=not args.no_cli,
grpc_port=args.grpc_port)
print("=" * 70)
print(" LOGGER-ONLY standalone — attach with `weightslab cli`, UI with `weightslab start`")
print(f" signals: train/loss, eval/loss, eval/accuracy log_dir={log_dir}")
print("=" * 70)
wl.start_training(timeout=3)
batches = iter(train_loader)
for step in range(1, args.steps + 1):
try:
inputs, targets = next(batches)
except StopIteration:
batches = iter(train_loader)
inputs, targets = next(batches)
with wl.guard_training_context:
inputs, targets = inputs.to(device), targets.to(device)
optimizer.zero_grad()
# step= is what places this point on the x-axis without a wrapped model.
per_sample = train_loss(model(inputs), targets, step=step)
per_sample.mean().backward()
optimizer.step()
if step % args.eval_every == 0:
with wl.guard_testing_context, torch.no_grad():
eval_inputs, eval_targets = next(iter(eval_loader))
eval_inputs = eval_inputs.to(device)
eval_targets = eval_targets.to(device)
logits = model(eval_inputs)
eval_loss(logits, eval_targets, step=step)
eval_acc.update(logits, eval_targets)
accuracy = eval_acc.compute(step=step)
eval_acc.reset()
print(f"[step {step:>5}] train/loss={per_sample.mean().item():.4f} "
f"eval/accuracy={float(accuracy) * 100:.1f}%")
# Persist the signal history; this is also what the report and the UI plots read.
history = wl.write_history(
path=str(log_dir / f"history.{args.history_format}"),
format=args.history_format,
)
print(f"[logger-level] history written to {history}")
print("[logger-level] `report --no-agent` in `weightslab cli` renders it as HTML")
wl.keep_serving(timeout=args.serve_timeout)
return 0
Important
Pass step= when no model is wrapped. The x-axis of a signal normally comes
from the registered model’s age; with the model level absent, the caller’s
step is what places the point — otherwise every value would land on the same
step. A wrapped model always wins over the argument, so the same call is correct
in a full integration.
Scope of a logger-only run:
step-level curves (
log=True) need nothing else — that is what the script above shows.per-sample / per-instance signals (
per_sample=True,per_instance=True,wl.save_signals(...)) route values to sample ids in the sample dataframe, which exists only once a dataset is tracked. Addflag="data"(see Data Exploration) when you want those.
The model level writes into this same history on its own (model/grad_norm,
model/parameters — see Model Interaction), so the two levels compose
without either being required.
CLI and UI surfaces¶
CLI:
statusevaluate/eval_status(needs a registered loader to evaluate; see Custom Evaluation Function to override what actually runs)report [--no-agent]— renders the logged history as HTML under<root_log_dir>/reports/
UI:
live signal plots
sample ranking by signal values
report button and signal diagnostics