core/optim/lr_scheduler library

Learning-rate schedulers.

A LRScheduler wraps an Optimizer and adjusts its lr in place via a step() call the training loop invokes once per optimizer step. All schedulers here are stateful (they track their own step counter) and idempotent under repeated lastLr reads.

The typical pattern:

final opt = Adam(model.parameters(), lr: 1.0); // base lr = 1.0
final sched = LinearWarmupCosineDecay(
  opt,
  warmupSteps: 100,
  totalSteps: 1000,
  maxLr: 3e-4,
  minLr: 3e-5,
);
for (int i = 0; i < 1000; i++) {
  opt.zeroGrad();
  loss(model, x, y).backward();
  opt.step();
  sched.step();
}

Note: schedulers overwrite optimizer.lr on every step(); the value the optimizer was constructed with is not used as the base LR — the scheduler carries its own maxLr / initialLr.

Classes

LinearWarmupCosineDecay

Linear warmup for the first warmupSteps steps, then cosine decay from maxLr down to minLr across the remaining totalSteps - warmupSteps steps. After totalSteps the LR stays at minLr.

LRScheduler

StepLR

Multiplies the LR by gamma every stepSize steps.