core/coop/lm_factory library
Small factory that produces either a GPT (attention-based) or
an AFTLanguageModel (attention-free) language model behind a
uniform interface, so the coop_* training scripts can flip
architectures with a single --arch=gpt|aft flag without
duplicating the training loop.
Both models already accept a rank-1 [seqLen] token tensor and
return rank-2 [seqLen, vocabSize] logits — the training loops in
bin/coop_*.dart never batch beyond that, so the two are drop-in
interchangeable at the .call(tokens) level.
DPTC compatibility: averageCheckpoints requires that every
input describes the same architecture (byte-identical DPTC header).
Mixing a GPT checkpoint with an AFT checkpoint will therefore
raise a clean ArgumentError at aggregation time — the coop
coordinator (Phase 1) advertises the arch via GET /config and
workers refuse to join a fleet whose arch doesn't match theirs.
Classes
- CoopLM
-
A language model wrapped so callers can treat GPT and AFT
identically: forward with
.call(tokens), get parameters via.module.parameters(), checkpoint viaCheckpoint.saveBytes(.module).
Enums
- Arch
- Which transformer family to build. Serialized as its lower-case name in JSON so coordinator ↔ worker exchange is trivial.
Functions
-
archToString(
Arch a) → String -
buildCoopLM(
{required Arch arch, required String modelSize, required int vocabSize, required Device device, required int seed}) → CoopLM -
Build a fresh model.
modelSizeistinyorsmalland picks matching dims for both families (so a 3-peer fleet running --arch=aft --model=tiny has the same param budget as one running --arch=gpt --model=tiny, within ~10%). -
parseArch(
String s) → Arch