gpt library

A minimal GPT-style causal language model.

Compared to TransformerLM this module follows the GPT-2 recipe:

Learned positional embeddings of shape [maxCtx, embedDim] (not sinusoidal).
An embedding dropout applied to tokenEmb + posEmb before the encoder.
The output head is weight-tied to the token embedding when tieWeights: true (default). No separate Linear and no head bias — the token-embedding matrix [V, D] is transposed and used to project the final hidden state [N, D] to logits [N, V]. Because the same tensor appears in both the embedding lookup and the head matmul, its gradient accumulates contributions from both paths automatically.
A trailing LayerNorm inside the TransformerEncoder (finalNorm: true, standard for pre-LN GPT).
A generate method that samples autoregressively with greedy, temperature, or top-k modes and truncates context to maxCtx.

Same 2D-only, single-sequence tensor convention as the rest of the library: tokens is a 1D [seqLen] tensor of class indices stored as float32.

core/nn/gpt library