llama_hf_loader library

Loads HuggingFace Llama (llama / LlamaForCausalLM architecture) weights from a safetensors file into a Llama.

Supported checkpoints (all pre-Llama-4, use RMSNorm + SwiGLU + RoPE + GQA — same recipe for Llama 1/2/3/3.1/3.2, Mistral 7B, Qwen 2, etc.):

meta-llama/Llama-3.2-1B-Instruct — 16 layers, 2048 embed, 32 heads, 8 kv-heads, ffn 8192, ropeBase 500000, tied head.
meta-llama/Llama-3.2-3B-Instruct — 28 layers, 3072 embed, 24 heads, 8 kv-heads, ffn 8192, ropeBase 500000, tied head.
meta-llama/Llama-3.1-8B-Instruct — 32 layers, 4096 embed, 32 heads, 8 kv-heads, ffn 14336, ropeBase 500000, untied.

HF key conventions handled here (Llama's namespace has no leading gpt_neox. — top-level model. prefix):

model.embed_tokens.weight — [V, D]
model.layers.{i}.input_layernorm.weight — [D]
model.layers.{i}.self_attn.q_proj.weight — [H·hd, D] (H = numHeads, hd = headDim). Row-split into per-head slices.
model.layers.{i}.self_attn.k_proj.weight — [Hkv·hd, D]
model.layers.{i}.self_attn.v_proj.weight — [Hkv·hd, D]
model.layers.{i}.self_attn.o_proj.weight — [D, D]
model.layers.{i}.post_attention_layernorm.weight — [D]
model.layers.{i}.mlp.gate_proj.weight — [F, D]
model.layers.{i}.mlp.up_proj.weight — [F, D]
model.layers.{i}.mlp.down_proj.weight — [D, F]
model.norm.weight — [D]
lm_head.weight — [V, D] (present only when the model does not tie its head, e.g. Llama 3.1 8B; ignored when tied).

Ignored (safe): per-layer self_attn.rotary_emb.inv_freq — we recompute RoPE tables from config.ropeBase.

core/nn/llama_hf_loader library