core/nn/vision_projector library
Small MLP that projects vision-encoder features into a language model's embedding space.
Two Linear layers with SiLU in the middle:
y = downProj( silu(upProj(x)) )
Shape: [N, inDim] → [N, outDim].
Used by LlamaVision to convert ViTBackbone per-patch outputs
(dim vitEmbedDim) into image tokens that can be prepended to
Llama's text embeddings (dim llamaEmbedDim).
This is the same shape of module LLaVA calls "mm_projector" — two dense layers, a nonlinearity, no bias tie. LLaVA uses GELU where we use SiLU (matches the rest of the Llama stack).