isSiglip2TokenizerJson function

bool isSiglip2TokenizerJson(Map<String, dynamic> json)

True when json — a decoded HuggingFace tokenizer.json — declares the SigLIP 2 text-tower convention: a fixed-width right padding block, and a post-processor that appends an EOS without prepending a BOS.

Both signals are required. A fixed padding block alone is not enough (other models pad), and a BOS-less template alone is not enough (a plain BPE export may declare no post-processor at all). Missing or malformed blocks answer false rather than throwing: the question is "does this file declare SigLIP 2", and anything it cannot establish is a no.

Implementation

bool isSiglip2TokenizerJson(Map<String, dynamic> json) =>
    _hasFixedPadding(json) && _appendsEosWithoutBos(json);