tokenTypeIds property

List<int>? tokenTypeIds
final

Token-type (segment) ids aligned 1:1 with ids. null for tokenizers that don't use them (Gemma SentencePiece); all-zeros for single-segment WordPiece input.

Implementation

final List<int>? tokenTypeIds;