saveJson method

String saveJson()

Serialize the tokenizer to a compact JSON string.

Implementation

String saveJson() => jsonEncode({
  'version': 1,
  'kind': 'byte-bpe',
  'vocabSize': vocabSize,
  'merges': merges,
});