token2json static method
Convert a Donut token sequence back to a JSON object.
Parses the special token format back into structured data.
Example:
token2json('<s_menu><s_nm>Latte</s_nm></s_menu>')
// → {'menu': {'nm': 'Latte'}}
Implementation
static dynamic token2json(String tokenStr) {
// Remove BOS/EOS tokens
tokenStr = tokenStr
.replaceAll(RegExp(r'</?s>'), '')
.replaceAll(RegExp(r'<s_[^>]+>$'), '') // trailing open tags
.trim();
return _parseTokensToJson(tokenStr);
}