tiktoken/src/common/utils library

Functions

bytePairEncode(ByteArray piece, HashMap<ByteArray, int> ranks) → List<int>

listEquals<T>(List<T> a, List<T> b) → bool

Typedefs

BytePairCallback<T> = T Function(int start, int end)