BpeTokenizerState constructor

const BpeTokenizerState(MainBpeTokenizerState mainBpeTokenizerState)

Implementation

const BpeTokenizerState(this.mainBpeTokenizerState);