BpeTokenizerClearing constructor
const
BpeTokenizerClearing(
- MainBpeTokenizerState mainBpeTokenizerState
Implementation
const BpeTokenizerClearing(MainBpeTokenizerState mainBpeTokenizerState)
: super(mainBpeTokenizerState);