tokenizer_parser 0.1.2

A lightweight tokenizer and parser-composition toolkit for Dart, designed to build custom language grammars and structured token streams.

0.1.0

• Initial version.

0.1.1

• Bump dependencies

0.1.2

• Improve package metadata and expand README documentation for pub.dev quality.
• Add API documentation comments across exported/public tokenizer primitives.
• Add new tests for core model behavior and tokenizer file/string workflows.
• Keep dependencies on latest available compatible versions.