worker_protocol library
Flutter-free toolkit for building a custom tokenize Web Worker entry
point against shiki_flutter's async protocol.
The built-in worker (installed by dart run shiki_flutter:install)
tokenizes with the embedded or pure-Dart port engine. A backend that can't
live in the core package (e.g. shiki_flutter_native_engine, which pulls in
a native build hook) ships its own separately-compiled worker instead. Such
an entry compiles against this library (plus the Flutter-free engine.dart)
so it never has to reach into shiki_flutter's src/.
It exposes exactly what a DedicatedWorkerGlobalScope message handler needs:
the JSON codec for the values that cross the postMessage structured-clone
boundary, the engine tag the main isolate selected, and the language
descriptor round-trip used to rehydrate grammars sent as JSON.
Holds no dart:isolate / dart:js_interop / dart:ui import: callers do
their own jsonEncode/jsonDecode and JS interop.
Classes
- LangDescriptor
- A plain, isolate-sendable description of a CodeLanguage and the grammars it embeds.
Functions
-
langDescriptorFromJson(
Map< String, dynamic> j) → LangDescriptor -
Decodes a LangDescriptor previously encoded by
langDescriptorToJson. -
rebuildBundledLanguage(
LangDescriptor d) → CodeLanguage -
Rebuilds a CodeLanguage from a LangDescriptor inside the worker, with
an
embeddedLanguagesthunk that closes over the descriptor's children. -
tokenizeOptionsFromJson(
Map< String, dynamic> j) → TokenizeOptions -
Decodes a TokenizeOptions previously encoded by
tokenizeOptionsToJson. Missing keys fall back to per-field defaults. -
tokensToJson(
List< List< lines) → ListThemedToken> > -
Encodes a per-line token list (as returned by tokenization) to JSON,
applying
themedTokenToJsonto each token. -
workerConfigFromJson(
Map< String, dynamic> j) → WorkerConfig -
Rebuilds a
WorkerConfigfrom JSON. engine is intentionally left null: each prebuilt worker is compiled for one engine (the main isolate picks which worker script to load fromengineTag), so no engine backend needs to be reconstructed (or depended on) here.