plusparse/stream_splitter library
Splits Markdown source into independently-parsable top-level segments, used by gpt_markdown's incremental rendering mode.
Segments are separated by blank lines, except inside fenced code blocks
(```) and block LaTeX (\[ ... \]), which stay whole. Each segment can be
parsed and rendered on its own; during streaming only the last segment's
text changes, so all earlier segments' widgets can be cached and reused —
that caps per-chunk rebuild/layout cost at the tail instead of the whole
message.
Divergence from a full-document parse: a list whose items are separated by blank lines becomes multiple adjacent list segments instead of one list. Rendering is visually equivalent (every item is its own row widget either way).
Classes
- MarkdownSegmentCache
- Retains settled segment strings between appends. Only the previous tail and appended source are split again. Edits or CR normalization fall back to a full split. Callers still compare source prefixes; this is not O(1).
Functions
-
splitStreamSegments(
String src, {MarkdownBlockRegistry? blockRegistry}) → List< String>