llm_dataset library

Classes

CallbackTranslationVariationGenerator: Translation variations backed by a user-supplied translate callback.
ClassificationGenerator: Emits a classification example with an optional label from metadata.
CodingExerciseCatalog: Catalog of built-in coding exercises.
CodingExerciseGenerator: Builds coding exercise entries from CodingPatternConfig definitions.
CodingPatternConfig: Configuration for one coding exercise pattern.
CompositeValidator: Runs multiple validators and merges their results.
CurriculumBuilder: Builds curriculum stages into a shared DatasetStore.
CurriculumBuildResult: Summary of a CurriculumBuilder invocation.
CurriculumDataset: Training consumption API for curriculum-tagged dataset slices.
CurriculumLifecycle: Curriculum-aware lifecycle helpers over a DatasetStore.
CurriculumManifest: Top-level curriculum manifest loaded from JSON.
CurriculumMetadataKeys: Curriculum-specific metadata keys stamped during progressive training builds.
CurriculumMixing: Interleaves primary and review stage entry streams per mixing policy.
CurriculumMixingPolicy: Mixing policy for a curriculum stage at training time.
CurriculumPhaseDataset: A Dataset view for one curriculum stage, including mixed review interleaving.
CurriculumSourceDefinition: Declarative input for a stage builder.
CurriculumStageBuilderRegistry: Registry that resolves CurriculumSourceDefinition into build plans.
CurriculumStageBuildPlan: Resolved source + generator for one curriculum stage source entry.
CurriculumStageBuildResult: Summary of building one curriculum stage.
CurriculumStageDefinition: One curriculum training phase.
CurriculumTaggingGenerator: Wraps a DatasetGenerator and stamps curriculum metadata on each entry.
CurriculumVariationOptions: Controls meaning-preserving variation generation during curriculum builds.
Dataset: Dataset consumption API for training packages.
DatasetEntry: A single training example in an LLM dataset.
DatasetExporter: Exports DatasetEntry values to JSON or JSONL.
DatasetGenerator: Produces canonical DatasetEntry values from a source document.
DatasetImporter: Imports DatasetEntry values from JSON or JSONL.
DatasetLifecycle: Dataset naming and version management over a DatasetStore.
DatasetPipeline: Incremental pipeline: source → generator → variations → validators → store.
DatasetPipelineResult: Summary of a DatasetPipeline.run invocation.
DatasetProvenance: Origin and transformation history for a DatasetEntry.
DatasetQuery: Fluent, immutable query builder over a DatasetStore.
DatasetQuerySpec: Immutable filter / sampling specification for DatasetQuery.
DatasetSource: Stream-based producer of DatasetSourceDocument values.
DatasetSourceDocument: A raw document loaded from a DatasetSource before generation.
DatasetSpecialTokens: The target model's special tokens, as plain strings.
DatasetStore: Persistent or in-memory storage for DatasetEntry values.
DatasetTextFormatter: Renders DatasetEntry values into the flat text a trainer packs.
DatasetToolCall: A single tool / function invocation within a training example.
DatasetValidationResult: Outcome of validating a single DatasetEntry.
DatasetValidator: Validates a DatasetEntry before persistence.
DatasetVariationGenerator: Produces variation entries linked to a canonical (or parent) entry.
DirectorySource: Streams text files from a directory as DatasetSourceDocuments.
DuplicateValidator: Detects duplicate ids and optional content fingerprints without requiring the full dataset in memory when a DatasetStore is provided.
EmptyContentValidator: Rejects entries with empty DatasetEntry.input (and empty output when requireOutput is true).
ExerciseCatalog<TPattern>: Shared catalog contract for built-in exercise modules.
ExerciseMetadataKeys: Unified metadata keys used across exercise generators and curriculum tagging.
ExtractionGenerator: Emits an extraction example targeting structured fields.
GeneratorConfig: Shared configuration for built-in generators.
JsonlSource: Streams DatasetSourceDocuments from a JSON Lines file.
LanguageBasicsCatalog: Catalog of simple language-basics patterns for initial curriculum stages.
LanguageBasicsGenerator: Builds simplest phrase entries for early curriculum stages.
LanguageValidator: Restricts DatasetEntry.language to an allow-list.
LengthValidator: Configurable min/max character lengths for input/output.
LogicExerciseCatalog: Catalog of built-in logic exercises.
LogicExerciseGenerator: Builds logic exercise entries from LogicPatternConfig definitions.
LogicPatternConfig: Configuration for one logic exercise pattern.
MathExerciseCatalog: Catalog of built-in math exercises.
MathExerciseGenerator: Builds math exercise entries from MathPatternConfig definitions.
MathPatternConfig: Configuration for one math exercise pattern.
MeaningPreservingVariationGenerator: Produces meaning-preserving surface variations by swapping lexicon alternates.
MemoryDatasetStore: In-memory DatasetStore suitable for tests and small datasets.
MemorySource: DatasetSource that yields an in-memory collection of documents.
MetadataValidator: Requires specific metadata keys (and optionally non-null values).
ParagraphGenerator: Builds canonical paragraph entries from ParagraphPatternConfig.
ParagraphPatternConfig: Configuration for a multi-sentence paragraph.
PhraseCombination: One fully resolved phrase from a template, lemma assignment, and surface forms.
PhraseCombinationExpander: Expands phrase templates across all category word and synonym combinations.
PhraseCombinatorialGenerator: Generates every phrase combination for each template document.
PhraseGenerator: Builds canonical phrase entries from PhrasePatternConfig definitions.
PhrasePatternConfig: Configuration for building one phrase from a template and semantic slots.
PhraseTemplate: Declarative phrase template: {slot} placeholders filled from categories.
PhraseTemplateLibrary: A language-specific collection of phrase templates and a word category bank.
PhraseTemplateLocaleManifest: Index manifest for a locale directory under a phrase template store.
PhraseTemplateStore: Loaded phrase-template dataset assembled from JSON files.
PhraseTemplateStoreCodec: Parses and validates phrase template store JSON documents.
PhraseTemplateStoreLoader: Loads PhraseTemplateStore datasets from a locale directory on disk.
QuestionAnswerGenerator: Emits a simple question/answer pair derived from the document.
RuleBasedTranslationVariationGenerator: Deterministic stub translator for tests and offline demos.
RuleBasedVariationGenerator: Built-in, provider-independent variation generator.
SentencePatternConfig: One sentence inside a paragraph pattern.
SqliteDatasetStore: SQLite-backed DatasetStore for datasets larger than available RAM.
SummarizationGenerator: Emits a summarization example from the document.
TextExerciseMetadata: Metadata keys stored on generated text entries for variation rebuild.
TextGenerator: Emits a plain-text completion entry from the document content.
TextLexicon: Lexicon of interchangeable words for meaning-preserving text variations.
TokenCountValidator: Configurable approximate token-count bounds (whitespace tokenization).
ToolCallGenerator: Emits a tool-calling dialogue skeleton from the document.
TrainingPhaseDescriptor: Descriptor exported for external shallow-model trainers.
TranslationGenerator: Emits a translation-style example (source language → target label).
TranslationVariationGenerator: Base class for translation DatasetVariationGenerator implementations.
VariationGenerateOptions: Options passed by DatasetPipeline when invoking variation generators.
WordCategoryBank: Groups lexicon lemma keys into semantic categories for template filling.

Enums

CodingPatternKind: Kind of coding exercise pattern.
DatasetEntryType: Semantic category of a DatasetEntry.
DatasetPairRendering: How an input/output pair is written out.
LogicPatternKind: Kind of logic exercise pattern.
MathPatternKind: Kind of math exercise pattern.
PipelineStoreErrorPolicy: How DatasetPipeline handles store failures while persisting entries.
VariationSelection: How variation groups are selected in a query or training stream.
VariationStrategy: Named variation strategies supported by built-in generators.

Mixins

CatalogExerciseGenerator<TPattern>: Resolves DatasetSourceDocument.id against a catalog patternMap.

Extensions

CurriculumStageDefinitionCopy on CurriculumStageDefinition: Extension helpers for stage definition copies (testing / export overrides).
DatasetStoreCapabilities on DatasetStore: Optional store capabilities beyond the core DatasetStore interface.

Constants

datasetMessagesMetadataKey → const String: Well-known metadata key for ordered chat / tool / agent message turns.
defaultVariationGenerateOptions → const VariationGenerateOptions: Default options when callers invoke generators directly.
englishParagraphPatterns → const List<ParagraphPatternConfig>: English paragraph patterns for text exercises.
englishPhraseLibrary → const PhraseTemplateLibrary: English phrase template library with word categories.
englishPhraseTemplateLibrary → const PhraseTemplateLibrary: English phrase templates filled from englishWordCategoryBank.
englishTextLexicon → const TextLexicon: English lexicon for phrase/paragraph exercises.
englishWordCategoryBank → const WordCategoryBank: English word categories for phrase templates.
portugueseParagraphPatterns → const List<ParagraphPatternConfig>: Portuguese paragraph patterns for text exercises.
portuguesePhraseLibrary → const PhraseTemplateLibrary: Portuguese phrase template library with word categories.
portuguesePhraseTemplateLibrary → const PhraseTemplateLibrary: Portuguese phrase templates filled from portugueseWordCategoryBank.
portugueseTextLexicon → const TextLexicon: Portuguese lexicon for phrase/paragraph exercises.
portugueseWordCategoryBank → const WordCategoryBank: Portuguese word categories for phrase templates.

Properties

englishPhrasePatterns → List<PhrasePatternConfig>: English phrase patterns (category-driven templates).
no setter
portuguesePhrasePatterns → List<PhrasePatternConfig>: Portuguese phrase patterns (category-driven templates).
no setter

Functions

allTextExerciseDocuments() → MemorySource: Runs phrase then paragraph pipelines for English and Portuguese.
applyQueryPostProcessing(List<DatasetEntry> filtered, DatasetQuerySpec spec) → List<DatasetEntry>: Applies ordering, variation selection, sampling, offset, and limit in Dart.
buildFromTemplate(String template, Map<String, String> slots) → String: Fills {slot} placeholders in template with slots.
buildParagraphSentenceSpecs(ParagraphPatternConfig pattern, TextLexicon lexicon) → List<Map<String, dynamic>>: Sentence specs with resolved slots for storage in entry metadata.
codingExerciseCatalogFor({required String language}) → CodingExerciseCatalog: Returns the coding catalog for language.
curriculumExerciseMetadata({required String curriculumId, required String stageId, required int stageOrder, int complexityTier = 0}) → Map<String, dynamic>: Builds curriculum metadata to merge into DatasetEntry.metadata.
datasetEntryTypeFromName(String name) → DatasetEntryType: Parses a stable wire name into DatasetEntryType.
englishParagraphDocuments() → List<DatasetSourceDocument>: Source documents for English paragraph patterns.
englishParagraphPatternMap() → Map<String, ParagraphPatternConfig>: Map of English paragraph patterns by id.
englishPhraseDocuments() → List<DatasetSourceDocument>: Source documents for English phrase patterns.
englishPhrasePatternMap() → Map<String, PhrasePatternConfig>: Map of English phrase patterns by id.
evaluateMathOperands(int a, int b, String operatorSymbol) → int: Evaluates a operatorSymbol b.
flattenParagraphSpecs(List<Map<String, dynamic>> sentenceSpecs) → ({Map<String, String> semanticKeys, Map<String, String> slots}): Flattens paragraph sentence specs into prefixed slot / semantic-key maps.
languageBasicsCatalogFor({required String locale, required String profile}) → LanguageBasicsCatalog: Returns the language-basics catalog for locale and profile.
logicExerciseCatalogFor({required String locale}) → LogicExerciseCatalog: Returns the logic catalog for locale.
mathExerciseCatalogFor({required String locale}) → MathExerciseCatalog: Returns the math catalog for locale.
normalizeTargetLanguages({String? targetLanguage, Iterable<String>? targetLanguages}) → List<String>: Normalizes targetLanguage and targetLanguages into a non-empty, de-duplicated language list (first occurrence wins).
paragraphExerciseMetadata({required String patternId, required List<Map<String, dynamic>> sentenceSpecs, required String joiner}) → Map<String, dynamic>
phraseExerciseMetadata({required String patternId, required String template, required Map<String, String> slots, required Map<String, String> semanticKeys}) → Map<String, dynamic>
phraseTemplateStoreCombinationCount({required PhraseTemplateLibrary library, required TextLexicon lexicon}) → int: Counts expanded combinations for a loaded phrase template store.
portugueseParagraphDocuments() → List<DatasetSourceDocument>: Source documents for Portuguese paragraph patterns.
portugueseParagraphPatternMap() → Map<String, ParagraphPatternConfig>: Map of Portuguese paragraph patterns by id.
portuguesePhraseDocuments() → List<DatasetSourceDocument>: Source documents for Portuguese phrase patterns.
portuguesePhrasePatternMap() → Map<String, PhrasePatternConfig>: Map of Portuguese phrase patterns by id.
rebuildExerciseText(Map<String, dynamic> metadata, {Map<String, String>? slotOverrides}) → String: Rebuilds phrase/paragraph text from exercise metadata and slot overrides.
resolveCurriculumVariationsPerEntry({required CurriculumVariationOptions? builderOptions, required int? stageVariationsPerEntry, required int? sourceVariationsPerEntry, required bool combinatorial}) → int?: Resolves the effective variations-per-entry count for a source.
textLexiconForLanguage(String language) → TextLexicon?: Returns the built-in lexicon for language, or null if unsupported.
wordCategoryBankForLanguage(String language) → WordCategoryBank?: Returns the built-in category bank for language, or null if unsupported.

Typedefs

DatasetQueryExecutor = Future<List<DatasetEntry>> Function(DatasetQuerySpec spec): Function that executes a DatasetQuerySpec against a store backend.
DatasetQueryStreamExecutor = Stream<DatasetEntry> Function(DatasetQuerySpec spec): Streams DatasetQuerySpec results without loading the full match set.
TranslateTextFn = Future<String> Function(String text, {required String sourceLanguage, required String targetLanguage}): Translates text from a source language into a targetLanguage.

Exceptions / Errors

CurriculumException: Thrown when curriculum manifest or build configuration is invalid.
DatasetVersionExistsException: Thrown when a pipeline refuses to overwrite an existing dataset version.
DuplicateEntryException: Thrown when inserting an entry whose id already exists in the store.
PhraseTemplateStoreException: Error while loading or validating a phrase template file store.

llm_dataset library

Classes

Enums

Mixins

Extensions

Constants

Properties

Functions

Typedefs

Exceptions / Errors

llm_dataset package

llm_dataset library