typed_llm 0.1.0

Type-safe, validated, structured output from LLM providers (OpenAI, Gemini, Claude, and OpenAI-compatible servers) via generated JSON Schemas. No dart:mirrors, no runtime reflection, pure Dart.

typed_llm

Type-safe, validated, structured output from LLM providers — no dart:mirrors, no runtime reflection, pure Dart.

@LlmSchema()
class Invoice {
  Invoice({required this.vendorName, required this.totalAmount, required this.dueDate});
  factory Invoice.fromValidatedJson(Map<String, dynamic> json) =>
      _$InvoiceFromValidatedJson(json);

  final String vendorName;
  final double totalAmount;
  final DateTime dueDate;
}

final extractor = Extractor(provider: OpenAiProvider(apiKey: apiKey, model: 'gpt-4o-2024-08-06'));
final invoice = await extractor.extract<Invoice>(
  prompt: 'Extract the invoice from this text: ...',
  schema: InvoiceSchema,
  fromJson: Invoice.fromValidatedJson,
);

One call. Returns a parsed, validated Invoice — or throws a typed TypedLlmException, never a raw FormatException or a silently-wrong object.

Naming note: InvoiceSchema and _$InvoiceFromValidatedJson are generated by typed_llm_generator (see Setup below) — they don't exist until you run build_runner.

Why

LLM "JSON mode" gets you a string that is usually JSON shaped like what you asked for. It is not validated, not typed, and providers disagree on how to even ask for it. typed_llm turns an annotated Dart class into a JSON Schema at build time, sends it to the provider's native structured-output mode, validates the response against that same schema, retries once with the validation errors fed back to the model, and hands you back a real, constructed Invoice — or a specific exception telling you exactly what went wrong.

Setup

dependencies:
  typed_llm: ^0.1.0

dev_dependencies:
  build_runner: ^2.4.0
  typed_llm_generator: ^0.1.0

Annotate your class, add the part directive and a fromValidatedJson wrapper (the wrapper is one line you write — the generator can't add a public member to your class for you, only emit a top-level function it calls):

import 'package:typed_llm/typed_llm.dart';

part 'invoice.g.dart';

@LlmSchema()
class Invoice {
  Invoice({
    @LlmField(description: 'The legal name of the vendor issuing the invoice')
    required this.vendorName,
    required this.totalAmount,
    required this.dueDate,
  });

  factory Invoice.fromValidatedJson(Map<String, dynamic> json) =>
      _$InvoiceFromValidatedJson(json);

  final String vendorName;
  final double totalAmount;
  final DateTime dueDate;
}

Then generate:

dart run build_runner build --delete-conflicting-outputs

@LlmField(description:) descriptions are optional but materially improve extraction accuracy — the model sees them in the schema you send it.

The generator also supports enums, List<T>, nested @LlmSchema classes (inlined recursively into both the schema and the parser), and freezed classes — see packages/example in this repo for a complete, runnable demonstration of all of these, including a freezed class.

Providers

Provider	Structured-output mechanism	Guarantee
OpenAiProvider	response_format: {type: "json_schema", strict: true}	Schema-enforced by OpenAI (strict mode)
GeminiProvider	generationConfig.responseMimeType + responseSchema	Schema-enforced by Gemini
ClaudeProvider	Forced tool use (tool_choice: {type: "tool", name: ...})	Schema-enforced via the tool's input_schema
OpenAiCompatibleProvider	Either of the above, or a JSON-mode + schema-in-system-prompt fallback	Depends on the server; weaker in fallback mode

All four implement the same LlmProvider interface, so Extractor doesn't care which one you use. Every provider that speaks OpenAI's Chat Completions shape — Ollama, Groq, vLLM, LM Studio, and similar — should work through OpenAiCompatibleProvider; set supportsStrictSchema based on what your server actually supports, since this package does not try to detect it by sniffing errors.

Error handling

Every failure mode is a distinct, typed exception under the sealed TypedLlmException hierarchy:

try {
  final invoice = await extractor.extract<Invoice>(...);
} on TypedLlmException catch (e) {
  switch (e) {
    case SchemaValidationException():
      // valid JSON, but it didn't match the schema — e.errors has the detail
    case MalformedJsonException():
      // the model's final attempt still wasn't valid JSON
    case ProviderException():
      // the HTTP call failed — e.retryable tells you if it already retried
    case ExtractionTimeoutException():
      // the provider didn't respond within ExtractorConfig.timeout
  }
}

On malformed JSON or a schema violation, Extractor retries once by default (ExtractorConfig.maxValidationRetries), appending the validation errors to the prompt. On a retryable HTTP failure (429/5xx), it retries up to ExtractorConfig.maxHttpAttempts times with exponential backoff, honoring the provider's Retry-After header when present.

API keys

Do not ship a provider API key inside a client app (mobile, web, or desktop). Anyone can extract it from the compiled binary or from network traffic, and a leaked key is billed to you.

// Insecure — do not do this in a shipped app:
final extractor = Extractor(provider: OpenAiProvider(apiKey: 'sk-...', model: '...'));

Instead, run a thin backend proxy that holds the key server-side, forwards the prompt/schema, and returns the provider's response. Point a provider's baseUrl at your proxy (all four providers accept one) so the request shape stays identical:

final extractor = Extractor(
  provider: OpenAiProvider(
    apiKey: 'unused-or-your-proxy-auth-token',
    model: 'gpt-4o-2024-08-06',
    baseUrl: 'https://your-backend.example.com/openai-proxy/v1',
  ),
);

Direct-from-client calls (with a key you control, e.g. in a server-side Dart process, a CLI tool, or during local development) are fine — the constraint is specifically about keys embedded in something end users can inspect.

How it compares

vs. hand-rolled jsonDecode + casts — that gives you no schema sent to the model (so it has to guess your shape from the prompt alone), no validation before you touch the data, and a TypeError at the first bad cast instead of a typed exception with the specific field and reason. This package's entire job is doing that work once, correctly, with retries.

vs. langchain_dart — langchain_dart is a broad agent/orchestration framework (chains, memory, vector stores, tool-calling agents) with structured output as one feature among many. typed_llm does one thing: turn an annotated class into a validated, typed extraction call. Reach for langchain_dart if you're building an agent; reach for typed_llm if you just need String → T.

Testing your own code against this package

Implement LlmProvider directly for a hand-rolled fake, or construct any of the built-in providers with an injected http.Client (each constructor accepts httpClient:) — pass a package:http/testing.dart MockClient to assert on request shape or script canned responses, exactly as this package's own test suite does.

Contributing

See CONTRIBUTING.md in the repository root — this package is developed as part of a monorepo alongside typed_llm_generator.

License

MIT — see LICENSE.