typed_llm

Type-safe, validated, structured output from LLM providers — no dart:mirrors, no runtime reflection, pure Dart.

@LlmSchema()
class Invoice {
  Invoice({required this.vendorName, required this.totalAmount, required this.dueDate});

  final String vendorName;
  final double totalAmount;
  final DateTime dueDate;
}

final extractor = Extractor(provider: OpenAiProvider(apiKey: apiKey, model: 'gpt-4o-2024-08-06'));
final invoice = await extractor.extract(
  $Invoice,
  prompt: 'Extract the invoice from this text: ...',
);

One call. Returns a parsed, validated Invoice — or throws a typed TypedLlmException, never a raw FormatException or a silently-wrong object.

Naming note: $Invoice is generated by typed_llm_generator (see Setup below) — it doesn't exist until you run build_runner. It bundles the JSON Schema with the factory that rebuilds Invoice, which is what lets extract infer the return type and makes it impossible to pair one class's schema with another's parser.

Why

LLM "JSON mode" gets you a string that is usually JSON shaped like what you asked for. It is not validated, not typed, and providers disagree on how to even ask for it. typed_llm turns an annotated Dart class into a JSON Schema at build time, sends it to the provider's native structured-output mode, validates the response against that same schema, retries once with the validation errors fed back to the model, and hands you back a real, constructed Invoice — or a specific exception telling you exactly what went wrong.

How it works

Two phases: build time (top) runs once via build_runner and turns an @LlmSchema class into a schema constant and a parser. Runtime (bottom) runs on every call — the provider's raw response is validated, an invalid result loops back once with the validation errors fed to the model, and a valid one becomes the typed object. If it's still invalid after the retry, you get a typed exception, never a silently-wrong object.

Setup

dependencies:
  typed_llm: ^0.2.0

dev_dependencies:
  build_runner: ^2.4.0
  typed_llm_generator: ^0.2.0

typed_llm itself runs on Dart 3.4+; typed_llm_generator needs Dart 3.9+, since it is a dev-time dependency built on the current analyzer.

Annotate your class and add the part directive — that's all you write:

import 'package:typed_llm/typed_llm.dart';

part 'invoice.g.dart';

@LlmSchema()
class Invoice {
  Invoice({
    @LlmField(description: 'The legal name of the vendor issuing the invoice')
    required this.vendorName,
    required this.totalAmount,
    required this.dueDate,
  });

  final String vendorName;
  final double totalAmount;
  final DateTime dueDate;
}

Then generate:

dart run build_runner build

This generates $Invoice (an LlmType<Invoice>, what you pass to extract), plus InvoiceSchema — the raw JSON Schema map, if you want to send or inspect it yourself.

@LlmField(description:) descriptions are optional but materially improve extraction accuracy — the model sees them in the schema you send it.

The generator also supports enums, List<T>, nested @LlmSchema classes (inlined recursively into both the schema and the parser), and freezed classes — see packages/example in this repo for a complete, runnable demonstration of all of these, including a freezed class.

Providers

Provider	Structured-output mechanism	Guarantee
`OpenAiProvider`	`response_format: {type: "json_schema", strict: true}`	Schema-enforced by OpenAI (strict mode)
`GeminiProvider`	`generationConfig.responseMimeType` + `responseSchema`	Schema-enforced by Gemini
`ClaudeProvider`	Forced tool use (`tool_choice: {type: "tool", name: ...}`)	Schema-enforced via the tool's `input_schema`
`OpenAiCompatibleProvider`	Either of the above, or a JSON-mode + schema-in-system-prompt fallback	Depends on the server; weaker in fallback mode

All four implement the same LlmProvider interface, so Extractor doesn't care which one you use. Every provider that speaks OpenAI's Chat Completions shape — Ollama, Groq, vLLM, LM Studio, and similar — should work through OpenAiCompatibleProvider; set supportsStrictSchema based on what your server actually supports, since this package does not try to detect it by sniffing errors.

Error handling

Every failure mode is a distinct, typed exception under the sealed TypedLlmException hierarchy:

try {
  final invoice = await extractor.extract<Invoice>(...);
} on TypedLlmException catch (e) {
  switch (e) {
    case SchemaValidationException():
      // valid JSON, but it didn't match the schema — e.errors has the detail
    case MalformedJsonException():
      // the model's final attempt still wasn't valid JSON
    case ProviderException():
      // the HTTP call failed — e.retryable tells you if it already retried
    case ExtractionTimeoutException():
      // the provider didn't respond within ExtractorConfig.timeout
  }
}

On malformed JSON or a schema violation, Extractor retries once by default (ExtractorConfig.maxValidationRetries), appending the validation errors to the prompt. On a retryable HTTP failure (429/5xx), it retries up to ExtractorConfig.maxHttpAttempts times with exponential backoff, honoring the provider's Retry-After header when present.

API keys

Do not ship a provider API key inside a client app (mobile, web, or desktop). Anyone can extract it from the compiled binary or from network traffic, and a leaked key is billed to you.

// Insecure — do not do this in a shipped app:
final extractor = Extractor(provider: OpenAiProvider(apiKey: 'sk-...', model: '...'));

Instead, run a thin backend proxy that holds the key server-side, forwards the prompt/schema, and returns the provider's response. Point a provider's baseUrl at your proxy (all four providers accept one) so the request shape stays identical:

final extractor = Extractor(
  provider: OpenAiProvider(
    apiKey: 'unused-or-your-proxy-auth-token',
    model: 'gpt-4o-2024-08-06',
    baseUrl: 'https://your-backend.example.com/openai-proxy/v1',
  ),
);

Direct-from-client calls (with a key you control, e.g. in a server-side Dart process, a CLI tool, or during local development) are fine — the constraint is specifically about keys embedded in something end users can inspect.

How it compares

vs. hand-rolled jsonDecode + casts — that gives you no schema sent to the model (so it has to guess your shape from the prompt alone), no validation before you touch the data, and a TypeError at the first bad cast instead of a typed exception with the specific field and reason. This package's entire job is doing that work once, correctly, with retries.

vs. langchain_dart — langchain_dart is a broad agent/orchestration framework (chains, memory, vector stores, tool-calling agents) with structured output as one feature among many. typed_llm does one thing: turn an annotated class into a validated, typed extraction call. Reach for langchain_dart if you're building an agent; reach for typed_llm if you just need String → T.

Testing your own code against this package

Implement LlmProvider directly for a hand-rolled fake, or construct any of the built-in providers with an injected http.Client (each constructor accepts httpClient:) — pass a package:http/testing.dart MockClient to assert on request shape or script canned responses, exactly as this package's own test suite does.

Contributing

See CONTRIBUTING.md in the repository root — this package is developed as part of a monorepo alongside typed_llm_generator.

License

MIT — see LICENSE.