Foundation Models Framework

⚠️ BETA STATUS: This package is in beta. Core functionality including streaming is stable, but structured generation and tool calling features are still in development.

A Flutter package for integrating with Apple's Foundation Models framework on iOS and macOS devices. This package provides access to on-device AI capabilities through language model sessions, leveraging Apple Intelligence features.

Features

✅ Cross-Platform Support: Works on both iOS 26.0+ and macOS 15.0+
✅ Persistent Sessions: Maintain conversation context across multiple interactions
✅ Streaming Responses: Real-time token streaming with delta updates
✅ Generation Options: Control temperature, token limits, and sampling strategies
✅ Transcript History: Access full conversation history with role-based entries
✅ Guardrail Levels: Configure content safety levels (strict/standard/permissive)
✅ Rich Responses: Get detailed responses with raw content and transcript metadata
✅ Security Features: Built-in prompt validation and injection protection
✅ Type-safe API: Built with Pigeon for reliable platform communication
✅ Privacy-First: All processing happens on-device with Apple Intelligence

Requirements

iOS: 26.0 or later
macOS: 15.0 or later
Flutter: 3.0.0 or later
Dart: 3.8.1 or later
Xcode: 16.0 or later
Apple Intelligence: Must be enabled on device

Installation

Add this package to your pubspec.yaml:

dependencies:
  foundation_models_framework: ^0.2.0

Then run:

flutter pub get

Apple Intelligence Requirement on macOS

To use this package on macOS, you must enable Apple Intelligence in your system settings. Go to System Settings → Apple Intelligence & Siri, enable Apple Intelligence, and allow the system to finish downloading the required on-device models. Without this step, the Foundation Models framework will not be available and the package will return an availability error.

iOS Setup

1. Update iOS Deployment Target

In your ios/Podfile, ensure the iOS deployment target is set to 26.0 or higher:

platform :ios, '26.0'

2. Update iOS Project Settings

In your ios/Runner.xcodeproj, set:

iOS Deployment Target: 26.0
Swift Language Version: 5.0

Usage

Physical Device (Recommended):

Full Foundation Models functionality
Real Apple Intelligence features
Requires iOS 26.0+ device

Checking Availability

Before using Foundation Models features, check if they're available on the device:

import 'package:foundation_models_framework/foundation_models_framework.dart';

final foundationModels = FoundationModelsFramework.instance;

try {
  final availability = await foundationModels.checkAvailability();
  
  if (availability.isAvailable) {
    print('Foundation Models is available on iOS ${availability.osVersion}');
    // Proceed with AI operations
  } else {
    print('Foundation Models not available: ${availability.errorMessage}');
  }
} catch (e) {
  print('Error checking availability: $e');
}

Creating a Language Model Session

Create a session to interact with Apple's Foundation Models:

// Create a basic session
final session = foundationModels.createSession();

// Create a session with custom instructions and guardrails
final customSession = foundationModels.createSession(
  instructions: 'You are a helpful assistant. Keep responses concise.',
  guardrailLevel: GuardrailLevel.standard,
);

// Send a prompt and get a response
try {
  final response = await session.respond(prompt: 'Hello, how are you?');

  if (response.errorMessage == null) {
    print('Response: ${response.content}');

    // Access transcript history
    if (response.transcriptEntries != null) {
      for (final entry in response.transcriptEntries!) {
        print('${entry?.role}: ${entry?.content}');
      }
    }
  } else {
    print('Error: ${response.errorMessage}');
  }
} catch (e) {
  print('Failed to get response: $e');
}

// Don't forget to dispose when done
await session.dispose();

Using Generation Options

Control the model's generation behavior:

final session = foundationModels.createSession();

// Configure generation options
final options = GenerationOptionsRequest(
  temperature: 0.7,  // 0.0 = deterministic, 1.0 = creative
  maximumResponseTokens: 500,
  samplingTopK: 40,  // Top-K sampling
);

final response = await session.respond(
  prompt: 'Write a short story',
  options: options,
);

print('Generated: ${response.content}');

Convenience Method for Single Prompts

For single interactions, you can use the convenience method:

try {
  final response = await foundationModels.sendPrompt(
    'What is the weather like today?',
    instructions: 'Be brief and factual',
    guardrailLevel: GuardrailLevel.strict,
    options: GenerationOptionsRequest(temperature: 0.3),
  );

  if (response.errorMessage == null) {
    print('Response: ${response.content}');
  } else {
    print('Error: ${response.errorMessage}');
  }
} catch (e) {
  print('Failed to send prompt: $e');
}

Session-Based Conversation

For multi-turn conversations, reuse the same session:

final session = foundationModels.createSession();

// First interaction
var response = await session.respond(prompt: 'Tell me about Swift programming.');
print('AI: ${response.content}');

// Continue the conversation
response = await session.respond(prompt: 'Can you give me an example?');
print('AI: ${response.content}');

// Ask follow-up questions
response = await session.respond(prompt: 'How does that compare to Dart?');
print('AI: ${response.content}');

// Don't forget to dispose when done
await session.dispose();

Streaming Responses

For real-time token streaming:

final session = foundationModels.createSession(
  instructions: 'You are a helpful assistant',
  guardrailLevel: GuardrailLevel.standard,
);

// Stream tokens as they're generated
final stream = session.streamResponse(
  prompt: 'Write a detailed explanation of quantum computing',
  options: GenerationOptionsRequest(
    temperature: 0.7,
    maximumResponseTokens: 1000,
  ),
);

// Process tokens as they arrive
await for (final chunk in stream) {
  if (chunk.delta != null) {
    print('New tokens: ${chunk.delta}');
  }

  if (chunk.isFinal) {
    print('Complete response: ${chunk.cumulative}');
  }

  if (chunk.hasError) {
    print('Error: ${chunk.errorMessage}');
    break;
  }
}

Handling Stream Cancellation

You can cancel a stream at any time:

final stream = session.streamResponse(prompt: 'Long text generation...');

final subscription = stream.listen((chunk) {
  print('Received: ${chunk.delta}');

  // Cancel after receiving some content
  if (chunk.cumulative?.length ?? 0 > 100) {
    subscription.cancel(); // This will stop the stream
  }
});

API Reference

FoundationModelsFramework

The main class for accessing Foundation Models functionality.

Methods

`checkAvailability()`

Returns: Future<AvailabilityResponse>
Description: Checks if Foundation Models is available on the device
Note: Returns true only if iOS 26.0+/macOS 15.0+ and Apple Intelligence is available

`createSession({String? instructions, GuardrailLevel? guardrailLevel})`

Parameters:
- instructions: Optional system instructions for the session
- guardrailLevel: Content safety level (strict/standard/permissive)
Returns: LanguageModelSession
Description: Creates a new language model session with optional configuration

`sendPrompt(String prompt, {String? instructions, GuardrailLevel? guardrailLevel, GenerationOptionsRequest? options})`

Parameters:
- prompt: The text prompt to send
- instructions: Optional system instructions
- guardrailLevel: Optional content safety level
- options: Optional generation configuration
Returns: Future<ChatResponse>
Description: Convenience method to send a single prompt without managing a session

LanguageModelSession

A persistent session for interacting with Apple's Foundation Models.

Methods

`respond({required String prompt, GenerationOptionsRequest? options})`

Parameters:
- prompt: The text prompt to send to the model
- options: Optional generation configuration
Returns: Future<ChatResponse>
Description: Sends a prompt to the language model and returns the response

`prewarm()`

Returns: Future<void>
Description: Pre-warms the session to reduce first-token latency

`dispose()`

Returns: Future<void>
Description: Disposes of the session and releases resources

`streamResponse({required String prompt, GenerationOptionsRequest? options})`

Parameters:
- prompt: The text prompt to send to the model
- options: Optional generation configuration
Returns: Stream<StreamChunk>
Description: Streams response tokens in real-time as they are generated

Data Classes

`AvailabilityResponse`

bool isAvailable: Whether Foundation Models is available
String osVersion: The OS version
String? reasonCode: Structured reason code if unavailable
String? errorMessage: Human-readable error message

`ChatResponse`

String content: The response content from the model
String? rawContent: Raw response data
List<TranscriptEntry?>? transcriptEntries: Conversation history
String? errorMessage: Error message if the request failed

`TranscriptEntry`

String id: Unique identifier for the entry
String role: Role (user/assistant/instructions/etc.)
String content: The text content
List<String>? segments: Individual text segments

`GenerationOptionsRequest`

double? temperature: Controls randomness (0.0-1.0)
int? maximumResponseTokens: Maximum tokens to generate
int? samplingTopK: Top-K sampling parameter
double? samplingProbabilityThreshold: Probability threshold for sampling

`GuardrailLevel`

strict: Maximum content safety
standard: Balanced safety and flexibility
permissive: More permissive content transformations

`StreamChunk`

String streamId: Unique identifier for the stream
String? delta: New tokens in this chunk
String? cumulative: All tokens received so far
String? rawContent: Raw response data
bool isFinal: Whether this is the last chunk
String? errorCode: Error code if streaming failed
String? errorMessage: Error message if streaming failed
bool hasError: Convenience getter for error checking

Error Handling

The package handles errors gracefully and returns them in the response:

try {
  final response = await session.respond(prompt: 'Your prompt here');
  
  if (response.errorMessage != null) {
    // Handle specific errors
    switch (response.errorMessage) {
             case 'Foundation Models requires iOS 26.0 or later':
        print('Device not supported');
        break;
      case 'Foundation Models not available on this device':
        print('Apple Intelligence not available');
        break;
      default:
        print('Error: ${response.errorMessage}');
    }
  } else {
    print('Success: ${response.content}');
  }
} catch (e) {
  print('Unexpected error: $e');
}

Important Notes

Device Compatibility

Foundation Models requires iOS 26.0 or later
Only works on Apple Intelligence-enabled devices in supported regions

Privacy and Performance

All processing happens on-device using Apple's Foundation Models
No data is sent to external servers
Performance may vary based on device capabilities

Development Considerations

Always check availability before using features
Handle errors gracefully for better user experience
Consider providing fallback options for unsupported devices
Test on actual devices with Apple Intelligence enabled

Example App

The package includes a complete example app demonstrating:

Availability checking
Session creation and management
Prompt-response interactions
Error handling

Run the example:

cd example
flutter run

Contributing

Contributions are welcome! Submit pull requests for any improvements.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Changelog

See CHANGELOG.md for details about changes in each version.

Support

For issues and questions:

Create an issue on GitHub
Check the example app for usage patterns
Review Apple's Foundation Models documentation
Check Apple's iOS 26.0+ release notes for hardware compatibility

References

This package implementation is based on Apple's Foundation Models framework:

Apple Developer Documentation: Official API reference

Important: This package integrates with Apple's Foundation Models framework. Ensure you comply with Apple's terms of service and review their documentation for production use.

Foundation Models Framework

Features

Requirements

Installation

Apple Intelligence Requirement on macOS

iOS Setup

1. Update iOS Deployment Target

2. Update iOS Project Settings

Usage

Checking Availability

Creating a Language Model Session

Using Generation Options

Convenience Method for Single Prompts

Session-Based Conversation

Streaming Responses

Handling Stream Cancellation

API Reference

FoundationModelsFramework

Methods

checkAvailability()

createSession({String? instructions, GuardrailLevel? guardrailLevel})

sendPrompt(String prompt, {String? instructions, GuardrailLevel? guardrailLevel, GenerationOptionsRequest? options})

LanguageModelSession

Methods

respond({required String prompt, GenerationOptionsRequest? options})

prewarm()

dispose()

streamResponse({required String prompt, GenerationOptionsRequest? options})

Data Classes

AvailabilityResponse

ChatResponse

TranscriptEntry

GenerationOptionsRequest

GuardrailLevel

StreamChunk

Error Handling

Important Notes

Device Compatibility

Privacy and Performance

Development Considerations

Example App

Contributing

License

Changelog

Support

References

Libraries

foundation_models_framework package

`checkAvailability()`

`createSession({String? instructions, GuardrailLevel? guardrailLevel})`

`sendPrompt(String prompt, {String? instructions, GuardrailLevel? guardrailLevel, GenerationOptionsRequest? options})`

`respond({required String prompt, GenerationOptionsRequest? options})`

`prewarm()`

`dispose()`

`streamResponse({required String prompt, GenerationOptionsRequest? options})`

`AvailabilityResponse`

`ChatResponse`

`TranscriptEntry`

`GenerationOptionsRequest`

`GuardrailLevel`

`StreamChunk`