ikon_sdk 1.12.0

Official Ikon AI client SDK for native Flutter frontends — connect to an Ikon server and render the server-driven Parallax UI with audio streaming.

Ikon SDK for Dart & Flutter

ikon_sdk is the official client SDK for building native Flutter frontends for Ikon AI apps. It connects to an Ikon server, streams the server-driven Parallax UI tree over the binary Ikon protocol, and renders it as native Flutter widgets — including the platform's own audio streaming, image capture and push-to-talk.

Install

flutter pub add ikon_sdk

Quickstart

Connect to a server, wrap the connection in an IkonUiCore, and render it with IkonParallaxView:

import 'package:flutter/material.dart';
import 'package:ikon_sdk/ikon_sdk.dart';

class IkonScreen extends StatefulWidget {
  const IkonScreen({super.key});

  @override
  State<IkonScreen> createState() => _IkonScreenState();
}

class _IkonScreenState extends State<IkonScreen> {
  IkonClient? _client;
  IkonUiCore? _uiCore;

  @override
  void initState() {
    super.initState();
    _connect();
  }

  Future<void> _connect() async {
    // Local dev server (auto-authenticates).
    final client = await IkonClient.connectLocal(host: 'localhost', port: 8443);
    setState(() {
      _client = client;
      _uiCore = IkonUiCore(client);
    });
  }

  @override
  void dispose() {
    _uiCore?.dispose();
    _client?.dispose();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    final uiCore = _uiCore;
    if (uiCore == null) {
      return const Center(child: CircularProgressIndicator());
    }
    return IkonPlatformOverlay(
      client: _client!,
      uiCore: uiCore,
      mediaManager: null,
      child: IkonParallaxView(uiCore: uiCore, client: _client),
    );
  }
}

IkonPlatformOverlay hosts the platform's own surfaces over the app: the feedback button (shown when the space enables feedback and admits the user), the server-rendered feedback sheet on the ikon.platform mount, and a client-drawn fallback form for when that sheet cannot arrive. It also answers ikon.client.captureView with the screenshot taken when the user opened feedback, once and within 30 seconds; the server cannot capture the screen on its own. Long-press the button, or call IkonPlatformOverlay.maybeOf(context)?.showPlatformMenu(), for the menu that turns the button on or off.

Connecting

IkonClient exposes three connection entry points:

• IkonClient.connectLocal(...) — local development server, no identity needed.
• IkonClient.connectGuest(...) — anonymous connection to a deployed space.
• IkonClient.connectWithToken(...) — authenticated connection with an OAuth / session token.

Subscribe to client.onStateChange (an IkonConnectionState stream) to drive your connecting / connected / error UI.

Highlights

• IkonClient — connection, the binary protocol, function registry, reactive state and global state.
• IkonParallaxView — renders the server-driven Parallax UI as Flutter widgets; supports named mounts via the mount parameter. Without one it renders the app's ikon-ui stream, or, when there is none, its only stream besides the platform overlay's.
• IkonPlatformOverlay — the feedback button, sheet and fallback form over the app, and the app-surface capture the sheet attaches.
• IkonMediaManager — the platform's own audio streaming (Opus playback and capture over the Ikon protocol), live WebRTC media, and the share and copy-to-clipboard actions.
• IkonWebRtcMedia — live WebRTC audio & video: the Dart client becomes a WebRTC peer to the server SFU, so live audio plays (and keeps playing in the background) and std.video-canvas renders live video — the same as the web client. The Opus playback/capture stack remains the automatic fallback.

Live WebRTC audio & video

Enable WebRTC once the client is connected, then hand it to the media manager:

final webRtcMedia = enableIkonWebRtcMedia(client);   // audio + video by default
await webRtcMedia.start();                            // negotiates with the server

final mediaManager = IkonMediaManager(
  audioPlayback: enableIkonAudioPlayback(client),     // Opus fallback (kept intact)
  audioCapture: enableIkonAudioCapture(client),
  webRtcMedia: webRtcMedia,                            // live A/V receive path
);

return IkonParallaxView(uiCore: uiCore, client: client, mediaManager: mediaManager);

Remote audio plays automatically through the native WebRTC audio engine, and any std.video-canvas node binds to its stream by streamId.

Required native configuration

flutter_webrtc needs platform setup in your app (a package can't edit the host app's manifests). Live audio also needs a background-audio capability so it keeps playing when the app is backgrounded.

iOS (ios/Runner/Info.plist), plus set the iOS deployment target to 14.0+:

<key>NSMicrophoneUsageDescription</key><string>Voice input</string>
<key>NSCameraUsageDescription</key><string>Camera capture</string>
<key>UIBackgroundModes</key>
<array>
  <string>audio</string>
  <string>voip</string>
</array>

Android (android/app/src/main/AndroidManifest.xml), with minSdk 24+:

<uses-permission android:name="android.permission.INTERNET"/>
<uses-permission android:name="android.permission.RECORD_AUDIO"/>
<uses-permission android:name="android.permission.CAMERA"/>
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS"/>
<!-- For background audio, add a foreground service (type mediaPlayback). -->
<uses-permission android:name="android.permission.FOREGROUND_SERVICE"/>
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MEDIA_PLAYBACK"/>

The mic/camera capture permissions are the same ones the Opus capture path already requires (via permission_handler), so an app that already streams microphone audio only needs to add the background-audio capability and the camera entries for video.

License

MIT — see LICENSE.