streamdeck_flutter 0.2.0

Build Elgato Stream Deck plugins with Flutter. Renders UI offscreen, slices frames into key-sized tiles, and pushes them via the Stream Deck WebSocket protocol.

streamdeck_flutter

A Flutter framework for building Elgato Stream Deck plugins. Renders Flutter UI offscreen, slices frames into key-sized tiles and touchscreen strips, and pushes them as base64 PNG via the Stream Deck WebSocket protocol. Input events from the Stream Deck are injected back into Flutter's gesture system.

Target device: Stream Deck + (4x2 keys at 144x144px, 4 encoder slots with 200x100px touchscreen strips).

Requirements

macOS App Sandbox must be disabled. The plugin connects to the Stream Deck app via a local WebSocket, which is blocked by the default sandbox. Disable it in your macos/Runner/DebugProfile.entitlements and macos/Runner/Release.entitlements:

<key>com.apple.security.app-sandbox</key>
<false/>

Or, if you prefer to keep the sandbox enabled, add the network client entitlement:

<key>com.apple.security.network.client</key>
<true/>

Quick Start

1. Add the dependency

dependencies:
  streamdeck_flutter: ^0.1.0

2. Create your plugin

import 'package:flutter/material.dart';
import 'package:streamdeck_flutter/streamdeck_flutter.dart';

const _binary = 'my_plugin';

void main(List<String> args) {
  HeadlessFlutterBinding.ensureInitialized();
  runApp(MyPlugin(setup: PluginSetup.fromArgs(args)));
}

class MyPlugin extends StatelessWidget {
  const MyPlugin({super.key, required this.setup});

  final PluginSetup setup;

  @override
  Widget build(BuildContext context) {
    return StreamDeckPlugin(
      setup: setup,
      manifest: PluginManifest(
        uuid: 'com.example.my-plugin',
        name: 'My Plugin',
        author: 'Your Name',
        description: 'A Stream Deck plugin built with Flutter.',
        icon: 'assets/imgs/plugin-icon',
        version: '1.0.0.0',
        codePath: '$_binary.app/Contents/MacOS/$_binary',
        actions: [
          ManifestAction(
            id: 'counter',
            name: 'Counter',
            tooltip: 'A simple tap counter.',
            icon: 'assets/imgs/action-icon',
            states: const [ManifestState(image: 'assets/imgs/action-icon')],
            builder: (context, action) => CounterAction(action: action),
          ),
        ],
        os: const [ManifestOS(platform: 'mac', minimumVersion: '10.15')],
        software: const ManifestSoftware(minimumVersion: '6.9'),
      ),
    );
  }
}

3. Create an action widget

class CounterAction extends StatefulWidget {
  const CounterAction({super.key, required this.action});

  final ActionInfo action;

  @override
  State<CounterAction> createState() => _CounterActionState();
}

class _CounterActionState extends State<CounterAction> {
  int _count = 0;

  @override
  Widget build(BuildContext context) {
    return StreamDeckAction(
      onKeyDown: () => setState(() => _count++),
      onDialRotate: (ticks) => setState(() => _count += ticks),
      onDialDown: () => setState(() => _count = 0),
      keypadChild: Container(
        color: Colors.black,
        child: Center(
          child: Text(
            '$_count',
            style: const TextStyle(
              color: Colors.white,
              fontSize: 32,
            ),
          ),
        ),
      ),
      encoderChild: Container(
        color: Colors.black,
        child: Center(
          child: Text('$_count taps',
            style: const TextStyle(color: Colors.white, fontSize: 20),
          ),
        ),
      ),
    );
  }
}

4. Build and install

flutter pub run streamdeck_flutter:build

Builds the app, generates manifest.json, bundles the plugin, links it to the Stream Deck app, and restarts. DevTools URL is printed automatically (profile mode by default).

Action Types

Keypad + Encoder (default)

ManifestAction
(
id: 'my-action',
name: 'My Action',
states: const [ManifestState(image: 'assets/imgs/icon')],
encoder: const ManifestEncoder(
triggerDescription: ManifestTriggerDescription(
rotate: 'Adjust value',
push: 'Confirm',
touch: 'Toggle',
),
),
builder: (ctx, action) => MyActionWidget(action: action),
)

Keypad only

ManifestAction.keypad
(
id: 'key-action',
name: 'Key Action',
states: const [ManifestState(image: 'assets/imgs/icon')],
builder: (ctx, action) => MyKeyWidget(action: action),
)

Encoder only

ManifestAction.encoder
(
id: 'dial-action',
name: 'Dial Action',
states: const [ManifestState(image: 'assets/imgs/icon')],
builder: (ctx, action) => MyDialWidget(action: action),
)

Multi-state actions

Each state can have its own builder. The SD app toggles states on key press and the widget switches automatically:

ManifestAction.keypad
(
id: 'toggle',
name: 'Toggle',
states: [
ManifestState(
image: 'assets/imgs/off',
name: 'Off',
builder: (ctx, action) => OffWidget(),
),
ManifestState(
image: 'assets/imgs/on',
name: 'On',
builder: (ctx, action) => OnWidget(),
),
]
,
)

You can also use action.state in a single builder:

ManifestAction.keypad
(
id: 'toggle',
name: 'Toggle',
states: [
ManifestState(image: 'assets/imgs/off', name: 'Off'),
ManifestState(image: 'assets/imgs/on', name: 'On'),
],
builder: (ctx, action) {
if (action.state == 0) return OffWidget();
return OnWidget();
},
)

StreamDeckAction Widget

Wrap your action UI in StreamDeckAction to handle SD events:

StreamDeckAction
(
onKeyDown: () { /* key pressed */ },
onKeyUp: () { /* key released */ },
onDialRotate: (ticks) { /* dial rotated */ },
onDialDown: () { /* dial pressed */ },
onDialUp: () { /* dial released */ },
onTouchTap: (position) { /* touchscreen tapped */ },
onSettings: (json) { /* settings changed */ },
keypadChild: MyKeyWidget(), // rendered on keys
encoderChild: MyDialWidget(
)
, // rendered on touchscreen strip
)

Settings

Access action settings with typed helpers:

final color = action.setting<String>('color') ?? '#FFFFFF';
final speed = action.setting<double>('speed', 1.0);

Persist settings:

StreamDeck.save(context, {'color': '#FF0000', 'speed': 2.0});

Show an alert on the key:

StreamDeck.alert(context);

Connection State

Monitor the WebSocket connection in your action widgets:

ValueListenableBuilder<SdConnectionState>(
  valueListenable: StreamDeck.clientOf(context).connectionState,
  builder: (context, state, child) {
    return switch (state) {
      SdConnectionState.connected => child!,
SdConnectionState.reconnecting => const Text('Reconnecting...'),
SdConnectionState.failed => const Text('Disconnected'),
_ => const CircularProgressIndicator(),
};
},
child
:
MyActionContent
(
)
,
)

Set State

Programmatically change the action state:

final client = StreamDeck.clientOf(context);
final ctx = StreamDeck.actionOf(context)!.context;
client.setState(ctx, 1);

CLI Commands

# Build, install, and start DevTools (profile mode)
flutter pub run streamdeck_flutter:build

# Build in release mode (no DevTools, smaller binary)
flutter pub run streamdeck_flutter:build -m release

# Build in debug mode
flutter pub run streamdeck_flutter:build -m debug

# Scaffold a new action
flutter pub run streamdeck_flutter:create my_action
flutter pub run streamdeck_flutter:create my_dial --encoder-only

# Generate README from manifest
flutter pub run streamdeck_flutter:docs

Testing

Action widgets get their ActionInfo from context via StreamDesk.actionOf(context), so they can be tested with standard Flutter widget tests using the StreamDeckScope and StreamDesk inherited widgets.

flutter test

## Example Actions

The `example/` directory includes several actions demonstrating different features:

| Action           | Type             | Features                                             |
|------------------|------------------|------------------------------------------------------|
| **Tapper**       | Keypad + Encoder | REST API polling, settings persistence, dose logging |
| **Animation**    | Keypad + Encoder | 8 animation styles, dial speed control, tap to cycle |
| **Dial Driven**  | Keypad + Encoder | Dial-scrubbed animation, auto-play toggle            |
| **Hello World**  | Keypad + Encoder | Simple tap counter, dial adjust, reset on press      |
| **Toggle**       | Keypad           | Multi-state ON/OFF with per-state builders           |
| **Color Picker** | Keypad + Encoder | Hue dial, clipboard copy, live color preview         |

## Project Structure

lib/ main.dart # Plugin entry point with manifest + actions widgets/ # Action widget implementations assets/ imgs/ # Plugin and action icons (SVG/PNG) layouts/ # Encoder touchscreen layouts (auto-provided) ui/ # Property inspector HTML files

## Architecture

Stream Deck App (WS Server) | WebSocket JSON Flutter macOS App (WS Client / Plugin) |-- StreamDeckPlugin -> top-level widget, manifest, action routing |-- StreamDeckClient -> connects, registers, sends setImage/setFeedback |-- RasterService -> captures UI, slices into tiles, change detection '-- StreamDeckAction -> maps SD events to Flutter callbacks

The app runs headless (hidden macOS window). Full frames are captured once, sliced per action context, and only changed
tiles are sent (hash comparison) to minimize WebSocket traffic. Always captures at 2.0x for Retina Stream Deck displays.

The `manifest.json` is generated automatically at build time from the `PluginManifest` defined in your Dart code. Action
UUIDs use short IDs (e.g. `'counter'`) that are auto-prefixed with the plugin UUID (e.g.
`com.example.my-plugin.counter`).

## Encoder Layout (canvas.json)

The framework renders encoder touchscreen strips as pixel data. This requires a `canvas.json` layout file that defines a
full-size pixmap region for the encoder strip. The build script automatically generates this file at
`{assets}/layouts/canvas.json` if it doesn't already exist:

```json
{
  "$schema": "https://schemas.elgato.com/streamdeck/plugins/layout.json",
  "id": "canvas-layout",
  "items": [
    {
      "key": "canvas",
      "type": "pixmap",
      "rect": [0, 0, 200, 100]
    }
  ]
}

The manifest.json encoder block is also generated automatically with the correct path to this file, derived from your plugin's assets directory. You do not need to call setFeedbackLayout at runtime — the Stream Deck app reads the layout from the manifest on action appear.

The build script always writes this file, overriding any existing canvas.json. Do not modify it — it is a generated file managed by the framework.