sorisdk_flutter

Add SORI-powered audio recognition to Flutter apps, so audiences can interact with TV, radio, cinema, and other media from their phones. When registered media is recognized, your app can show the matching campaign, product information, coupon, event, or action link managed in SORI Console.

This package wraps the native SORI Android and iOS SDKs and adapts @sorisdk/web-audio for Flutter Web with one Flutter API for starting recognition, receiving campaign/result events, updating the recognition database where supported, and reporting handled action URLs.

When to use it

Use this plugin when your Flutter app needs a second-screen experience for:

  • interactive ads and commerce links
  • coupons, events, quizzes, or promotions tied to broadcast content
  • product, cast, location, or soundtrack information while users watch or listen
  • exposure and click reporting for media campaigns

Learn more in the SORI API docs, Use Cases, and SORI Console overview.

Setup

Before integrating the Flutter package, prepare SORI resources in SORI Console:

  1. Create an app_id and secret_key in Managing Application.
  2. Register the media that should be recognized in Managing Material.
  3. Link a campaign, image, and action URL in Managing Campaign.

Android and iOS

Create a recognizer with your SORI mobile application credentials, subscribe to events, and start recognition.

final recognizer = SORIAudioRecognizer(
  applicationId: 'your-application-id',
  secretKey: 'your-secret-key',
);

await recognizer.configure(
  config: const SORIRecognitionConfig(audiomarker: true),
);

recognizer.events.listen((event) {
  final campaign = event.campaign;
  if (campaign != null) {
    // Render campaign.name, campaign.imageUrl, and campaign.actionUrl.
  }
  final marker = event.audioMarker;
  if (marker != null) {
    // Optionally use the near-ultrasonic audiomarker code.
  }
});

await recognizer.startRecognition(
  notification: const SORIAndroidNotificationOptions(
    title: 'Listening',
    body: 'SORI audio recognition is active',
  ),
);

Web

Browser builds must not contain the long-lived SORI secretKey. Install the companion npm package in the Flutter application and prepare the browser runtime before each Web build:

npm install --save-dev @sorisdk/flutter-web-bridge
npx sori-flutter-web prepare
flutter build web

The preparation command resolves the pinned @sorisdk/web-audio version and bundles its ESM, WASM, and model files into the application's web/sorisdk/ directory. Flutter copies those application-owned files to build/web; the JS/WASM/model artifacts are not embedded in this pub.dev package.

Create the Web recognizer with an endpoint on your application server:

final recognizer = SORIAudioRecognizer.web(
  applicationId: 'your-application-id',
  webAuth: SORIWebAuthOptions.ephemeralKeyEndpoint(
    Uri.parse('/api/sori/ephemeral-key'),
    // Use `include` only when the endpoint relies on cross-origin cookies.
    requestCredentials: SORIWebRequestCredentials.sameOrigin,
  ),
);

await recognizer.configure();
recognizer.events.listen((event) {
  // ready, lifecycle, match, audiomarker, campaign, and error events use the
  // same SORIRecognitionEvent stream as the mobile implementations.
});

// Call this from a user action. Browser microphone permission is requested
// here, not while the recognizer is constructed or configured.
await recognizer.startRecognition();

The endpoint keeps both SORI_APP_ID and SORI_SECRET_KEY in the server environment, exchanges them at https://console.soriapi.com/api/auth/ephemeral, and returns only the ephemeral value to the browser. For example, the essential server-side flow is:

app.post("/api/sori/ephemeral-key", async (_request, response) => {
  const exchange = await fetch(
    "https://console.soriapi.com/api/auth/ephemeral",
    {
      method: "POST",
      body: new URLSearchParams({
        app_id: process.env.SORI_APP_ID,
        secret_key: process.env.SORI_SECRET_KEY,
      }),
    },
  );
  if (!exchange.ok) {
    response.sendStatus(exchange.status);
    return;
  }

  const payload = await exchange.json();
  response.json({
    ephemeralKey: payload.ephemeral_key ?? payload.ephemeralKey,
  });
});

Protect this application endpoint with the same session, origin, CSRF, and rate limits used for other authenticated application APIs. The plugin transparently requests the ephemeral key when startRecognition() runs, passes it to the SORI auth flow, loads the authorized audiopack, attaches the activity token, and then starts microphone recognition. It never receives the long-lived secret.

When a user taps a campaign action URL that your app has decided to open, report that interaction back to SORI:

final actionUrl = campaign.actionUrl;
if (actionUrl != null && actionUrl.isNotEmpty) {
  await recognizer.handleActionUrl(actionUrl);
}

App Requirements

Android permissions and the foreground microphone service declaration are merged from the plugin. On Android, startRecognition() requests runtime microphone permission when needed and continues the same start request after permission is granted. On Android 13 and later, request notification permission if your app targets that API level and you want the foreground-service notification to be shown normally.

iOS apps must add NSMicrophoneUsageDescription to Info.plist. Add UIBackgroundModes with audio only if the app is designed to continue recognition while backgrounded.

Flutter Web requires a secure context (https:// or http://localhost), a modern browser with Web Audio and getUserMedia(), the prepared companion bundle, and an application-server ephemeral-key endpoint. updateDatabase() currently returns an unsupported result on Web because @sorisdk/web-audio does not expose a refresh operation that avoids starting microphone capture.

Do not ship SORI credentials in sample or public source. Create and rotate app keys in SORI Console. Mobile apps load them through the normal release configuration process; Web apps must keep the secret on the server and must not pass SORI_SECRET_KEY to flutter build web.

Releasing

Maintainers should publish this package by creating a GitHub Release with a new vX.Y.Z tag, such as v0.2.3, targeting the release commit on main. GitHub creates the tag for the Release, and that tag push triggers the pub.dev publish workflow. The workflow intentionally does not use the release event because pub.dev automated publishing with GitHub OIDC rejects tokens from that event. Ensure pub.dev automated publishing is configured for this repository with the matching v{{version}} tag pattern; permission to create a matching release tag is effectively permission to publish the package.

If pubspec.yaml includes build metadata, such as 0.2.3+4, use the base semantic version for the release tag (v0.2.3). The workflow publishes from a temporary workspace with the build metadata removed. Creating a Release from an already existing tag does not create a new tag push, so it will not start a new publish run; in that case, push the missing tag or rerun the existing tag workflow instead.

The companion npm package has independent SemVer and uses tags such as web-bridge-v0.1.0. A matching tag runs .github/workflows/publish-web-bridge.yml; configure npm trusted publishing for this repository and workflow before creating the first tag. Update the exact @sorisdk/web-audio dependency and the bridge version together whenever the browser runtime is upgraded.

Notes

The plugin exposes recognition lifecycle, campaign/result/error events, database updates, and an explicit handleActionUrl() method for URLs that your app has inspected. By default recognition is fingerprint-only; set SORIRecognitionConfig(audiomarker: true) to run near-ultrasonic audiomarker recognition alongside fingerprint recognition. Audiomarker results are exposed as event.audioMarker and, when attached to a campaign, as event.campaign?.trait?.marker. Marker-only state changes are emitted as SORIRecognitionEventType.audioMarkerChanged; a null marker means the native recognizer cleared its current marker state. Location, metadata provider, raw audio buffers, and automatic action URL opening are not part of the public Flutter API.

Activity identity and marker refinement

Campaign and recognition events expose an optional event.activityId. When a platform reporter supports activity refinement, the initial campaign event and its marker-enriched result carry the same non-empty activity id. Applications can use that server identity to replace an existing result instead of counting the refinement as a second campaign. Older native artifacts and servers remain compatible and yield null; material id alone is not a safe replacement key.

The server refinement window is 30 seconds. A marker already available on the first recognition is included in its create request. A marker that enriches the same recognition within the window is sent by a supporting platform reporter as an update to the original activity; a later independent recognition creates a new activity. audioMarkerChanged remains a marker-state notification and is not itself proof that a server activity was refined.

Transport support is release-gated. The bundled Android and iOS artifacts and the pinned @sorisdk/web-audio package must each implement the Console activity contract before a Flutter release can claim cross-platform refinement. The Web reporter work is tracked in iplateia/nada#144. The example already uses activityId additively when available while preserving the legacy append behavior when it is absent.

The bundled Android 5.0.5 callback exposes the exact server activity id through the compatible SORICampaign model. Flutter maps that value directly and never infers identity from a campaign or material id.

See the example/ app for a complete recognition flow that reads mobile credentials or a Web ephemeral endpoint from --dart-define values and renders recognized campaigns.

Libraries

sorisdk_flutter
sorisdk_flutter_method_channel
sorisdk_flutter_platform_interface