sorisdk_flutter 0.3.2

Add SORI-powered audio recognition, campaign discovery, and action-link handling to Flutter apps on Android, iOS, and Web.

sorisdk_flutter

Add SORI-powered audio recognition to Flutter apps, so audiences can interact with TV, radio, cinema, and other media from their phones. When registered media is recognized, your app can show the matching campaign, product information, coupon, event, or action link managed in SORI Console.

This package wraps the native SORI Android and iOS SDKs and adapts @sorisdk/web-audio for Flutter Web with one Flutter API for starting recognition, receiving campaign/result events, updating the recognition database where supported, and reporting handled action URLs.

When to use it

Use this plugin when your Flutter app needs a second-screen experience for:

• interactive ads and commerce links
• coupons, events, quizzes, or promotions tied to broadcast content
• product, cast, location, or soundtrack information while users watch or listen
• exposure and click reporting for media campaigns

Learn more in the SORI API docs, Use Cases, and SORI Console overview.

Setup

Before integrating the Flutter package, prepare SORI resources in SORI Console:

1. Create an app_id and secret_key in Managing Application.
2. Register the media that should be recognized in Managing Material.
3. Link a campaign, image, and action URL in Managing Campaign.

Android and iOS

Create a recognizer with your SORI mobile application credentials, subscribe to events, and start recognition.

final recognizer = SORIAudioRecognizer(
  applicationId: 'your-application-id',
  secretKey: 'your-secret-key',
);

await recognizer.configure(
  config: const SORIRecognitionConfig(audiomarker: true),
);

recognizer.events.listen((event) {
  final campaign = event.campaign;
  if (campaign != null) {
    // Render campaign.name, campaign.imageUrl, and campaign.actionUrl.
  }
  final marker = event.audioMarker;
  if (marker != null) {
    // Optionally use the near-ultrasonic audiomarker code.
  }
});

await recognizer.startRecognition(
  notification: const SORIAndroidNotificationOptions(
    title: 'Listening',
    body: 'SORI audio recognition is active',
  ),
);

Web

Browser builds must not contain the long-lived SORI secretKey. Install the companion npm package in the Flutter application and prepare the browser runtime before each Web build:

npm install --save-dev @sorisdk/flutter-web-bridge
npx sori-flutter-web prepare
flutter build web

The preparation command resolves the pinned @sorisdk/web-audio version and bundles its ESM, WASM, and model files into the application's web/sorisdk/ directory. Flutter copies those application-owned files to build/web; the JS/WASM/model artifacts are not embedded in this pub.dev package.

Create the Web recognizer with an endpoint on your application server:

final recognizer = SORIAudioRecognizer.web(
  applicationId: 'your-application-id',
  webAuth: SORIWebAuthOptions.ephemeralKeyEndpoint(
    Uri.parse('/api/sori/ephemeral-key'),
    // Use `include` only when the endpoint relies on cross-origin cookies.
    requestCredentials: SORIWebRequestCredentials.sameOrigin,
  ),
);

await recognizer.configure();
recognizer.events.listen((event) {
  // ready, lifecycle, match, audiomarker, campaign, and error events use the
  // same SORIRecognitionEvent stream as the mobile implementations.
});

// Call this from a user action. Browser microphone permission is requested
// here, not while the recognizer is constructed or configured.
await recognizer.startRecognition();

The endpoint keeps both SORI_APP_ID and SORI_SECRET_KEY in the server environment, exchanges them at https://console.soriapi.com/api/auth/ephemeral, and returns only the ephemeral value to the browser. For example, the essential server-side flow is:

app.post("/api/sori/ephemeral-key", async (_request, response) => {
  const exchange = await fetch(
    "https://console.soriapi.com/api/auth/ephemeral",
    {
      method: "POST",
      body: new URLSearchParams({
        app_id: process.env.SORI_APP_ID,
        secret_key: process.env.SORI_SECRET_KEY,
      }),
    },
  );
  if (!exchange.ok) {
    response.sendStatus(exchange.status);
    return;
  }

  const payload = await exchange.json();
  response.json({
    ephemeralKey: payload.ephemeral_key ?? payload.ephemeralKey,
  });
});

Protect this application endpoint with the same session, origin, CSRF, and rate limits used for other authenticated application APIs. The plugin transparently requests the ephemeral key when startRecognition() runs, passes it to the SORI auth flow, loads the authorized audiopack, attaches the activity token, and then starts microphone recognition. It never receives the long-lived secret.

When a user taps a campaign action URL that your app has decided to open, report that interaction back to SORI:

final actionUrl = campaign.actionUrl;
if (actionUrl != null && actionUrl.isNotEmpty) {
  await recognizer.handleActionUrl(actionUrl);
}

App Requirements

Android permissions and the foreground microphone service declaration are merged from the plugin. On Android, startRecognition() requests runtime microphone permission when needed and continues the same start request after permission is granted. On Android 13 and later, request notification permission if your app targets that API level and you want the foreground-service notification to be shown normally.

iOS apps must add NSMicrophoneUsageDescription to Info.plist. Add UIBackgroundModes with audio only if the app is designed to continue recognition while backgrounded.

Flutter Web requires a secure context (https:// or http://localhost), a modern browser with Web Audio and getUserMedia(), the prepared companion bundle, and an application-server ephemeral-key endpoint. updateDatabase() currently returns an unsupported result on Web because @sorisdk/web-audio does not expose a refresh operation that avoids starting microphone capture.

Do not ship SORI credentials in sample or public source. Create and rotate app keys in SORI Console. Mobile apps load them through the normal release configuration process; Web apps must keep the secret on the server and must not pass SORI_SECRET_KEY to flutter build web.

Releasing

Maintainers should publish this package by creating a GitHub Release with a new vX.Y.Z tag, such as v0.2.3, targeting the release commit on main. GitHub creates the tag for the Release, and that tag push triggers the pub.dev publish workflow. The workflow intentionally does not use the release event because pub.dev automated publishing with GitHub OIDC rejects tokens from that event. Ensure pub.dev automated publishing is configured for this repository with the matching v{{version}} tag pattern; permission to create a matching release tag is effectively permission to publish the package.

If pubspec.yaml includes build metadata, such as 0.2.3+4, use the base semantic version for the release tag (v0.2.3). The workflow publishes from a temporary workspace with the build metadata removed. Creating a Release from an already existing tag does not create a new tag push, so it will not start a new publish run; in that case, push the missing tag or rerun the existing tag workflow instead.

The companion npm package has independent SemVer and uses tags such as web-bridge-v0.1.0. A matching tag runs .github/workflows/publish-web-bridge.yml; configure npm trusted publishing for this repository and workflow before creating the first tag. Update the exact @sorisdk/web-audio dependency and the bridge version together whenever the browser runtime is upgraded.

Notes

The plugin exposes recognition lifecycle, campaign/result/error events, database updates, and an explicit handleActionUrl() method for URLs that your app has inspected. By default recognition is fingerprint-only; set SORIRecognitionConfig(audiomarker: true) to run near-ultrasonic audiomarker recognition alongside fingerprint recognition. Audiomarker results are exposed as event.audioMarker and, when attached to a campaign, as event.campaign?.trait?.marker. Marker-only state changes are emitted as SORIRecognitionEventType.audioMarkerChanged; a null marker means the native recognizer cleared its current marker state. Location, metadata provider, raw audio buffers, and automatic action URL opening are not part of the public Flutter API.

Activity identity and marker refinement

Campaign and recognition events expose an optional event.activityId. When a platform reporter supports activity refinement, the initial campaign event and its marker-enriched result carry the same non-empty activity id. Applications must use that value only as the exact server mutation identity; campaign and material ids are never substitutes for it.

The example keeps an explicit ordered current segment. An exact activity refinement can replace only that current row, confirmed marker evidence is sticky, and an intervening material seals the segment. Therefore A -> A' -> A (marker miss) remains one marker-bearing row, while A -> B -> A' cannot rewrite the older A row. Returning A with a new activity id creates a new row. Legacy events without an activity id remain parseable, but their material fallback is limited to the current segment. A stopped or destroyed recognizer also closes that boundary.

Supporting platform reporters apply the same boundary to server transport. A marker available on the first observation is included in the POST; marker evidence that arrives later may produce at most one PUT for that activity. Repeated observations can renew a separate local continuity lease, but the activity's recognized_at, position, and refinement deadline remain anchored to the first observation. audioMarkerChanged remains independently observable marker state and never creates or refines a campaign row by itself.

Flutter 0.3.2+2 uses these published inputs:

• Android SDK 5.0.7, release commit 7d61d346056e0eed65447e578a07645d41387378, published AAR SHA-256 27b173d9cdf7863127ee407e8618cd9177da96ab1dcac57bb567c6d2d9f1615f.
• iOS SDK 5.0.6, release commit e0c1ce465826a45f507da7ef01caef9928a40ba0, published XCFramework archive SHA-256 3a8ccd02ff13e4234a5e050af2460e915e564b58a442b26026bc407b1557ed51.
• @sorisdk/web-audio 0.6.7, release commit d57b454c2f7ab5602ef33761835830b828b2eff5, pinned by @sorisdk/flutter-web-bridge 0.1.2 and the root npm lockfile.

The Flutter bridges surface activity, material, and marker data from those reporters; they do not duplicate the native or Web HTTP lifecycle state machines. The upstream contract work is tracked in sorisdk-android#54, sorisdk-ios#99, and nada#149.

See the example/ app for a complete recognition flow that reads mobile credentials or a Web ephemeral endpoint from --dart-define values and renders recognized campaigns.