sorisdk_flutter 0.3.4

Add SORI-powered audio recognition, campaign discovery, and action-link handling to Flutter apps on Android, iOS, and Web.

sorisdk_flutter

Add SORI-powered audio recognition to Flutter apps, so audiences can interact with TV, radio, cinema, and other media from their phones. When registered media is recognized, your app can show the matching campaign, product information, coupon, event, or action link managed in SORI Console.

This package wraps the native SORI Android and iOS SDKs and adapts @sorisdk/web-audio for Flutter Web with one Flutter API for starting recognition, receiving campaign/result events, updating the recognition database where supported, and reporting handled action URLs.

When to use it

Use this plugin when your Flutter app needs a second-screen experience for:

• interactive ads and commerce links
• coupons, events, quizzes, or promotions tied to broadcast content
• product, cast, location, or soundtrack information while users watch or listen
• exposure and click reporting for media campaigns

Learn more in the SORI API docs, Use Cases, and SORI Console overview.

Setup

Before integrating the Flutter package, prepare SORI resources in SORI Console:

1. Create an app_id and secret_key in Managing Application.
2. Register the media that should be recognized in Managing Material.
3. Link a campaign, image, and action URL in Managing Campaign.

Android and iOS

Create a recognizer with your SORI mobile application credentials, subscribe to events, and start recognition.

final recognizer = SORIAudioRecognizer(
  applicationId: 'your-application-id',
  secretKey: 'your-secret-key',
);

await recognizer.configure(
  config: const SORIRecognitionConfig(audiomarker: true),
);

recognizer.events.listen((event) {
  final campaign = event.campaign;
  if (campaign != null) {
    // Render campaign.name, campaign.imageUrl, and campaign.actionUrl.
  }
  final marker = event.audioMarkerIdentity;
  if (marker != null) {
    // Use only the id and name managed in SORI Console.
    print('${marker.id}: ${marker.name}');
  }
});

await recognizer.startRecognition(
  notification: const SORIAndroidNotificationOptions(
    title: 'Listening',
    body: 'SORI audio recognition is active',
  ),
);

Web

Browser builds must not contain the long-lived SORI secretKey. Install the companion npm package in the Flutter application and prepare the browser runtime before each Web build:

npm install --save-dev @sorisdk/flutter-web-bridge
npx sori-flutter-web prepare
flutter build web

The preparation command resolves the pinned @sorisdk/web-audio version and bundles its ESM, WASM, and model files into the application's web/sorisdk/ directory. Flutter copies those application-owned files to build/web; the JS/WASM/model artifacts are not embedded in this pub.dev package.

Create the Web recognizer with an endpoint on your application server:

final recognizer = SORIAudioRecognizer.web(
  applicationId: 'your-application-id',
  webAuth: SORIWebAuthOptions.ephemeralKeyEndpoint(
    Uri.parse('/api/sori/ephemeral-key'),
    // Use `include` only when the endpoint relies on cross-origin cookies.
    requestCredentials: SORIWebRequestCredentials.sameOrigin,
  ),
);

await recognizer.configure();
recognizer.events.listen((event) {
  // ready, lifecycle, match, audiomarker, campaign, and error events use the
  // same SORIRecognitionEvent stream as the mobile implementations.
});

// Call this from a user action. Browser microphone permission is requested
// here, not while the recognizer is constructed or configured.
await recognizer.startRecognition();

The endpoint keeps both SORI_APP_ID and SORI_SECRET_KEY in the server environment, exchanges them at https://console.soriapi.com/api/auth/ephemeral, and returns only the ephemeral value to the browser. For example, the essential server-side flow is:

app.post("/api/sori/ephemeral-key", async (_request, response) => {
  const exchange = await fetch(
    "https://console.soriapi.com/api/auth/ephemeral",
    {
      method: "POST",
      body: new URLSearchParams({
        app_id: process.env.SORI_APP_ID,
        secret_key: process.env.SORI_SECRET_KEY,
      }),
    },
  );
  if (!exchange.ok) {
    response.sendStatus(exchange.status);
    return;
  }

  const payload = await exchange.json();
  response.json({
    ephemeralKey: payload.ephemeral_key ?? payload.ephemeralKey,
  });
});

Protect this application endpoint with the same session, origin, CSRF, and rate limits used for other authenticated application APIs. The plugin transparently requests the ephemeral key when startRecognition() runs, passes it to the SORI auth flow, loads the authorized audiopack, attaches the activity token, and then starts microphone recognition. It never receives the long-lived secret.

When a user taps a campaign action URL that your app has decided to open, report that interaction back to SORI:

final actionUrl = campaign.actionUrl;
if (actionUrl != null && actionUrl.isNotEmpty) {
  await recognizer.handleActionUrl(actionUrl);
}

App Requirements

Android permissions and the foreground microphone service declaration are merged from the plugin. On Android, startRecognition() requests runtime microphone permission when needed and continues the same start request after permission is granted. On Android 13 and later, request notification permission if your app targets that API level and you want the foreground-service notification to be shown normally.

iOS apps must add NSMicrophoneUsageDescription to Info.plist. Add UIBackgroundModes with audio only if the app is designed to continue recognition while backgrounded.

Flutter Web requires a secure context (https:// or http://localhost), a modern browser with Web Audio and getUserMedia(), the prepared companion bundle, and an application-server ephemeral-key endpoint. updateDatabase() currently returns an unsupported result on Web because @sorisdk/web-audio does not expose a refresh operation that avoids starting microphone capture.

Do not ship SORI credentials in sample or public source. Create and rotate app keys in SORI Console. Mobile apps load them through the normal release configuration process; Web apps must keep the secret on the server and must not pass SORI_SECRET_KEY to flutter build web.

Releasing

Flutter 0.3.4 includes the published Android 5.0.9 and iOS 5.0.8 artifacts from merged NADA 784725e85c34696ec40a987cedd55d030d7e50be, retaining legacy marker defaults. The 0.3.4 release evidence records the source revisions and archive checksums. Publication requires matching vendored file hashes.

Maintainers should publish this package by creating a GitHub Release with a new vX.Y.Z tag, such as v0.2.3, targeting the release commit on main. GitHub creates the tag for the Release, and that tag push triggers the pub.dev publish workflow. The workflow intentionally does not use the release event because pub.dev automated publishing with GitHub OIDC rejects tokens from that event. Ensure pub.dev automated publishing is configured for this repository with the matching v{{version}} tag pattern; permission to create a matching release tag is effectively permission to publish the package.

If pubspec.yaml includes build metadata, such as 0.2.3+4, use the base semantic version for the release tag (v0.2.3). The workflow publishes from a temporary workspace with the build metadata removed. Creating a Release from an already existing tag does not create a new tag push, so it will not start a new publish run; in that case, push the missing tag or rerun the existing tag workflow instead.

The companion npm package has independent SemVer and uses tags such as web-bridge-v0.1.0. A matching tag runs .github/workflows/publish-web-bridge.yml; configure npm trusted publishing for this repository and workflow before creating the first tag. Update the exact @sorisdk/web-audio dependency and the bridge version together whenever the browser runtime is upgraded.

Notes

The plugin exposes recognition lifecycle, campaign/result/error events, database updates, and an explicit handleActionUrl() method for URLs that your app has inspected. By default recognition is fingerprint-only; set SORIRecognitionConfig(audiomarker: true) to run near-ultrasonic audiomarker recognition alongside fingerprint recognition. A completed platform activity response containing nonblank audio_marker_id and audio_marker_name values is emitted as SORIRecognitionEventType.audioMarkerFound; read its structured event.audioMarkerIdentity. This event is independent from campaign delivery, so a marker resolved by a campaign-less activity response remains observable.

Local Android/iOS/WASM detector labels such as code_000 stay inside the platform reporter as request evidence. Flutter never maps trait.marker, a raw detector result, or a legacy String marker into the resolved identity. Missing, failed, disabled, or malformed resolution emits no fallback event.

SORIRecognitionEvent.audioMarker remains as a deprecated String view for source compatibility and now returns only the Console-resolved marker name. Use audioMarkerIdentity for both id and name. The legacy SORIRecognitionEventType.audioMarkerChanged value remains parseable, but current bridges emit audioMarkerFound; a legacy String payload is ignored so an internal codebook label cannot become public. SORIActivityTrait and SORICampaign.trait remain available for source compatibility with manually constructed or older application-owned models, but current SDK events do not populate them with marker evidence. Location, metadata provider, raw audio buffers, and automatic action URL opening are not part of the public Flutter API.

Activity identity and marker refinement

Campaign and recognition events expose an optional event.activityId. When a platform reporter supports activity refinement, the initial campaign event and its marker-enriched result carry the same non-empty activity id. Applications must use that value only as the exact server mutation identity; campaign and material ids are never substitutes for it.

The example keeps an explicit ordered current segment. An exact activity refinement can replace only that current row, a nonblank Console marker response makes the refined campaign fields sticky, and an intervening material seals the segment. Therefore A -> A' -> A (marker miss) retains the resolved campaign row, while A -> B -> A' cannot rewrite the older A row. Returning A with a new activity id creates a new row. Legacy events without an activity id remain parseable, but their material fallback is limited to the current segment. A stopped or destroyed recognizer also closes that boundary.

Supporting platform reporters apply the same boundary to server transport. A marker available on the first observation is included in the POST; marker evidence that arrives later may produce at most one PUT for that activity. Repeated observations can renew a separate local continuity lease, but the activity's recognized_at, position, and refinement deadline remain anchored to the first observation. audioMarkerFound remains independently observable, including for campaign-less responses, and never creates or refines a campaign row by itself.

Flutter 0.3.4 bundles these coordinated published releases:

• Android SDK 5.0.9 was released from sorisdk-android#61, merged source db1f1599aee39feb10d40bc0fb0e9b1580d70738, under tag v5.0.9. The exact Maven AAR has SHA-256 965538963f73dfff85efcbe7c1d2b5b94200bda0113cbfc7c1abd20ae268822e. Classes, all four JNI libraries, model and consumer rules are extracted from that archive. The existing resource alias remains compatible with its unchanged raw/model resource surface.
• iOS SDK 5.0.8 was released from sorisdk-ios#105, merged source f9184de69fb6b3010239a9b8b5c7de20e7a78d37, under tag v5.0.8. The public XCFramework archive has SHA-256 a244ea4af56a988abc225b1aad35cf1454e374fe273483d900c7c59a4a7da0c4 and matches the authenticated GitHub release asset. This repository vendors the framework extracted from those published bytes. Both native SDK sources pin NADA 784725e85c34696ec40a987cedd55d030d7e50be and retain legacy defaults.
• @sorisdk/web-audio 0.6.8 was released from NADA commit f46225c37f7654aef73f3213b5d5049fcaa56ac1 under tag wasm/v0.6.8. @sorisdk/flutter-web-bridge 0.1.3 pins the exact npm package, whose registry tarball has SHA-256 d4ad3018a820038c074d95fbed3888ef6440d032b28baeb84b5f725aca275b63 and npm integrity sha512-SZinJkMAFNIY0tb/0Ws/Mh2apPHL3I2gfN+NRteUF9gOA8PW0eHYlnlhUo+rBmzOTXeDLKOIEcgWSeAZsODpbw==. This release uses bridge ABI 2 because the marker event changed from a legacy String to the structured Console id/name payload; stale ABI 1 bundles are rejected explicitly.
• The campaign-less server response from sori_console#536 shipped in Console v1.15.7 from merge commit 3191c373acc52cdbc8f747e87a1b3ebcd5c7329d; its production marker-name smoke completed before this Flutter release.

The Flutter bridges surface activity, material, and marker data from those reporters; they do not duplicate the native or Web HTTP lifecycle state machines. The resolved-marker contract work is tracked in sorisdk-android#56, sorisdk-ios#101, nada#151, and sori_console#536.

See the example/ app for a complete recognition flow that reads mobile credentials or a Web ephemeral endpoint from --dart-define values and renders recognized campaigns.