sorisdk_flutter 0.3.3

Add SORI-powered audio recognition, campaign discovery, and action-link handling to Flutter apps on Android, iOS, and Web.

sorisdk_flutter

Add SORI-powered audio recognition to Flutter apps, so audiences can interact with TV, radio, cinema, and other media from their phones. When registered media is recognized, your app can show the matching campaign, product information, coupon, event, or action link managed in SORI Console.

This package wraps the native SORI Android and iOS SDKs and adapts @sorisdk/web-audio for Flutter Web with one Flutter API for starting recognition, receiving campaign/result events, updating the recognition database where supported, and reporting handled action URLs.

When to use it

Use this plugin when your Flutter app needs a second-screen experience for:

• interactive ads and commerce links
• coupons, events, quizzes, or promotions tied to broadcast content
• product, cast, location, or soundtrack information while users watch or listen
• exposure and click reporting for media campaigns

Learn more in the SORI API docs, Use Cases, and SORI Console overview.

Setup

Before integrating the Flutter package, prepare SORI resources in SORI Console:

1. Create an app_id and secret_key in Managing Application.
2. Register the media that should be recognized in Managing Material.
3. Link a campaign, image, and action URL in Managing Campaign.

Android and iOS

Create a recognizer with your SORI mobile application credentials, subscribe to events, and start recognition.

final recognizer = SORIAudioRecognizer(
  applicationId: 'your-application-id',
  secretKey: 'your-secret-key',
);

await recognizer.configure(
  config: const SORIRecognitionConfig(audiomarker: true),
);

recognizer.events.listen((event) {
  final campaign = event.campaign;
  if (campaign != null) {
    // Render campaign.name, campaign.imageUrl, and campaign.actionUrl.
  }
  final marker = event.audioMarkerIdentity;
  if (marker != null) {
    // Use only the id and name managed in SORI Console.
    print('${marker.id}: ${marker.name}');
  }
});

await recognizer.startRecognition(
  notification: const SORIAndroidNotificationOptions(
    title: 'Listening',
    body: 'SORI audio recognition is active',
  ),
);

Web

Browser builds must not contain the long-lived SORI secretKey. Install the companion npm package in the Flutter application and prepare the browser runtime before each Web build:

npm install --save-dev @sorisdk/flutter-web-bridge
npx sori-flutter-web prepare
flutter build web

The preparation command resolves the pinned @sorisdk/web-audio version and bundles its ESM, WASM, and model files into the application's web/sorisdk/ directory. Flutter copies those application-owned files to build/web; the JS/WASM/model artifacts are not embedded in this pub.dev package.

Create the Web recognizer with an endpoint on your application server:

final recognizer = SORIAudioRecognizer.web(
  applicationId: 'your-application-id',
  webAuth: SORIWebAuthOptions.ephemeralKeyEndpoint(
    Uri.parse('/api/sori/ephemeral-key'),
    // Use `include` only when the endpoint relies on cross-origin cookies.
    requestCredentials: SORIWebRequestCredentials.sameOrigin,
  ),
);

await recognizer.configure();
recognizer.events.listen((event) {
  // ready, lifecycle, match, audiomarker, campaign, and error events use the
  // same SORIRecognitionEvent stream as the mobile implementations.
});

// Call this from a user action. Browser microphone permission is requested
// here, not while the recognizer is constructed or configured.
await recognizer.startRecognition();

The endpoint keeps both SORI_APP_ID and SORI_SECRET_KEY in the server environment, exchanges them at https://console.soriapi.com/api/auth/ephemeral, and returns only the ephemeral value to the browser. For example, the essential server-side flow is:

app.post("/api/sori/ephemeral-key", async (_request, response) => {
  const exchange = await fetch(
    "https://console.soriapi.com/api/auth/ephemeral",
    {
      method: "POST",
      body: new URLSearchParams({
        app_id: process.env.SORI_APP_ID,
        secret_key: process.env.SORI_SECRET_KEY,
      }),
    },
  );
  if (!exchange.ok) {
    response.sendStatus(exchange.status);
    return;
  }

  const payload = await exchange.json();
  response.json({
    ephemeralKey: payload.ephemeral_key ?? payload.ephemeralKey,
  });
});

Protect this application endpoint with the same session, origin, CSRF, and rate limits used for other authenticated application APIs. The plugin transparently requests the ephemeral key when startRecognition() runs, passes it to the SORI auth flow, loads the authorized audiopack, attaches the activity token, and then starts microphone recognition. It never receives the long-lived secret.

When a user taps a campaign action URL that your app has decided to open, report that interaction back to SORI:

final actionUrl = campaign.actionUrl;
if (actionUrl != null && actionUrl.isNotEmpty) {
  await recognizer.handleActionUrl(actionUrl);
}

App Requirements

Android permissions and the foreground microphone service declaration are merged from the plugin. On Android, startRecognition() requests runtime microphone permission when needed and continues the same start request after permission is granted. On Android 13 and later, request notification permission if your app targets that API level and you want the foreground-service notification to be shown normally.

iOS apps must add NSMicrophoneUsageDescription to Info.plist. Add UIBackgroundModes with audio only if the app is designed to continue recognition while backgrounded.

Flutter Web requires a secure context (https:// or http://localhost), a modern browser with Web Audio and getUserMedia(), the prepared companion bundle, and an application-server ephemeral-key endpoint. updateDatabase() currently returns an unsupported result on Web because @sorisdk/web-audio does not expose a refresh operation that avoids starting microphone capture.

Do not ship SORI credentials in sample or public source. Create and rotate app keys in SORI Console. Mobile apps load them through the normal release configuration process; Web apps must keep the secret on the server and must not pass SORI_SECRET_KEY to flutter build web.

Releasing

Maintainers should publish this package by creating a GitHub Release with a new vX.Y.Z tag, such as v0.2.3, targeting the release commit on main. GitHub creates the tag for the Release, and that tag push triggers the pub.dev publish workflow. The workflow intentionally does not use the release event because pub.dev automated publishing with GitHub OIDC rejects tokens from that event. Ensure pub.dev automated publishing is configured for this repository with the matching v{{version}} tag pattern; permission to create a matching release tag is effectively permission to publish the package.

If pubspec.yaml includes build metadata, such as 0.2.3+4, use the base semantic version for the release tag (v0.2.3). The workflow publishes from a temporary workspace with the build metadata removed. Creating a Release from an already existing tag does not create a new tag push, so it will not start a new publish run; in that case, push the missing tag or rerun the existing tag workflow instead.

The companion npm package has independent SemVer and uses tags such as web-bridge-v0.1.0. A matching tag runs .github/workflows/publish-web-bridge.yml; configure npm trusted publishing for this repository and workflow before creating the first tag. Update the exact @sorisdk/web-audio dependency and the bridge version together whenever the browser runtime is upgraded.

Notes

The plugin exposes recognition lifecycle, campaign/result/error events, database updates, and an explicit handleActionUrl() method for URLs that your app has inspected. By default recognition is fingerprint-only; set SORIRecognitionConfig(audiomarker: true) to run near-ultrasonic audiomarker recognition alongside fingerprint recognition. A completed platform activity response containing nonblank audio_marker_id and audio_marker_name values is emitted as SORIRecognitionEventType.audioMarkerFound; read its structured event.audioMarkerIdentity. This event is independent from campaign delivery, so a marker resolved by a campaign-less activity response remains observable.

Local Android/iOS/WASM detector labels such as code_000 stay inside the platform reporter as request evidence. Flutter never maps trait.marker, a raw detector result, or a legacy String marker into the resolved identity. Missing, failed, disabled, or malformed resolution emits no fallback event.

SORIRecognitionEvent.audioMarker remains as a deprecated String view for source compatibility and now returns only the Console-resolved marker name. Use audioMarkerIdentity for both id and name. The legacy SORIRecognitionEventType.audioMarkerChanged value remains parseable, but current bridges emit audioMarkerFound; a legacy String payload is ignored so an internal codebook label cannot become public. SORIActivityTrait and SORICampaign.trait remain available for source compatibility with manually constructed or older application-owned models, but current SDK events do not populate them with marker evidence. Location, metadata provider, raw audio buffers, and automatic action URL opening are not part of the public Flutter API.

Activity identity and marker refinement

Campaign and recognition events expose an optional event.activityId. When a platform reporter supports activity refinement, the initial campaign event and its marker-enriched result carry the same non-empty activity id. Applications must use that value only as the exact server mutation identity; campaign and material ids are never substitutes for it.

The example keeps an explicit ordered current segment. An exact activity refinement can replace only that current row, a nonblank Console marker response makes the refined campaign fields sticky, and an intervening material seals the segment. Therefore A -> A' -> A (marker miss) retains the resolved campaign row, while A -> B -> A' cannot rewrite the older A row. Returning A with a new activity id creates a new row. Legacy events without an activity id remain parseable, but their material fallback is limited to the current segment. A stopped or destroyed recognizer also closes that boundary.

Supporting platform reporters apply the same boundary to server transport. A marker available on the first observation is included in the POST; marker evidence that arrives later may produce at most one PUT for that activity. Repeated observations can renew a separate local continuity lease, but the activity's recognized_at, position, and refinement deadline remain anchored to the first observation. audioMarkerFound remains independently observable, including for campaign-less responses, and never creates or refines a campaign row by itself.

Flutter 0.3.3+3 bundles these coordinated published releases:

• Android SDK 5.0.8 was reviewed at 7b272acd90e56adb164f8bf70cf067798ee99831 in sorisdk-android#57 and released from merge commit 54f619bfd26b5989ffc6edd61bdb9c9748035170 as tag v5.0.8. The exact com.iplateia:sorisdk:5.0.8 Maven AAR has SHA-256 0f894732afc6212d1d1b977f866f1bdcf8b00558d358703dff3927db270fc3da; its classes, JNI libraries, model, and consumer rules were revalidated and the vendored components were extracted from those published bytes.
• iOS SDK 5.0.7 was reviewed at 1757ae69c68d72d99777b2f03d66b7516581b6e4 in sorisdk-ios#102 and released from merge commit d8175fc630ed7cd566f04d97cb7de801e3cdda34 as tag v5.0.7. The GitHub XCFramework archive and its versioned S3 copy are byte-identical with SHA-256 f20448c2d5a3f935b66a30f189790d7dbebf15baa054f6be3f9dba37195204b8; this repository vendors the XCFramework extracted from that archive.
• @sorisdk/web-audio 0.6.8 was released from NADA commit f46225c37f7654aef73f3213b5d5049fcaa56ac1 under tag wasm/v0.6.8. @sorisdk/flutter-web-bridge 0.1.3 pins the exact npm package, whose registry tarball has SHA-256 d4ad3018a820038c074d95fbed3888ef6440d032b28baeb84b5f725aca275b63 and npm integrity sha512-SZinJkMAFNIY0tb/0Ws/Mh2apPHL3I2gfN+NRteUF9gOA8PW0eHYlnlhUo+rBmzOTXeDLKOIEcgWSeAZsODpbw==. This release uses bridge ABI 2 because the marker event changed from a legacy String to the structured Console id/name payload; stale ABI 1 bundles are rejected explicitly.
• The campaign-less server response from sori_console#536 shipped in Console v1.15.7 from merge commit 3191c373acc52cdbc8f747e87a1b3ebcd5c7329d; its production marker-name smoke completed before this Flutter release.

The Flutter bridges surface activity, material, and marker data from those reporters; they do not duplicate the native or Web HTTP lifecycle state machines. The resolved-marker contract work is tracked in sorisdk-android#56, sorisdk-ios#101, nada#151, and sori_console#536.

See the example/ app for a complete recognition flow that reads mobile credentials or a Web ephemeral endpoint from --dart-define values and renders recognized campaigns.