sorisdk_flutter 0.3.8

Add SORI-powered audio recognition, campaign discovery, and action-link handling to Flutter apps on Android, iOS, and Web.

sorisdk_flutter

Add SORI-powered audio recognition to Flutter apps, so audiences can interact with TV, radio, cinema, and other media from their phones. When registered media is recognized, your app can show the matching campaign, product information, coupon, event, or action link managed in SORI Console.

This package wraps the native SORI Android and iOS SDKs and adapts @sorisdk/web-audio for Flutter Web with one Flutter API for starting recognition, receiving campaign/result events, updating the recognition database where supported, and reporting handled action URLs.

When to use it

Use this plugin when your Flutter app needs a second-screen experience for:

• interactive ads and commerce links
• coupons, events, quizzes, or promotions tied to broadcast content
• product, cast, location, or soundtrack information while users watch or listen
• exposure and click reporting for media campaigns

Learn more in the SORI API docs, Use Cases, and SORI Console overview.

Setup

Before integrating the Flutter package, prepare SORI resources in SORI Console:

1. Create an app_id and secret_key in Managing Application.
2. Register the media that should be recognized in Managing Material.
3. Link a campaign, image, and action URL in Managing Campaign.

Android and iOS

Create a recognizer with your SORI mobile application credentials, subscribe to events, and start recognition.

final recognizer = SORIAudioRecognizer(
  applicationId: 'your-application-id',
  secretKey: 'your-secret-key',
);

await recognizer.configure(
  config: const SORIRecognitionConfig(audiomarker: true),
);

recognizer.events.listen((event) {
  final campaign = event.campaign;
  if (campaign != null) {
    // Render campaign.name, campaign.imageUrl, and campaign.actionUrl.
  }
  final marker = event.audioMarkerIdentity;
  if (marker != null) {
    // Use only the id and name managed in SORI Console.
    print('${marker.id}: ${marker.name}');
  }
});

await recognizer.startRecognition(
  notification: const SORIAndroidNotificationOptions(
    title: 'Listening',
    body: 'SORI audio recognition is active',
  ),
);

Web

Browser builds must not contain the long-lived SORI secretKey. Install the companion npm package in the Flutter application and prepare the browser runtime before each Web build:

npm install --save-dev @sorisdk/flutter-web-bridge
npx sori-flutter-web prepare
flutter build web

The preparation command resolves the installed @sorisdk/web-audio version and bundles its ESM, WASM, and model files into the application's web/sorisdk/ directory. Flutter copies those application-owned files to build/web; the JS/WASM/model artifacts are not embedded in this pub.dev package.

Bridge 0.1.5 supports Web SDK ^0.6.10 (>=0.6.10 <0.7.0) with the existing Flutter 0.3.7 Dart package and ABI 2. Compatible Web SDK patches do not require another Flutter or bridge release. Keep the application's npm lockfile for reproducible builds with npm ci, then update on your own schedule:

npm update @sorisdk/web-audio
npx sori-flutter-web prepare
flutter build web

Bridge 0.1.5 raises the minimum Web SDK to 0.6.10 so fresh installations use the shared level-robust marker default. Existing Dart application code and ABI 2 remain compatible; audiomarker: false still disables marker recognition.

Test and commit the updated lockfile before deployment. The generated sori-bridge-manifest.json records the resolved Web SDK version; deployed apps retain their bundled runtime until rebuilt and deployed.

Create the Web recognizer with an endpoint on your application server:

final recognizer = SORIAudioRecognizer.web(
  applicationId: 'your-application-id',
  webAuth: SORIWebAuthOptions.ephemeralKeyEndpoint(
    Uri.parse('/api/sori/ephemeral-key'),
    // Use `include` only when the endpoint relies on cross-origin cookies.
    requestCredentials: SORIWebRequestCredentials.sameOrigin,
  ),
);

await recognizer.configure();
recognizer.events.listen((event) {
  // ready, lifecycle, match, audiomarker, campaign, and error events use the
  // same SORIRecognitionEvent stream as the mobile implementations.
});

// Call this from a user action. Browser microphone permission is requested
// here, not while the recognizer is constructed or configured.
await recognizer.startRecognition();

The endpoint keeps both SORI_APP_ID and SORI_SECRET_KEY in the server environment, exchanges them at https://console.soriapi.com/api/auth/ephemeral, and returns only the ephemeral value to the browser. For example, the essential server-side flow is:

app.post("/api/sori/ephemeral-key", async (_request, response) => {
  const exchange = await fetch(
    "https://console.soriapi.com/api/auth/ephemeral",
    {
      method: "POST",
      body: new URLSearchParams({
        app_id: process.env.SORI_APP_ID,
        secret_key: process.env.SORI_SECRET_KEY,
      }),
    },
  );
  if (!exchange.ok) {
    response.sendStatus(exchange.status);
    return;
  }

  const payload = await exchange.json();
  response.json({
    ephemeralKey: payload.ephemeral_key ?? payload.ephemeralKey,
  });
});

Protect this application endpoint with the same session, origin, CSRF, and rate limits used for other authenticated application APIs. The plugin transparently requests the ephemeral key when startRecognition() runs, passes it to the SORI auth flow, loads the authorized audiopack, attaches the activity token, and then starts microphone recognition. It never receives the long-lived secret.

When a user taps a campaign action URL that your app has decided to open, report that interaction back to SORI:

final actionUrl = campaign.actionUrl;
if (actionUrl != null && actionUrl.isNotEmpty) {
  await recognizer.handleActionUrl(actionUrl);
}

App Requirements

Android permissions and the foreground microphone service declaration are merged from the plugin. On Android, startRecognition() requests runtime microphone permission when needed and continues the same start request after permission is granted. On Android 13 and later, request notification permission if your app targets that API level and you want the foreground-service notification to be shown normally.

See recognition lifecycle ownership for repeated starts, pending-request cancellation, and the required native terminal-state contracts.

iOS apps must add NSMicrophoneUsageDescription to Info.plist. Add UIBackgroundModes with audio only if the app is designed to continue recognition while backgrounded.

Flutter Web requires a secure context (https:// or http://localhost), a modern browser with Web Audio and getUserMedia(), the prepared companion bundle, and an application-server ephemeral-key endpoint. updateDatabase() currently returns an unsupported result on Web because @sorisdk/web-audio does not expose a refresh operation that avoids starting microphone capture.

Do not ship SORI credentials in sample or public source. Create and rotate app keys in SORI Console. Mobile apps load them through the normal release configuration process; Web apps must keep the secret on the server and must not pass SORI_SECRET_KEY to flutter build web.

Releasing

Flutter 0.3.8 bundles the published Android 5.0.13 and iOS 5.0.12 artifacts. Their merged source commits, public archive checksums, copied-file evidence and validation limits are recorded in the release evidence and the provenance manifest.

npm run test:native-build checks the vendored bytes during development. The tag publisher requires node scripts/check-native-release-provenance.mjs to verify the published dependencies and all vendored file hashes and framework executable modes. Physical acoustic and real Console field acceptance remain separate. Android improvement is user-reported; this release does not claim new quantitative physical-device measurements or completion of NADA #154.

Adopt exact public native artifacts and update the package version on a feature branch. Merge the validated implementation PR, then tag that verified commit on main for publication.

Maintainers publish this package by pushing a new vX.Y.Z tag, such as v0.3.8, at the verified release commit on main. That tag push triggers the pub.dev publish workflow. The workflow intentionally does not use the release event because pub.dev automated publishing with GitHub OIDC rejects tokens from that event. Ensure pub.dev automated publishing is configured for this repository with the matching v{{version}} tag pattern; permission to create a matching release tag is effectively permission to publish the package.

If pubspec.yaml includes build metadata, such as 0.2.3+4, use the base semantic version for the release tag (v0.2.3). The workflow publishes from a temporary workspace with the build metadata removed. Creating a Release from an already existing tag does not create a new tag push, so it will not start a new publish run; in that case, push the missing tag or rerun the existing tag workflow instead.

The companion npm package has independent SemVer and uses tags such as web-bridge-v0.1.0. A matching tag runs .github/workflows/publish-web-bridge.yml; configure npm trusted publishing for this repository and workflow before creating the first tag. Publish a new bridge when its minimum supported Web SDK, compatibility range, or bridge behavior changes. Consumers can adopt later patches within ^0.6.10 by updating their npm lockfiles and rebuilding, without another bridge release. Review compatibility before expanding the range to Web SDK 0.7.0 or later.

Notes

The plugin exposes recognition lifecycle, campaign/result/error events, database updates, and an explicit handleActionUrl() method for URLs that your app has inspected. By default recognition is fingerprint-only; set SORIRecognitionConfig(audiomarker: true) to run near-ultrasonic audiomarker recognition alongside fingerprint recognition. A completed platform activity response containing nonblank audio_marker_id and audio_marker_name values is emitted as SORIRecognitionEventType.audioMarkerFound; read its structured event.audioMarkerIdentity. This event is independent from campaign delivery, so a marker resolved by a campaign-less activity response remains observable.

Local Android/iOS/WASM detector labels such as code_000 stay inside the platform reporter as request evidence. Flutter never maps trait.marker, a raw detector result, or a legacy String marker into the resolved identity. Missing, failed, disabled, or malformed resolution emits no fallback event.

SORIRecognitionEvent.audioMarker remains as a deprecated String view for source compatibility and now returns only the Console-resolved marker name. Use audioMarkerIdentity for both id and name. The legacy SORIRecognitionEventType.audioMarkerChanged value remains parseable, but current bridges emit audioMarkerFound; a legacy String payload is ignored so an internal codebook label cannot become public. SORIActivityTrait and SORICampaign.trait remain available for source compatibility with manually constructed or older application-owned models, but current SDK events do not populate them with marker evidence. Location, metadata provider, raw audio buffers, and automatic action URL opening are not part of the public Flutter API.

Current marker state and accepted activity results

On Android and iOS, audiomarker: true enables SDK-owned current marker state. audioMarkerStateChanged carries event.audioMarkerState: inactive, listening, detected (no public identity yet), identified, or cleared. Only identified state contains a Console marker. Fresh evidence bridges the next material cycle; three approximately one-second misses retain it and the fourth clears it. Native expiry also runs without callbacks or during silence. Stop/reset and invalid capture clear state. Reading a snapshot, receiving a material result or waiting for a server response never renews evidence.

activityResult carries event.activityResult, including the exact server activityId, optional campaign and that activity's Console-resolved marker. A result may have no campaign. A late marker can upgrade the existing activity without another successful material match. Current-state expiry does not remove historical result markers. The native SDK owns trait selection and POST/PUT; applications do not construct traits or issue activity updates.

The package provides a small presentation reducer for ordered snapshots and exact-ID result updates:

final state = SORIRecognitionState();
final subscription = recognizer.events.listen((event) {
  if (state.accept(event)) {
    // Render state.currentAudioMarker and state.activities.
    // Results retain insertion order; enrichment updates the same activity.
  }
});

// Optional, for an explicit read. Native event reattachment also replays state.
final snapshot = await recognizer.getCurrentAudioMarkerState();
if (snapshot != null) state.acceptAudioMarkerState(snapshot);

The reducer only applies SDK values, ignores old channel deliveries and prevents a delayed initial duplicate from removing an accepted result marker. It does not infer eligibility or run an expiry timer. Native generation and revision are preserved; sequence orders bridge delivery/snapshot reads within one plugin instance. Optional Android evidence timestamps use native monotonic time, not Unix time; they are diagnostic values, not a Dart timer contract.

Legacy campaignFound, recognitionResult and resolved discovery callbacks remain compatible. event.audioMarkerIdentity retains discovery semantics; read event.activityResult?.audioMarkerIdentity for an activity's identity. Existing result payloads with a valid activity ID also gain the typed result. A material or campaign ID is never substituted for a missing activity ID.

Tested platform boundary

Capability	Android 5.0.13 build	iOS 5.0.12 build	Web 0.6.10
Resolved Console discovery	Yes	Yes	Existing support
Native current state / snapshot / explicit clear	Yes	Yes	Not available; snapshot returns null
State-driven initial traits and independent late activity upgrade	Native implementation	Native implementation	Not claimed
Existing exact-ID result/refinement decoding	Yes	Yes	Preserved and tested
Physical acoustic / real Console field acceptance	Pending	Pending	Not established by this change

The npm registry and published @sorisdk/web-audio 0.6.10 declarations were checked on 2026-09-14: they expose resolved audiomarker and campaign events, but no current-state/snapshot API. Bridge 0.1.5 retains ABI 2 and supports compatible patches through ^0.6.10. Web needs an upstream state/expiry/independent-upgrade implementation and validation before parity can be claimed; this gap is recorded under Flutter #27. Web scheduling #153 is related work, not evidence of the missing state contract. No Dart Web expiry or PUT fallback is supplied.

See the example/ app for a complete recognition flow that reads mobile credentials or a Web ephemeral endpoint from --dart-define values and renders recognized campaigns.