sorisdk_flutter 0.4.1

Add SORI-powered audio recognition, campaign discovery, and action-link handling to Flutter apps on Android, iOS, and Web.

sorisdk_flutter

Add SORI-powered audio recognition to Flutter apps, so audiences can interact with TV, radio, cinema, and other media from their phones. When registered media is recognized, your app can show the matching campaign, product information, coupon, event, or action link managed in SORI Console.

This package wraps the native SORI Android and iOS SDKs and adapts @sorisdk/web-audio for Flutter Web with one Flutter API for starting recognition, receiving campaign/result events, updating the recognition database where supported, and reporting handled action URLs.

When to use it

Use this plugin when your Flutter app needs a second-screen experience for:

• interactive ads and commerce links
• coupons, events, quizzes, or promotions tied to broadcast content
• product, cast, location, or soundtrack information while users watch or listen
• exposure and click reporting for media campaigns

Learn more in the SORI API docs, Use Cases, and SORI Console overview.

Setup

Before integrating the Flutter package, prepare SORI resources in SORI Console:

1. Create an app_id and secret_key in Managing Application.
2. Register the media that should be recognized in Managing Material.
3. Link a campaign, image, and action URL in Managing Campaign.

Android and iOS

Create a recognizer with your SORI mobile application credentials, subscribe to events, and start recognition.

final recognizer = SORIAudioRecognizer(
  applicationId: 'your-application-id',
  secretKey: 'your-secret-key',
);

await recognizer.configure(
  config: const SORIRecognitionConfig(audiomarker: true),
);

recognizer.events.listen((event) {
  final campaign = event.campaign;
  if (campaign != null) {
    // Render campaign.name, campaign.imageUrl, and campaign.actionUrl.
  }
  final marker = event.audioMarkerIdentity;
  if (marker != null) {
    // Use only the id and name managed in SORI Console.
    print('${marker.id}: ${marker.name}');
  }
});

await recognizer.startRecognition(
  notification: const SORIAndroidNotificationOptions(
    title: 'Listening',
    body: 'SORI audio recognition is active',
  ),
);

Web

Browser builds must not contain the long-lived SORI secretKey. Install the companion npm package in the Flutter application and prepare the browser runtime before each Web build:

npm install --save-dev @sorisdk/flutter-web-bridge
npx sori-flutter-web prepare
flutter build web

The preparation command resolves the installed @sorisdk/web-audio version and bundles its ESM, WASM, and model files into the application's web/sorisdk/ directory. Flutter copies those application-owned files to build/web; the JS/WASM/model artifacts are not embedded in this pub.dev package.

Bridge 0.1.5 supports Web SDK ^0.6.10 (>=0.6.10 <0.7.0) with the existing Flutter 0.3.7 Dart package and ABI 2. Compatible Web SDK patches do not require another Flutter or bridge release. Keep the application's npm lockfile for reproducible builds with npm ci, then update on your own schedule:

npm update @sorisdk/web-audio
npx sori-flutter-web prepare
flutter build web

Bridge 0.1.5 raises the minimum Web SDK to 0.6.10 so fresh installations use the shared level-robust marker default. Existing Dart application code and ABI 2 remain compatible; audiomarker: false still disables marker recognition.

Test and commit the updated lockfile before deployment. The generated sori-bridge-manifest.json records the resolved Web SDK version; deployed apps retain their bundled runtime until rebuilt and deployed.

Create the Web recognizer with an endpoint on your application server:

final recognizer = SORIAudioRecognizer.web(
  applicationId: 'your-application-id',
  webAuth: SORIWebAuthOptions.ephemeralKeyEndpoint(
    Uri.parse('/api/sori/ephemeral-key'),
    // Use `include` only when the endpoint relies on cross-origin cookies.
    requestCredentials: SORIWebRequestCredentials.sameOrigin,
  ),
);

await recognizer.configure();
recognizer.events.listen((event) {
  // ready, lifecycle, match, audiomarker, campaign, and error events use the
  // same SORIRecognitionEvent stream as the mobile implementations.
});

// Call this from a user action. Browser microphone permission is requested
// here, not while the recognizer is constructed or configured.
await recognizer.startRecognition();

The endpoint keeps both SORI_APP_ID and SORI_SECRET_KEY in the server environment, exchanges them at https://console.soriapi.com/api/auth/ephemeral, and returns only the ephemeral value to the browser. For example, the essential server-side flow is:

app.post("/api/sori/ephemeral-key", async (_request, response) => {
  const exchange = await fetch(
    "https://console.soriapi.com/api/auth/ephemeral",
    {
      method: "POST",
      body: new URLSearchParams({
        app_id: process.env.SORI_APP_ID,
        secret_key: process.env.SORI_SECRET_KEY,
      }),
    },
  );
  if (!exchange.ok) {
    response.sendStatus(exchange.status);
    return;
  }

  const payload = await exchange.json();
  response.json({
    ephemeralKey: payload.ephemeral_key ?? payload.ephemeralKey,
  });
});

Protect this application endpoint with the same session, origin, CSRF, and rate limits used for other authenticated application APIs. The plugin transparently requests the ephemeral key when startRecognition() runs, passes it to the SORI auth flow, loads the authorized audiopack, attaches the activity token, and then starts microphone recognition. It never receives the long-lived secret.

When a user taps a campaign action URL that your app has decided to open, report that interaction back to SORI:

final actionUrl = campaign.actionUrl;
if (actionUrl != null && actionUrl.isNotEmpty) {
  await recognizer.handleActionUrl(actionUrl);
}

App Requirements

Android permissions and the foreground microphone service declaration are merged from the plugin. On Android, startRecognition() requests runtime microphone permission when needed and continues the same start request after permission is granted. On Android 13 and later, request notification permission if your app targets that API level and you want the foreground-service notification to be shown normally.

See recognition lifecycle ownership for repeated starts, pending-request cancellation, and the required native terminal-state contracts.

iOS apps must add NSMicrophoneUsageDescription to Info.plist. Add UIBackgroundModes with audio only if the app is designed to continue recognition while backgrounded.

Flutter Web requires a secure context (https:// or http://localhost), a modern browser with Web Audio and getUserMedia(), the prepared companion bundle, and an application-server ephemeral-key endpoint. updateDatabase() currently returns an unsupported result on Web because @sorisdk/web-audio does not expose a refresh operation that avoids starting microphone capture.

Do not ship SORI credentials in sample or public source. Create and rotate app keys in SORI Console. Mobile apps load them through the normal release configuration process; Web apps must keep the secret on the server and must not pass SORI_SECRET_KEY to flutter build web.

iOS app deployment target and Spring 2027 submissions

Apple ContentDelivery warning 90068, recorded on September 14, 2026 in #37, is advance notice: starting in Spring 2027, iOS apps uploaded/distributed through App Store Connect (including TestFlight) must have MinimumOSVersion 15.0 or later. The warning did not fail that upload and did not specify an exact enforcement date. Prepare consumer host apps before Spring 2027.

In the host app's ios/Runner.xcworkspace, set iOS Deployment Target (IPHONEOS_DEPLOYMENT_TARGET) to 15.0 or later for the Runner project and Runner target in every shipping configuration, including Release/Profile and any custom flavors. Keep Debug and test targets aligned. Set the host ios/Podfile to the same minimum:

platform :ios, '15.0'

Run flutter pub get from the host app root, then pod install from its ios directory and rebuild the app. Do not apply a blanket minimum-version override to every Pods target. Apple's MinimumOSVersion documentation describes the generated app field; configure the deployment target instead of editing a built plist. See also Flutter's Runner settings guide. Compiling with a recent iOS SDK/Xcode is a separate submission requirement and does not by itself raise the app's minimum supported OS.

This is an application submission setting, separate from reusable-library compatibility. The plugin podspec still supports iOS 13.0; the vendored SORI 5.1.1 device framework retains its 12.0 plist/Mach-O floor. An iOS 15 host can use those libraries. This warning alone does not require a native SDK rebuild or release. Raising a library support floor needs a separate compatibility decision, coordinated release notes and verified artifact adoption. Never rewrite vendored framework metadata or provenance to suppress the warning.

Before submission, inspect the actual exported app, not just source settings. After building/exporting the host IPA with your normal signing configuration, extract it on macOS and inspect its app plist and executable:

ipa_check_dir="$(mktemp -d)"
ditto -x -k /path/to/YourApp.ipa "$ipa_check_dir"
app="$ipa_check_dir/Payload/YourApp.app" # Use the actual exported .app name.
/usr/libexec/PlistBuddy -c 'Print :MinimumOSVersion' "$app/Info.plist"
app_executable="$(/usr/libexec/PlistBuddy -c 'Print :CFBundleExecutable' "$app/Info.plist")"
xcrun otool -arch all -l "$app/$app_executable"

Confirm the app's MinimumOSVersion and every executable slice's LC_BUILD_VERSION → minos (or LC_VERSION_MIN_IPHONEOS → version) match the intended deployment target and are 15.0 or later. The Mach-O sdk field is the build SDK version, not the deployment target. For an archive, inspect YourApp.xcarchive/Products/Applications/YourApp.app the same way; repeat on the exported IPA intended for upload. Embedded reusable frameworks may retain lower compatible minimums. The maintained example pins its host to 15.0 and includes a configuration/artifact check.

AudioPack refresh (native 5.1.0)

await recognizer.updateDatabase() retains Future<SORIUpdateResult> and its four existing fields. Native 5.1.0 prepares and validates a candidate while recognition continues against the working database, then publishes it at a safe query boundary. Keep the existing recognition subscription attached; no stop or restart is needed for a refresh with the same configuration.

• Android: forwards native UpdateResult unchanged. A successful reply can describe a pack cached for the next start when the recognition service is not running/bound; it is not a portable guarantee that the active database changed.
• iOS: invokes the released asynchronous, void updateDatabase() API. Since it exposes no completion/result, Flutter returns success: false, updateAvailable: false, currentVersion: '', and the diagnostic AudioPack refresh requested; iOS SDK does not expose its outcome. These are unknown-result placeholders, not a native failure, no-change result or active version. This corrects the earlier unconditional success: true; callers must not retry merely because this iOS reply has success == false.
• Web: retains its existing unsupported reply. Bridge 0.1.5, Web Audio ^0.6.10 and their locked versions are unchanged; no npm release is needed.

The released native APIs have no refresh request IDs, terminal outcome callbacks, active-version replay or public APPLIED/PENDING/NO_CHANGE/SUPERSEDED/FAILED contract. The earlier design in #39 predates the delivered native releases; this plugin does not synthesize those outcomes from recognition or marker events. The example's native Refresh AudioPack button displays the available reply without claiming the new pack has become active.

Releasing

Flutter 0.4.1 bundles published Android and iOS 5.1.1 artifacts with NADA matcher 0.6.11, correcting multi-segment file recognition positions. iOS also includes rejected-authentication handling and cancellation fixes. Their tagged source commits, public archive checksums, copied-file evidence and validation limits are recorded in the release evidence and the provenance manifest.

npm run test:native-build checks the vendored bytes during development. The tag publisher requires node scripts/check-native-release-provenance.mjs to verify the published dependencies and all vendored file hashes and framework executable modes. Physical acoustic and real Console field acceptance remain separate. This adoption does not claim physical microphone continuity, large-account refresh timing/memory, or completion of NADA #154.

Implementation PRs may adopt exact-source local builds with explicit awaiting_publication provenance. Before Flutter publication, adopt and verify the public native artifacts and update that provenance on a feature branch. Merge the validated implementation PR, then tag the verified commit on main.

Maintainers publish this package by pushing a new vX.Y.Z tag, such as v0.4.1, at the verified release commit on main. That tag push triggers the pub.dev publish workflow. The workflow intentionally does not use the release event because pub.dev automated publishing with GitHub OIDC rejects tokens from that event. Ensure pub.dev automated publishing is configured for this repository with the matching v{{version}} tag pattern; permission to create a matching release tag is effectively permission to publish the package.

If pubspec.yaml includes build metadata, such as 0.2.3+4, use the base semantic version for the release tag (v0.2.3). The workflow publishes from a temporary workspace with the build metadata removed. Creating a Release from an already existing tag does not create a new tag push, so it will not start a new publish run; in that case, push the missing tag or rerun the existing tag workflow instead.

The companion npm package has independent SemVer and uses tags such as web-bridge-v0.1.0. A matching tag runs .github/workflows/publish-web-bridge.yml; configure npm trusted publishing for this repository and workflow before creating the first tag. Publish a new bridge when its minimum supported Web SDK, compatibility range, or bridge behavior changes. Consumers can adopt later patches within ^0.6.10 by updating their npm lockfiles and rebuilding, without another bridge release. Review compatibility before expanding the range to Web SDK 0.7.0 or later.

Notes

The plugin exposes recognition lifecycle, campaign/result/error events, database updates, and an explicit handleActionUrl() method for URLs that your app has inspected. By default recognition is fingerprint-only; set SORIRecognitionConfig(audiomarker: true) to run near-ultrasonic audiomarker recognition alongside fingerprint recognition. A completed platform activity response containing nonblank audio_marker_id and audio_marker_name values is emitted as SORIRecognitionEventType.audioMarkerFound; read its structured event.audioMarkerIdentity. This event is independent from campaign delivery, so a marker resolved by a campaign-less activity response remains observable.

Local Android/iOS/WASM detector labels such as code_000 stay inside the platform reporter as request evidence. Flutter never maps trait.marker, a raw detector result, or a legacy String marker into the resolved identity. Missing, failed, disabled, or malformed resolution emits no fallback event.

SORIRecognitionEvent.audioMarker remains as a deprecated String view for source compatibility and now returns only the Console-resolved marker name. Use audioMarkerIdentity for both id and name. The legacy SORIRecognitionEventType.audioMarkerChanged value remains parseable, but current bridges emit audioMarkerFound; a legacy String payload is ignored so an internal codebook label cannot become public. SORIActivityTrait and SORICampaign.trait remain available for source compatibility with manually constructed or older application-owned models, but current SDK events do not populate them with marker evidence. Location, metadata provider, raw audio buffers, and automatic action URL opening are not part of the public Flutter API.

Current marker state and accepted activity results

On Android and iOS, audiomarker: true enables SDK-owned current marker state. audioMarkerStateChanged carries event.audioMarkerState: inactive, listening, detected (no public identity yet), identified, or cleared. Only identified state contains a Console marker. Fresh evidence bridges the next material cycle; three approximately one-second misses retain it and the fourth clears it. Native expiry also runs without callbacks or during silence. Stop/reset and invalid capture clear state. Reading a snapshot, receiving a material result or waiting for a server response never renews evidence.

activityResult carries event.activityResult, including the exact server activityId, optional campaign and that activity's Console-resolved marker. A result may have no campaign. A late marker can upgrade the existing activity without another successful material match. Current-state expiry does not remove historical result markers. The native SDK owns trait selection and POST/PUT; applications do not construct traits or issue activity updates.

The package provides a small presentation reducer for ordered snapshots and exact-ID result updates:

final state = SORIRecognitionState();
final subscription = recognizer.events.listen((event) {
  if (state.accept(event)) {
    // Render state.currentAudioMarker and state.activities.
    // Results retain insertion order; enrichment updates the same activity.
  }
});

// Optional, for an explicit read. Native event reattachment also replays state.
final snapshot = await recognizer.getCurrentAudioMarkerState();
if (snapshot != null) state.acceptAudioMarkerState(snapshot);

The reducer only applies SDK values, ignores old channel deliveries and prevents a delayed initial duplicate from removing an accepted result marker. It does not infer eligibility or run an expiry timer. Native generation and revision are preserved; sequence orders bridge delivery/snapshot reads within one plugin instance. Optional Android evidence timestamps use native monotonic time, not Unix time; they are diagnostic values, not a Dart timer contract.

Legacy campaignFound, recognitionResult and resolved discovery callbacks remain compatible. event.audioMarkerIdentity retains discovery semantics; read event.activityResult?.audioMarkerIdentity for an activity's identity. Existing result payloads with a valid activity ID also gain the typed result. A material or campaign ID is never substituted for a missing activity ID.

Tested platform boundary

Capability	Android 5.1.1	iOS 5.1.1	Web 0.6.10
Resolved Console discovery	Yes	Yes	Existing support
Native current state / snapshot / explicit clear	Yes	Yes	Not available; snapshot returns null
State-driven initial traits and independent late activity upgrade	Native implementation	Native implementation	Not claimed
Existing exact-ID result/refinement decoding	Yes	Yes	Preserved and tested
Physical acoustic / real Console field acceptance	Pending	Pending	Not established by this change

The npm registry and published @sorisdk/web-audio 0.6.10 declarations were checked on 2026-09-14: they expose resolved audiomarker and campaign events, but no current-state/snapshot API. Bridge 0.1.5 retains ABI 2 and supports compatible patches through ^0.6.10. Web needs an upstream state/expiry/independent-upgrade implementation and validation before parity can be claimed; this gap is recorded under Flutter #27. Web scheduling #153 is related work, not evidence of the missing state contract. No Dart Web expiry or PUT fallback is supplied.

See the example/ app for a complete recognition flow that reads mobile credentials or a Web ephemeral endpoint from --dart-define values and renders recognized campaigns.