tugboat 0.10.0
tugboat: ^0.10.0 copied to clipboard
Screenshot-based session replay with compact interaction anchors for Tugboat.
0.10.0 #
Added #
- Device Farm launch inputs owned by the SDK:
TugboatLaunchOptionsreads Android Intent extras (tugboat_emit_scene_inventory,tugboat_accept_action_context,tugboat_collector_base_url) and iOS process environment (TUGBOAT_EMIT_SCENE_INVENTORY,TUGBOAT_ACCEPT_ACTION_CONTEXT,TUGBOAT_COLLECTOR_BASE_URL) through thetugboat/launchplugin channel, so host apps need no native code. TugboatReplayConfig.withDeviceFarmOverrides(): additive merge of launch capabilities plus a release-guarded, local-only collector URL override (resolveTugboatCollectorBaseUrl). Hosts collapse to a single call.
Fixed #
- Coalesce the back-to-back
hidden+pausedlifecycle callbacks into a singleapp_backgroundedevent instead of emitting one per state.
0.9.0 #
Changed #
- Replace
TugboatCaptureProfilewith one privacy-safe capture policy. - Use
TugboatReplayConfig.enabledfor lifecycle configuration. - Add explicit, additive capabilities for scene inventory, viewport semantic maps, capture diagnostics, and external action context.
- Keep
allTextAndMediaas the default screenshot mask when capabilities are enabled. - Make
TugboatParameterPolicy.allowAllmode-free and remove the duplicate production-specific parameter policy. - Remove capture-profile fields from SDK health and sink session context.
- Rename the controller action-window API to
setActionContextandclearActionContext.
Migration #
- Replace
profile: TugboatCaptureProfile.dormantwithenabled: false. - Replace
profile: TugboatCaptureProfile.productionLeanwithenabled: true. - Replace exploration mode with
enabled: trueplus only the required additive capabilities. - Remove the
profileargument fromTugboatReplay.activate.
0.8.17 #
Fixed #
- Reject blank iOS engine-surface captures and retry with explicit hierarchy capture before falling back to Flutter capture.
- Resolve the live
FlutterViewwithout selecting an unrelated controller view.
Changed #
- Raise the default degraded capture scale from
0.67to0.80to retain more screenshot detail when the capture budget is degraded. - Depend on CocoaPods
TugboatCaptureRuntime0.1.1.
0.8.16 #
Fixed #
- The iOS plugin compiles on Flutter 3.35.
registrar.viewControlleris a Flutter 3.38 API, so look it up at runtime and fall back to the key window when resolving the live Flutter view for opt-in native CPU capture.
0.8.15 #
The iOS plugin depends on CocoaPods TugboatCaptureRuntime 0.1.0 and
raises the plugin floor to iOS 15. Native CPU capture stays opt-in.
0.8.14 #
The Android plugin depends on Maven Central
com.gettugboat.sdk:capture-runtime:0.1.0 instead of compiling
platforms/android from source or stubbing native capture in published
packages. Native CPU capture stays opt-in.
0.8.13 #
Patch release after published 0.8.12.
Fixed #
- The Android plugin compiles
capture-runtimefrom monorepo sources instead of requiring unpublished Mavencom.gettugboat.sdk:capture-runtime:0.1.0. - The iOS plugin no longer depends on unpublished CocoaPod
TugboatCaptureRuntimeand keeps a deployment target of iOS 12. Native capture remains runtime-gated to iOS 15. - Align Android/iOS plugin packaging versions with Dart
0.8.13. - Bind iOS native capture to the registrar's Flutter view instead of the key window.
Added #
- Convert
tugboatinto a Flutter plugin with an experimental opt-in native CPU capture backend (nativeCpuExperimental) on Android (PixelCopy) and iOS (live Flutter-layer rendering). The default remainsflutterRepaintBoundary, with automatic fallback when native capture is unavailable or fails safely. - Record a regenerable JPEG size envelope for the current Dart encoder so the native CPU capture path can compare codec output against the same buffers.
- Native
encodeMicrosis the Pigeon capture round-trip only. Nested native stage timings stay on the backend trace soTugboatFrame.captureMicrosdoes not double-count PixelCopy, dHash, JPEG, and SHA-256. - Published
TugboatFramerecords includerequestedBackend,resolvedBackend, andfallbackReasonso production screenshots identify Flutter vs native withoutcapture_diagnosticevents. HTTP/v1/framesis still JPEG-only. route_changenow splits route identity (routeName,routeType,routeNamed, and the matchingfrom*fields) without changingroute,fromRoute, ornavigation.- Overlay classification is the closed set
page | sheet | dialog | popup | unknown. Bottom sheets aresheet(no longermodal). - Overlay pushes include presentation parent (
presentedOver*,hostPage*) and a bounded same-navigatorrouteStacksnapshot. - Claimed route changes copy
causeTargetFingerprintandcauseGesturefrom the interaction. Overlay after-frames still capture under exploration screenshot suppression whenoverlayKindis notpage. - Raise the example Android wrapper to Gradle 8.14 / AGP 8.11.1 / Kotlin 2.2.20 so Flutter 3.47 can build a release APK for device-lab capture gates.
- Document the mobile monorepo, native capture contracts, fallback, coverage, and path-aware version / license / API-surface checks. Native capture stays experimental.
0.8.12 #
Patch release after 0.8.11.
Fixed #
- Stamp the current runtime
userIdand traits snapshot on every session lifecycle POST (session_start,session_identify,traits_updated,user_changed,session_end). Slim payloads had omitted identity from traits-only updates and omitted both fields fromsession_end.
0.8.11 #
Patch release after 0.8.10.
Changed #
- Refactor capture, replay, semantic, model, and network paths to keep each function at or below a cyclomatic complexity of 10.
- Add a workspace complexity check that fails on warnings.
- Preserve locale-independent fingerprints and the 0.8.10 capture reliability behavior during the refactor.
0.8.10 #
Patch release after 0.8.9.
Fixed #
- Flush a newly enabled Flutter semantics tree only when no frame is pending. During dirty layout or paint, read the last stable tree and use scene inventory fallback instead of forcing an unsafe semantics flush.
- Replace fixed screenshot waits in release-gate widget tests with a bounded controller-idle wait. The full SDK suite now covers real screenshot encoding, masking, interaction action windows, and production-masked modal ownership.
- Align the workspace lockfile with the checked-in Flutter 3.35.7 and Dart 3.9.2 toolchain.
Compatibility #
- Document and verify that collector mapping preserves schema-v2
pan,zoom_in, andzoom_outfacts, includingpointerCountand zoomscale.
0.8.9 #
Patch release after 0.8.8.
Fixed #
- Keep a classified touch gesture open until all contacts lift. Replacement fingers join the same gesture, and zoom scale continues across contact changes.
- Detect pinches that start with contacts closer than the touch slop.
- Keep stationary third-finger taps until movement confirms a shared gesture.
- Include trackpad pan travel in
endPositionanddelta. - Continue touch travel after the primary finger lifts. Later contact movement
updates
endPositionanddeltawithout jumps when fingers join or lift. - Clear input contacts on pause, hide, and detach, even when Flutter does not deliver pointer cancellation. A later swipe starts a new gesture.
One-finger canvas pan remains swipe. Observed Flutter scrolling remains
scroll. This patch does not change the event or fingerprint schemas.
Touch scale describes contact-span ratios, not the host widget's transform.
0.8.8 #
This release follows 0.8.0 and stays on the 0.8.x line as 0.8.8.
Added #
- Exploration capture now snapshots fresh target, inventory, and viewport semantic evidence at primary pointer-down. It records bounded diagnostics and a closed failure reason when no safe target fingerprint is available.
- Actionable semantic nodes without a direct fingerprint link can use a guarded scene-inventory fallback. Fallback fingerprints have low confidence.
- Capture sessions and events now carry the active app locale. The SDK observes
Localizations, emitslocale_changedevidence, includes locale in exploration sessions, and supportsTugboatReplay.setLocale(...).
Changed #
- A completed exploration tap reuses its pointer-down evidence. Pan, zoom, scroll, cancellation, secondary pointers, production capture, and dormant capture do not run or publish this tap-only path.
- Fingerprint schema 6 now uses structural list and grid positions instead of visible text or icon data. Nested Flutter hit targets normalize to the same primary inventory control in exploration and production capture.
0.8.7 #
This release follows 0.8.0 and stays on the 0.8.x line as 0.8.7.
Added #
TugboatCollectorHost.fromPlatform()now snapshots optional session-start device facts onsession_start:batteryPercent,storageFreeMb(free internal storage),ramMb(Android physical RAM viadevice_info_plus), andnetworkType(wifi,cellular,ethernet,vpn,none, orother). Each field is omitted independently when the platform cannot observe it.- Three-finger shared translation now publishes one canonical
swipeinteraction withpayload.pointerCount: 3. Two-finger pan/zoom and one-finger swipe behavior is unchanged.
Changed #
- Bumped
device_info_plusto^11.5.0so Android RAM can be observed for the session-start snapshot.
0.8.6 #
This release follows 0.8.0 and stays on the 0.8.x line as 0.8.6.
Fixed #
- Default
TugboatReplay.eventHookparameter policy now retains bounded JSON-safe external-event values (allowAllInProduction). Parameter keys were already captured; values were omitted unless the host opted in. Hosts that must not retain values can still passnamesOnly, an allow-list, or a transform. Existing JSON and size bounds still apply.
Added #
- Canonical interactions now include
pan,zoom_in, andzoom_out. Two-pointer pinch and translation, plus trackpad pan/zoom events, publish one gesture. Nested payload carriesposition/endPosition/delta,pointerCountwhen above one, andscalefor zoom. Two-finger list scrolling still becomesscroll. Stationary two-finger contact remains two taps.
Changed #
productionLeancapture profiles no longer emitcapture_diagnosticsession or collector events. BoundedhealthSnapshot().captureDiagnosticscounters still update on device. Exploration profiles continue to emit full diagnostic events.
0.8.5 #
This release follows 0.8.0 and stays on the 0.8.x line as 0.8.5.
Breaking changes #
- The SDK now publishes only schema-v2 canonical
interactiongesture events. Removed legacy gesture projections, publication modes, session aliases, and compatibility constructors. - Removed the on-device outbox and its public configuration. Collector and exploration delivery are best-effort through bounded in-memory queues.
interaction.afterFramenow means a temporal post-interaction observation. An unclaimed route frame can satisfy it without creating route causality or an inferred result.
Changed #
- Production-friendly scroll capture now keeps scroll metrics independent from screenshots. In-motion scroll screenshots are opt-in, pressure-droppable, and disabled by default; pointer-linked scrolls retain one optionally deferred scroll-end observation.
- Screenshot output can be bounded with
captureMaxWidth/captureMaxHeight. A degraded screenshot budget appliesdegradedCaptureScalebefore GPU readback, reducing raw RGBA allocation as well as encoded size. - Low-priority captures dropped under active/queued capture pressure emit the
bounded
capture_pressure_dropdiagnostic outcome. capturePixelRatioaccepts values above1.0. Optional output bounds still cap the effective readback scale before allocation.- Network observation accepts bounded absolute paths with dynamic identifier segments. Schemes, queries, fragments, encoded data, network-path prefixes, backslashes, whitespace, and control characters remain rejected.
Fixed #
- A new pointer-down cancels a pending deferred scroll-end screenshot before
Flutter reports the next
ScrollStart. The completed scroll keeps its final offsets, overscroll count, target fingerprint, and interaction record, but it does not block or consume the new gesture. - Tap-only target, inventory, and viewport-semantic resolution now runs after a gesture remains a tap. Scroll gestures no longer pay that work on pointer-down.
- Scroll completion joins pointer-up and
ScrollEndNotificationin either callback order and publishes one canonical interaction. Pointer cancellation, replacement scrolls, lifecycle changes, and programmatic scrolls do not leave a late interaction screenshot request. - Interaction after-frames must complete after the interaction boundary. A pre-pointer-up frame cannot become an after-frame, including frames from a claimed route capture that settles while the pointer is still down. An unrelated automatic route cannot add causal route evidence to the interaction.
- Tap target resolution uses the pointer-down position so a slop-bounded release cannot resolve a sibling control or miss the recognizer's original target.
- Screenshot capture pixel-ratio clamp results are explicitly converted to
doublefor sound null-safety.
0.8.0 #
Changed #
- Raw SDK writers no longer emit
stateAnchor,stateSignature, orstate_changeevents. Session wire schema 10 identifies this contract and the serializedinteractionframe trigger. Completed interactions request one fresh after-frame. - Collector event mapping now omits
stateAnchor. Deploy the serial collector compatibility patch before sending 0.8.0 recordings to a collector that still requires that key. - Production collector events for
interactionandroute_changeuse flat schema-v2 wire shapes (interactionSchemaorrouteChangeSchema==2) with facts-only fields. The mapper no longer emits emptytargetAnchorobjects or duplicatesstreaminside genericpayload. Interaction v2 drops inferredresult, nestedorigin/result, and tap-settle outcome computation. interactionschema v2 carries gesture-specific facts under a nestedpayload(positionfor tap;position/endPosition/deltafor swipe, pan, and zoom;scale/pointerCountfor zoom and two-pointer pan;position/startOffset/endOffset/overscrollCountfor scroll). Cancelled interactions omitpayload. The SDK no longer emitsscroll_start,scroll_end, orpointer_cancel— scroll and cancel semantics live oninteractiononly.
0.7.1 #
Changed #
- Screenshot capture performance — remove the post-capture state-signature
short circuit and replace it with a paint-signature gate that skips the full
GPU readback/encode path when the capture subtree (including nested
RepaintBoundarys) has not painted. Diagnostic outcomestate_signature_short_circuitis replaced bypaint_generation_unchanged. - Encode path — JPEG encoding, SHA-256, mask fills, and dHash now run on a persistent background isolate with transferable RGBA input. dHash coalesce tolerates Hamming distance ≤ 2. Default screenshot budget is 60 ms / 5 s.
- Collector uploads — frame wire format docs corrected to JPEG. Pending frames are not superseded on enqueue: events reference exact frame IDs and multipart upload has no hash alias.
0.7.0 #
Added #
- Explicit production parameter opt-in —
TugboatParameterPolicy.allowAllInProductionretains JSON-safe external-event parameter values in production capture profiles.namesOnlyremains the default.allowAllremains an exploration-only escape hatch and still downgrades to names-only outside exploration. The existing JSON and size bounds still apply. This policy can retain feedback, search terms, URLs, IDs, and other user content. Hosts must confirm consent, privacy, access, and retention rules before they use it.
0.6.0 #
Changed #
- Canonical interactions are now the recording default —
TugboatReplayConfig.interactionPublishModedefaults tocanonicalOnly, so each finalized gesture emits one semanticinteractioninstead of also emittingtap,tap_settled, orswipecompatibility rows. - Legacy gesture publication is deprecated —
dualWriteandlegacyOnlyremain explicit migration overrides for historical consumers. New integrations must not enable them; removal prerequisites and the searchableTODO(tugboat-legacy-projection-removal)marker are documented in the SDK README.
Added #
- Provider-neutral coded-event hook —
TugboatReplay.eventHookrecords one logicalexternal_eventon the evidence stream with a bounded parameter policy (namesOnly,allowList,transform, or exploration-onlyallowAll). Values are deep-copied at hook time; dormant/disabled calls are safe no-ops. - Generic network observation —
TugboatReplay.beginNetworkCallexposes an exactly-once token for method, safe route template, status, outcome, and duration. No headers, queries, bodies, raw errors, or stack traces are retained. - Evidence isolation — external and network evidence stamp session identity
only and never inherit active exploration
actionId,relatedEventId, or target/state anchors. - Evidence health counters —
TugboatSdkHealth.evidenceexposes bounded accepted/dropped/duplicate-finish counts without retaining rejected raw values. tugboat_diocompanion package — Dio interceptor that maps request lifecycle callbacks onto the core network token without importing Dio into core.
Fixed #
- Session-bound evidence completion — in-flight network tokens can no longer finish into a replacement session, and session end fences reentrant evidence before publishing its terminal event.
- Production parameter policy — exploration-only
allowAllis downgraded to names-only outside exploration, and unsupported values contribute one drop to bounded diagnostics. - Session-end admission — session end claims its in-flight future before sync sink work, so evidence fencing no longer needs a separate ending bool.
- Deactivate evidence fence —
TugboatReplay.deactivatecloses evidence admission immediately; the activation gate still ownssession_endon teardown.
0.5.3 #
Added #
- Debounced identity coalesce — after
session_start,setUserIdandsetTraitswithin 3s consolidate into onesession_identifyPOST when both change; otherwiseuser_changedortraits_updated. Pending updates are flushed beforesession_end.
Changed #
- Pre-start identity still folds into a single
session_startwhen values are staged before or while start is pending.
0.5.2 #
Changed #
setUserIdfolds into pendingsession_start— when asession_startis still pending,CollectorHttpSink.setUserIdupdates the runtime id only and skipsuser_changed(same coalesce already used bysetTraits). Boot identity can land on a singlesession_startPOST.
0.5.1 #
Added #
- User traits via collector sessions —
TugboatReplay.setTraitspostseventType: traits_updatedonPOST /v1/sessionswith a full traits bag, caches the responsetraitsId, and stamps it on event batches.TugboatReplay.setUserIdpostsuser_changedand updates the runtime user id. Pre-set traits are included on the nextsession_start. No/v1/identifyroute.
Changed #
setUserIdskips unchanged ids — callingTugboatReplay.setUserId/CollectorHttpSink.setUserIdwith the same value as the current runtime user id does not postuser_changed.- Pre-initialize identify —
setTraits/setUserIdcalled after the controller mounts but before the HTTP sink is created retain identity for the nextsession_startinstead of dropping it.
0.5.0 #
Breaking changes #
- Control-value and semantic-annotation telemetry — session JSON writers
now emit schema version 9 and no longer write
controlValue,controlValueTransition, orsemanticAnnotationin eventdata. Readers that support historic schemas should continue to tolerate versions 6–8, where those fields may be present. - Removed public barrel exports —
TugboatEncodedControlScalar,TugboatVisibleControlValue,TugboatControlValueScope,TugboatControlValue, andTugboatSemanticAnnotation. - Removed public schema constants —
tugboatControlValueSchemaVersion,tugboatControlValueTransitionSchemaVersion, andtugboatSemanticAnnotationSchemaVersion. - Removed public extraction and merge helpers —
tugboatControlValueForWidget,tugboatControlValueFromSemanticsProperties,tugboatControlValueFromSemanticsNode,tugboatSemanticAnnotationFromProperties,tugboatSemanticAnnotationFromNode,tugboatMergeSemanticAnnotations, andtugboatMergeControlValues.
0.4.18 #
Fixed #
- Semantic parameter pairs in overlays — interactions in dialogs and popovers now retain the accessibility label and raw value supplied by the visible control, rather than metadata from an obscured control beneath it.
0.4.17 #
Changed #
- Raw control and semantic values — control values, semantic values, and semantic labels are now sent verbatim instead of being tokenized. This makes slider positions, durations, and template identifiers available for session summaries and aggregate analysis.
- Explicit custom-control values —
TugboatControlValueScopeexposes a stablecontrolKey, typed number/duration/enum value, optional unit, and numeric range metadata for controls whose value is not readable from a standard Flutter widget.
0.4.16 #
Added #
- Privacy-safe interaction metadata — valued controls and semantic annotations can enrich tap, settle, swipe, and scroll events without retaining arbitrary semantic text.
Changed #
- Causal control-value transitions — settled control values are captured from the original interaction target and use a distinct transition payload.
- Canonical interaction parity — canonical-only tap and swipe results retain their post-interaction control and semantic metadata without relying on legacy projection events.
- Cross-SDK semantics flags — checked-state capture compiles on the package's declared Flutter 3.35 minimum and newer enum-based SDKs.
0.4.15 #
Added #
- Canonical
interactionevents — each finalized gesture emits onestream: semanticrecord with immutableorigin,result,attribution, andevidenceEventIds(interactionSchema: 1). Legacytap/tap_settled/swipecontinue as dual-write peers onstream: legacy_projection. - Evidence stream —
route_change,state_change,scroll_start,scroll_end, andpointer_cancelemit onstream: evidenceso default semantic enrichment selects only canonical interactions. enrichmentCandidateon collector-mapped events — false for evidence, diagnostic, and legacy-projection records; true for canonicalinteraction(and compat semantic tap/tap_settled/swipe when canonical emission is off).- Delayed reconciliation window —
interactionClaimWindowdefaults to 1,250 ms. A released tap can claim the first eligible visible route/modal successor in that window (interactionAttribution: delayed_likely). Set the window toDuration.zeroto retain microtask-only same-turn claims. - Diagnostic stream isolation —
capture_diagnosticevents carrystream: diagnosticso enrichment/insight queries can ignore them by default. Session health still aggregates outcome counts. causedByInteractionIdon claimedroute_change/state_change(alongside existingcauseEventId).
Fixed #
- Swipe origin freeze — swipe events retain the pointer-down state anchor rather than refreshing live controller state at pointer-up.
- Settle waits for delayed successors — when the claim window is active,
tap settlement holds until a successor claims or the deadline expires instead
of finalizing
unknownimmediately. - Terminal cancelled interactions — abandoning a pending/released
transaction (lifecycle, session end, supersede, post-up cancel) publishes a
canonical
gesture=cancelledinteraction instead of silently dropping it.
0.4.14 #
Fixed #
- Automatic-navigation visual continuity — when a route transition
supersedes an in-flight tap capture,
tap_settlednow waits for and attaches the route's fresh frame as a non-causalvisual_successor. A pointer-generation fence prevents later user interactions from being attached to the earlier tap.
0.4.13 #
Fixed #
- Deferred tap emission —
tapis sampled at pointer-down but only emitted after gesture classification at pointer-up. Flick-scrolls no longer mint phantom taps; swipes carrystartCaptureCoordinateinstead ofrelatedEventIdto a never-settled tap. - Same-turn interaction claims — released pointer-up claims attribute
route_changeonly through the pointer-up turn (same_turn). A wall-clock claim window incorrectly bound automatic redirects to taps;interactionClaimWindowdefaults to zero and no longer extends attribution. Pre-up claims still work while the pointer is down (long-press → navigate). - Pre-up claim + swipe/cancel — routes that claim before pointer-up no longer
force a normal interaction tap; the buffered tap publishes as
causal_onlywhen the gesture finalizes as swipe/cancel (or atroute_changepublish). - Lifecycle claim fence — backgrounding drops pending/released pointer claims so resume/navigation cannot attribute a pre-background gesture.
- Session-end claim fence — pending pointers are abandoned without emitting
orphan
causal_onlytaps when the claimed route was cancelled before publish; further pointer-down/up/cancel is ignored. - Duplicate pointer-down — a second down on the same pointer abandons the prior pending claim instead of silently overwriting it.
- Claim map hygiene —
_claimsByTapEventIdentries are removed when a claim emits, drops, or expires;invalidatesRelatedTapis always a boolean. - Monotonic deferred publish — buffered taps /
tap_outside_treepublish at emissionatMswithsampledAtMspreserving pointer-down time. - Gesture promotion —
tap_gesture_resolved(promotesRelatedTap) plus an in-memoryreplayRolepatch promote genuine taps that were first published ascausal_onlyfor a pre-up route claim. - Missing-frame coordinates —
unavailableReason: missing_framenow keeps boundary-local / normalized geometry when the boundary rect is known, instead of zeroing local/normalized fields.
0.4.12 #
Added #
- Navigator and route-instance ownership — every observed Navigator gets a
session-local opaque
navigatorId; every pushed route gets arouteInstanceId. Stacked anonymous modals stay distinguishable. Install nested observers withTugboatReplay.createNavigatorObserver(). - Navigation origin contract —
route_changeevents carrynavigationOrigin(interaction|automatic_or_unknown) and optionalcauseEventId. Only observer-time single-use pending-interaction claims can bind a tap; timer/auth redirects never fabricate causality. - Versioned
captureCoordinate— taps retain legacy globalx/yand add a boundary-local / normalized / raster transform bound to the before-frame. Outside-boundary and generation-mismatch cases emit an unavailable reason instead of clamping. - Replay coherence characterization harness — deterministic, advanceable
scheduler/capture test seams (
debugNow,debugDelay,debugExecuteCapture,debugSeedFrame) plus reusable helpers that reproduce known navigation/frame races without wall-clock sleeps. Tracks milestone issue #5. - Pathless-tap snap (2026-07-04) — when hit-testing resolves a tap to a target
with a role but no canonical path (opaque
Texture, decorated boxes outside the token map), the tap is re-anchored to the smallest interactive scene-inventory element containing the tap point, so its fingerprint always joins the inventory. Guards: only interactive-tier entries qualify, and the candidate's area must be comparable to the render surface the pointer actually hit (occluded controls under opaque overlays are never falsely attributed). Snapped anchors keeptagFingerprint, drop stalefingerprintParts, and are markedfingerprintConfidence: 'low'. - Structurally addressable anchor preference —
_targetAtWithTokenMapprefers hit-test candidates that have both a role and a non-empty canonical path over role-only candidates, fixing joins for paywall dismiss buttons and overlay chrome. scene_inventoryevents — during exploration, emit a deduped structural inventory of actionable elements andImagewidgets per settled screen state (fingerprints, bounds, roles). Fingerprints match taptargetAnchorresolution for the same element.- Fingerprint aliases — inventory entries store alternate structural fingerprints for the
same logical control (e.g. wrapper layers like
InkWellvsGestureDetector). Tap resolution can match via primary fingerprint or alias. - Tap injection — when a tap resolves to a fingerprint not in the enumerated inventory, the tap target is appended to the inventory so exploration joins are guaranteed.
- Route epoch guard — stale delayed route callbacks no longer clobber
_currentRouteor emit incorrectroute_changeevents during rapid navigation. - Item-normalized state signatures —
[item:hash]segments in actionable paths are normalized to[item]for signature hashing only, so list scroll does not fork inventories for the same logical screen. - Capture attribution diagnostics — SDK emits
action_window_set,action_window_cleared,tap_outside_tree, andpointer_cancelevents during CLI exploration runs. recordPointerCancel— wired fromInputCaptureand the rootListenerso cancelled gestures are visible in the event stream.
Changed #
- Tap/navigation causality fence — tap settlement joins a route-capture barrier only when that route explicitly claimed the same tap event. Automatic redirects that overlap settlement, including successors of a tap-caused route, remain independent and cannot donate their frame or route event ID to the tap.
- Frame-owned coordinate geometry — frame provenance now retains the
capture boundary's logical rect and transform generation. Resizes, rotations,
and inset changes invalidate stale before-frame attachment and emit
generation_mismatchinstead of projecting a current tap onto older pixels. - Replay capture contract documentation — documents the implemented route/frame attribution invariant separately from the still-open production acceptance gaps for rapid modal chains, automatic navigation, and playback tap-coordinate alignment.
- Route-capture barrier — tap settlement now joins the matching route epoch instead of reusing the previous route's latest frame. Supersession, lifecycle cancellation, capture failure, and bounded timeout outcomes complete deterministically without allowing late readbacks to publish.
- Production viewport semantics stay local — production sessions can still
build viewport semantic maps for tap resolution, but
viewport_semantic_mapandscroll_semantic_snapshotevents are emitted only during exploration so UI text from semantic nodes is not uploaded in lean production captures. ExplorationCaptureSink.recordFrame— forwards frame metadata and PNG bytes over the exploration WebSocket instead of dropping them. The CLI persists these underframes/when Flutter capture produces them (local capture may still be suppressed after the socket connects for performance; seecollector-integration.md).actionableSummarydeduplication — count only leaf canonical controls (one perFilledButton/TextButton, not nested ink-well chrome). Fixes over-counting in modal, visibility, and rebuild fingerprint tests.nonAssetImagesOnlycontract narrowed to what is actually classified — README and enum docs now state the mode masks non-assetImagewidgets (such asImage.network,Image.file, andImage.memory) plus sensitive inputs. Custom-painted or decorated image surfaces (e.g.ExtendedImage,DecorationImage,CustomPaint) are not classified by this mode and must be wrapped inTugboatSensitivewhen private. Image-provenance detection is isolated in a single helper (_isNonAssetImageWidget) as the one place to extend classification later. Behavior is unchanged; regression tests cover asset-visible, memory-masked, and Sensitive-always-masked cases.- Typed route transitions in the controller —
route()converts raw navigator callback strings into an internal typed model (_RouteNavigationKind,_RouteTransition) and resolves visible navigation in one place (_resolveVisibleRouteChange), replacing string-driven branching and collapsing the two duplicateroute_removestack-cleanup checks into a single normalized decision. Theroute_changewire format (data.fromRoute/data.route/data.navigation) and epoch/capture semantics are unchanged; stack-cleanup removals frompushNamedAndRemoveUntilstill never bump the route epoch or cancel the pending destination capture (now covered by a dedicated regression test).
Removed #
control_inventorySDK events — the SDK no longer emits per-screen control inventories over the exploration WebSocket. Possible actions should be derived from screenshots (for example via a VLM) instead of SDK-side widget-tree scans. State anchors still includeactionableSummaryrole counts for fingerprinting only; that field is not a substitute for control discovery.
0.1.0 #
- Initial Tugboat Flutter SDK with screenshot evidence, interaction anchors, route transitions, privacy masking, and exploration transport.