validationMetadata property
Metadata for this detector. Must be non-null on shipping detectors — the audit test gates this.
This getter belongs to package:sleuth/validation.dart and is outside
Sleuth's semver promise, also on main-API detectors that override it.
Implementation
@override
DetectorMetadata get validationMetadata => const DetectorMetadata(
tier: EvidenceTier.reproducerOnly,
reproducerPath: 'test/validation/frame_timing_reproducer_test.dart',
coveredStableIds: {
'sustained_jank',
'jank_detected',
'raster_cache_thrashing',
'raster_cache_growing',
},
// `jank_detected` and `sustained_jank` are raised to runtimeVerified
// via `perStableIdTier`; the cache families stay reproducerOnly.
// `sustained_jank` counts severe frames in the 240-frame buffer, where
// an ambient severe frame would join the operator's injected ones.
// Its capture screen refuses any leg whose severe frames, judged by a
// rule at least as strict as the detector's, are not exactly the
// injected ones, so the bracket holds only legs without ambient
// severe frames.
perStableIdTier: {
'jank_detected': EvidenceTier.runtimeVerified,
'sustained_jank': EvidenceTier.runtimeVerified,
},
additionalBrackets: [
// sustained_jank is critical at 3 or more severe frames, its only
// severity. The at band [3, 4.5] holds 3 or 4, the above band
// (4.5, 7.5] 5 to 7. `last` reads the count when the span closed,
// so a frame excused later (`refreshOwnStartDelays`) lowers it
// instead of leaving a stale peak.
BracketSpec(
stableId: 'sustained_jank',
severityLabel: 'critical',
threshold: 3,
unit: 'frames',
coveredThresholds: {'sustained_jank.critical'},
profileCapturePaths: [
'test/validation/captures/frame_timing/sustained_jank_below.json',
'test/validation/captures/frame_timing/sustained_jank_at.json',
'test/validation/captures/frame_timing/sustained_jank_above.json',
],
atTolerance: 0.5,
aboveCeilingMultiplier: 2.5,
observedAxisArgKey: 'observedSevereCount',
observedAxisTolerance: 0.25,
observedAxisReduction: 'last',
requireUniqueDetectedAtMicros: true,
requireDetectorTraceRecord: true,
// The count repeats in every timings batch after the last spin,
// so an in-band leg holds many in-band records.
minInBandSamples: 2,
),
],
profileCapturePaths: [
'test/validation/captures/frame_timing/jank_detected_below.json',
'test/validation/captures/frame_timing/jank_detected_at.json',
'test/validation/captures/frame_timing/jank_detected_above.json',
],
bracketStableId: 'jank_detected',
bracketSeverityLabel: 'warning',
bracketThreshold: 16,
bracketUnit: 'percent',
bracketAtTolerance: 0.50,
aboveCeilingMultiplier: 1.85,
coveredThresholds: {'jank_detected.warning', 'sustained_jank.critical'},
observedAxisArgKey: 'observedJankPercent',
observedAxisTolerance: 0.25,
observedAxisReduction: 'last',
bracketRequireUniqueDetectedAtMicros: true,
rationale:
'A hermetic reproducer pins four stableIds. `sustained_jank` fires '
'on at least 3 severe frames in a 240-frame window. `jank_detected` '
'fires above 15% jank frames with a sample of at least 5 frames. '
'Both judge only the frames since the last route epoch and, '
'outside capture mode, only those within 10 s of the newest '
'frame\'s vsync, so a card clears about 10 s after the jank stops '
'on a screen that draws only Sleuth\'s own scan frames. '
'`raster_cache_thrashing` fires after at least 15 consecutive '
'frames in which the picture cache count changes by more than 20% '
'from a previous frame that held more than 5 pictures, or jumps '
'from 5 or fewer to more than 5. `raster_cache_growing` '
'fires after at least 30 consecutive frames of monotonic growth '
'in total raster cache bytes. The reproducer bypasses warmup with '
'`warmupDuration: Duration.zero`. Every stableId has a synthetic '
'`FrameStats` path and a real `FrameTiming` integration leg through '
'`handleTimingsForTest`, so hand-written synthetic fixtures cannot '
'encode the detector\'s own expected shape. A dedicated test pins '
'Impeller-zero suppression (all four cache metrics zero for at '
'least 30 frames). Every other cache-family test sets '
'`pictureCacheBytes: 1` so that suppression cannot arm and clear '
'the issues under test. `jank_detected.warning` is runtimeVerified '
'(via `perStableIdTier`) on three on-device captures (iPhone 12, '
'iOS 17.5, Flutter 3.47.6, 60 Hz, deadline rule) that bracket the '
'jank-percent threshold. `sustained_jank.critical` is '
'runtimeVerified on three iPhone 12 captures (iOS 17.5, Flutter '
'3.47.6, 60 Hz) whose legs inject 2, 3 and 6 severe frames. The '
'capture screen spins 40 ms inside chosen frames, counts severe '
'frames with its own FrameTiming rule (which keeps Sleuth\'s start '
'delay in, so it never counts fewer than the detector), and keeps a '
'leg only when its count equals the injected count and, for the at '
'and above legs, the detector\'s last `observedSevereCount` does '
'too. A leg with an '
'ambient severe frame is refused, not subtracted, and the claim '
'covers the iPhone 12. The cache stableIds stay reproducerOnly and '
'are not cited to Flutter engine sources. v0.17.0 extends FPS '
'semantics. `actualFps` (a rolling count in a 1 s window anchored '
'on the latest `rasterFinishUs`) and `throughputFps` '
'(latency-derived capacity) are separate metrics. StableId coverage '
'is unchanged, because FPS semantics are independent of jank '
'classification. v0.19.6 adds the capture-mode plumbing for the '
'`jank_detected.warning` `runtimeVerified` raise. '
'`SleuthConfig(captureMode: true)` skips the 3 s warmup gate. Each '
'emission carries an instance-monotonic `dedupIdentityMicros` '
'(wall-clock micros plus an `_emissionSeq` tie-breaker, kept across '
'`reset()` so multi-leg capture flows cannot collide identities). '
'The detector exports its observed axis values '
'(`observedJankCount`, `observedJankPercent`, '
'`observedWorstFrameMs`, `bufferSize`) into trace event args, so '
'the audit gate cross-checks the operator claim against the '
'detector observation (`observedJankPercent`, last value in the '
'scenario).',
);