validationMetadata property

  1. @override
DetectorMetadata get validationMetadata
override

Metadata for this detector. Must be non-null on shipping detectors — the audit test gates this.

This getter belongs to package:sleuth/validation.dart and is outside Sleuth's semver promise, also on main-API detectors that override it.

Implementation

@override
DetectorMetadata get validationMetadata => const DetectorMetadata(
  tier: EvidenceTier.reproducerOnly,
  reproducerPath: 'test/validation/frame_timing_reproducer_test.dart',
  coveredStableIds: {
    'sustained_jank',
    'jank_detected',
    'raster_cache_thrashing',
    'raster_cache_growing',
  },
  // `jank_detected` and `sustained_jank` are raised to runtimeVerified
  // via `perStableIdTier`; the cache families stay reproducerOnly.
  // `sustained_jank` counts severe frames in the 240-frame buffer, where
  // an ambient severe frame would join the operator's injected ones.
  // Its capture screen refuses any leg whose severe frames, judged by a
  // rule at least as strict as the detector's, are not exactly the
  // injected ones, so the bracket holds only legs without ambient
  // severe frames.
  perStableIdTier: {
    'jank_detected': EvidenceTier.runtimeVerified,
    'sustained_jank': EvidenceTier.runtimeVerified,
  },
  additionalBrackets: [
    // sustained_jank is critical at 3 or more severe frames, its only
    // severity. The at band [3, 4.5] holds 3 or 4, the above band
    // (4.5, 7.5] 5 to 7. `last` reads the count when the span closed,
    // so a frame excused later (`refreshOwnStartDelays`) lowers it
    // instead of leaving a stale peak.
    BracketSpec(
      stableId: 'sustained_jank',
      severityLabel: 'critical',
      threshold: 3,
      unit: 'frames',
      coveredThresholds: {'sustained_jank.critical'},
      profileCapturePaths: [
        'test/validation/captures/frame_timing/sustained_jank_below.json',
        'test/validation/captures/frame_timing/sustained_jank_at.json',
        'test/validation/captures/frame_timing/sustained_jank_above.json',
      ],
      atTolerance: 0.5,
      aboveCeilingMultiplier: 2.5,
      observedAxisArgKey: 'observedSevereCount',
      observedAxisTolerance: 0.25,
      observedAxisReduction: 'last',
      requireUniqueDetectedAtMicros: true,
      requireDetectorTraceRecord: true,
      // The count repeats in every timings batch after the last spin,
      // so an in-band leg holds many in-band records.
      minInBandSamples: 2,
    ),
  ],
  profileCapturePaths: [
    'test/validation/captures/frame_timing/jank_detected_below.json',
    'test/validation/captures/frame_timing/jank_detected_at.json',
    'test/validation/captures/frame_timing/jank_detected_above.json',
  ],
  bracketStableId: 'jank_detected',
  bracketSeverityLabel: 'warning',
  bracketThreshold: 16,
  bracketUnit: 'percent',
  bracketAtTolerance: 0.50,
  aboveCeilingMultiplier: 1.85,
  coveredThresholds: {'jank_detected.warning', 'sustained_jank.critical'},
  observedAxisArgKey: 'observedJankPercent',
  observedAxisTolerance: 0.25,
  observedAxisReduction: 'last',
  bracketRequireUniqueDetectedAtMicros: true,
  rationale:
      'A hermetic reproducer pins four stableIds. `sustained_jank` fires '
      'on at least 3 severe frames in a 240-frame window. `jank_detected` '
      'fires above 15% jank frames with a sample of at least 5 frames. '
      'Both judge only the frames since the last route epoch and, '
      'outside capture mode, only those within 10 s of the newest '
      'frame\'s vsync, so a card clears about 10 s after the jank stops '
      'on a screen that draws only Sleuth\'s own scan frames. '
      '`raster_cache_thrashing` fires after at least 15 consecutive '
      'frames in which the picture cache count changes by more than 20% '
      'from a previous frame that held more than 5 pictures, or jumps '
      'from 5 or fewer to more than 5. `raster_cache_growing` '
      'fires after at least 30 consecutive frames of monotonic growth '
      'in total raster cache bytes. The reproducer bypasses warmup with '
      '`warmupDuration: Duration.zero`. Every stableId has a synthetic '
      '`FrameStats` path and a real `FrameTiming` integration leg through '
      '`handleTimingsForTest`, so hand-written synthetic fixtures cannot '
      'encode the detector\'s own expected shape. A dedicated test pins '
      'Impeller-zero suppression (all four cache metrics zero for at '
      'least 30 frames). Every other cache-family test sets '
      '`pictureCacheBytes: 1` so that suppression cannot arm and clear '
      'the issues under test. `jank_detected.warning` is runtimeVerified '
      '(via `perStableIdTier`) on three on-device captures (iPhone 12, '
      'iOS 17.5, Flutter 3.47.6, 60 Hz, deadline rule) that bracket the '
      'jank-percent threshold. `sustained_jank.critical` is '
      'runtimeVerified on three iPhone 12 captures (iOS 17.5, Flutter '
      '3.47.6, 60 Hz) whose legs inject 2, 3 and 6 severe frames. The '
      'capture screen spins 40 ms inside chosen frames, counts severe '
      'frames with its own FrameTiming rule (which keeps Sleuth\'s start '
      'delay in, so it never counts fewer than the detector), and keeps a '
      'leg only when its count equals the injected count and, for the at '
      'and above legs, the detector\'s last `observedSevereCount` does '
      'too. A leg with an '
      'ambient severe frame is refused, not subtracted, and the claim '
      'covers the iPhone 12. The cache stableIds stay reproducerOnly and '
      'are not cited to Flutter engine sources. v0.17.0 extends FPS '
      'semantics. `actualFps` (a rolling count in a 1 s window anchored '
      'on the latest `rasterFinishUs`) and `throughputFps` '
      '(latency-derived capacity) are separate metrics. StableId coverage '
      'is unchanged, because FPS semantics are independent of jank '
      'classification. v0.19.6 adds the capture-mode plumbing for the '
      '`jank_detected.warning` `runtimeVerified` raise. '
      '`SleuthConfig(captureMode: true)` skips the 3 s warmup gate. Each '
      'emission carries an instance-monotonic `dedupIdentityMicros` '
      '(wall-clock micros plus an `_emissionSeq` tie-breaker, kept across '
      '`reset()` so multi-leg capture flows cannot collide identities). '
      'The detector exports its observed axis values '
      '(`observedJankCount`, `observedJankPercent`, '
      '`observedWorstFrameMs`, `bufferSize`) into trace event args, so '
      'the audit gate cross-checks the operator claim against the '
      'detector observation (`observedJankPercent`, last value in the '
      'scenario).',
);