Skip to main content

edge_vision_core 0.1.0 copy "edge_vision_core: ^0.1.0" to clipboard
edge_vision_core: ^0.1.0 copied to clipboard

Deterministic C++17 image & camera frame preprocessor, EXIF orienter, letterbox resizer, tensor normalizer, and YOLO postprocessor for Flutter Edge AI.

edge_vision_core #

pub package license

Deterministic C++17 image & live camera frame preprocessor, EXIF orienter, letterbox resizer, float32 tensor normalizer, and YOLO post-processor for Flutter Edge AI (Android, iOS, Windows, macOS, Linux).

Built around a lightweight, zero-external-dependency C++17 core with a pure extern "C" ABI (src/edge_vision_core.h) so mobile apps produce identical pixel math across iOS, Android, Desktop, and training pipelines.


Why edge_vision_core? #

Cross-Platform Challenge What Fails with OS Native APIs Solution in edge_vision_core
iOS vs. Android Accuracy Drift UIImage / CoreGraphics and Android Bitmap / Skia use different resize filters and color conversions. One shared C++17 resizing & normalization pipeline across every platform.
Broken EXIF Orientation Portrait photos rotated via EXIF tags (1..8) end up sideways or mirrored in inference tensors. Parses JPEG APP1 TIFF/IFD0 tags (0x0112) and physically rotates/flips pixels before resize.
YOLO Letterbox Coordinate Drift Unscaling 640x640 detections back to 1920x1080 requires tracking exact scale and padding offsets. Records TransformMeta (scale, padX, padY, oriented dims) and unscales detections in C++.
Slow Pure-Dart Decoding Pure-Dart image packages block or take seconds on 12MP–48MP photos. Native C++ decoding & packing (~2–5 ms resize/pack), with preprocessInIsolate support.
Heavy Binary Bloat Bundling OpenCV adds 15–30 MB to mobile app binaries. Lightweight header-only C++ core (stb_image, stb_image_resize, stb_image_write).

Architecture #

[ Flutter (dart:ffi) ]   [ React Native (JSI/Nitro) ]   [ Python / CMake ]
          \                           |                          /
           +--------------------------+-------------------------+
                                      |
                        [ C ABI: src/edge_vision_core.h ]
                                      |
                        [ Pure C++17 Vision Core (src/) ]
       1. Decode / YUV420->RGB  ->  2. EXIF 1..8 Orient  ->  3. Letterbox / Crop
       4. NCHW / NHWC Float32   ->  5. EXIF-free JPEG    ->  6. YOLO Postprocess + NMS

Features #

  • Encoded Image Decoding & EXIF Normalization: JPEG, PNG, BMP, GIF, TGA, PSD, HDR, PNM decoding with automatic JPEG EXIF orientation (1..8) physical correction.
  • Deterministic Resizing: ResizeMode.letterbox (standard YOLO 114 gray border), ResizeMode.stretch, and ResizeMode.centerCrop.
  • Single-Pass Tensor Packing: RGB / BGR, NCHW ([1, 3, H, W]) / NHWC ([1, H, W, 3]), custom scale, mean, stdDev, plus ready-to-use presets:
    • PreprocessConfig.yolo(size: 640)
    • PreprocessConfig.imageNet(width: 224, height: 224)
    • PreprocessConfig.clip(size: 224)
  • Live Camera YUV420 / NV12 / NV21 Support: preprocessYuv420(...) with deterministic BT.601 conversion and clockwise sensor rotation (0, 90, 180, 270).
  • EXIF-Free Cloud Upload Optimization: resizeAndEncodeJpeg(...) shrinks high-res photos deterministically and strips metadata before sending to Vision LLMs or REST APIs.
  • YOLO Post-Processing & Unscaling: postprocessYolo(...) supports both end-to-end NMS-free outputs ([1, N, 6], e.g. YOLO26 / YOLOv10) and classic transposed outputs ([1, 4 + C, N]) with class-aware Greedy NMS, returning coordinates mapped back to the oriented original image.
  • Concurrency Safe: Serializes concurrent calls on reusable native scratch buffers (Future.wait safe) and provides VisionPreprocessor.preprocessInIsolate(...) for heavy photos.

Getting Started #

Add edge_vision_core to your pubspec.yaml:

dependencies:
  edge_vision_core: ^0.1.0

1. End-to-End YOLO Preprocessing & Postprocessing #

import 'dart:typed_data';
import 'package:edge_vision_core/edge_vision_core.dart';

Future<void> runYoloPipeline(Uint8List photoBytes) async {
  final vision = VisionPreprocessor();

  try {
    // 1. Decode, apply EXIF orientation, letterbox to 640x640, normalize [0, 1] NCHW
    final prep = await vision.preprocessBytes(
      photoBytes,
      PreprocessConfig.yolo(size: 640, layout: TensorLayout.nchw),
    );

    // 2. Feed `prep.tensor` (Float32List) into ONNX Runtime / TFLite / ncnn
    final Float32List rawModelOutput = await myRuntime.run(prep.tensor, prep.shape);

    // 3. Post-process detections & unscale back to original oriented image coordinates
    final detections = vision.postprocessYolo(
      rawModelOutput,
      numClasses: 80,
      numAnchors: 300,
      isEndToEnd: true, // true for [1, N, 6] NMS-free YOLO, false for [1, 4+C, N]
      confThreshold: 0.35,
      iouThreshold: 0.45,
      meta: prep.meta,
    );

    for (final det in detections) {
      print('class=${det.classId} conf=${det.confidence.toStringAsFixed(2)} '
            'box=[${det.box.x1}, ${det.box.y1}, ${det.box.x2}, ${det.box.y2}]');
    }
  } finally {
    vision.dispose();
  }
}

2. Live Camera Frame (YUV420 / NV12) Preprocessing #

final prep = await vision.preprocessYuv420(
  yPlane: cameraFrame.yBytes,
  yStride: cameraFrame.yRowStride,
  uPlane: cameraFrame.uBytes,
  uStride: cameraFrame.uvRowStride,
  vPlane: cameraFrame.vBytes,
  vStride: cameraFrame.uvRowStride,
  uvPixelStride: cameraFrame.uvPixelStride, // 1 for planar I420, 2 for NV12/NV21
  width: cameraFrame.width,
  height: cameraFrame.height,
  rotationDegrees: 90,
  config: PreprocessConfig.yolo(size: 640),
);

3. Deterministic EXIF-Free JPEG Downscaling for Vision APIs #

final encoded = await vision.resizeAndEncodeJpeg(
  highResPhotoBytes,
  targetWidth: 1024,
  targetHeight: 1024,
  resizeMode: ResizeMode.letterbox,
  quality: 88,
);
// Send `encoded.bytes` cleanly over HTTP with EXIF orientation baked in & stripped.

Reusing the C++ Core Outside Flutter #

The pure C++17 core lives entirely inside src/ (edge_vision_core.h, edge_vision_core.cpp, exif.cpp, CMakeLists.txt). You can build it directly via CMake for React Native (JSI / Nitro Modules), Python (ctypes), or C++ servers:

cmake -S src -B build/cpp_test -DEVC_BUILD_TESTS=ON
cmake --build build/cpp_test --config Release
ctest --test-dir build/cpp_test -C Release --output-on-failure
1
likes
160
points
69
downloads

Documentation

API reference

Publisher

unverified uploader

Weekly Downloads

Deterministic C++17 image & camera frame preprocessor, EXIF orienter, letterbox resizer, tensor normalizer, and YOLO postprocessor for Flutter Edge AI.

Repository (GitHub)
View/report issues

Topics

#ai #computer-vision #yolo #image-processing #ffi

License

MIT (license)

Dependencies

ffi, flutter

More

Packages that depend on edge_vision_core

Packages that implement edge_vision_core