eas_passport_ocr 0.2.9

On-device passport MRZ recognition for Flutter Android and iOS.

eas_passport_ocr · Flutter SDK

Recognize TD3 passport information on Android and iOS. Images are processed on the device and are not uploaded to an OCR server.

The SDK reads the two-line machine-readable zone (MRZ) and returns core passport fields, rawMrz, and isExpired, using the same result schema as the JavaScript SDK. It does not read Chinese text or extract the issue date, place of issue, or issuing authority from the printed page.

Version 0.2.9 adds scanning sessions, latest-frame scheduling, state feedback, result locking, memory-frame input, cooperative cancellation and per-scan read/time budgets. Install the pinned release from pub.dev, or use the local path dependency below.

License

Copyright belongs to Easit. This package uses a proprietary Evaluation and Commercial License, not MIT or another open-source license.

Internal testing, evaluation, and modifications for evaluation are permitted. Production deployment, commercial use, publicly distributed applications or services, and redistribution of source or compiled code require prior, separate written authorization from Easit. Downloading the package from pub.dev does not grant a commercial license. Third-party dependencies remain subject to their own terms.

Installation

Requires Flutter 3.44 or later and Dart 3.12 or later. Add the pinned version to your app's pubspec.yaml:

dependencies:
  eas_passport_ocr: 0.2.9

Then run:

flutter pub get

For a local source distribution, extract it and use a path dependency instead:

dependencies:
  eas_passport_ocr:
    path: ../eas_passport_ocr

The default engine uses the community-maintained google_mlkit_text_recognition: 0.17.1 plugin. Native ML Kit models must be bundled with the app. Version 0.2.8 also includes a small iOS plugin that reads corrected rows with the system Apple Vision engine when early ML Kit reads are insufficient. Both engines run on the device; no additional downloadable model is introduced.

Scanning sessions and memory frames (0.2.9)

The host owns camera permissions, preview, sampling cadence and camera buffers. The SDK keeps at most one active recognition and one latest pending frame. New frames replace only the pending frame; they do not cancel active OCR.

import 'dart:typed_data';
import 'package:eas_passport_ocr/eas_passport_ocr.dart';

final sdk = EasPassportOcr();
final session = sdk.createScanSession(
  onState: (event) => updateScanState(event.state, event.reason),
  onResult: (result) => fillPassportForm(result), // Validated 11-field JSON only.
  onError: (error) => showError(error['msg'], error['suggestion']),
  // Optional maxOcrReads / maxDuration are per frame; default is full verification.
);
Future<void> startCameraFlow() => session.start();

// Call synchronously before the host camera reuses its BGRA buffers.
void onBgraFrame(Uint8List bytes, int width, int height, int rowStride, int rotation) {
  if (!session.acceptsFrames) return;
  session.offerFrame(PassportImageFrame(
    width: width, height: height, rotation: rotation,
    format: PassportPixelFormat.bgra8888,
    planes: [PassportImagePlane(
      bytes: bytes, bytesPerRow: rowStride, bytesPerPixel: 4,
    )],
  ));
}

// User actions (await each operation):
// await session.pause();
// await session.resume();
// await session.reset(); // Drain the old frame, unlock and scan the next passport.

// On leaving: stop camera delivery, then drain the session and release the SDK.
// await session.dispose();
// await sdk.dispose();

For timing correlation, pass a unique nonnegative safe integer with session.offerFrame(frame, frameId: id) and subscribe to onFrame. PassportScanFrameEvent contains frameId, phase (started/finished), queueDuration, elapsed, and a finished outcome (success or an error code), with no image or identity. Replaced pending frames produce no events; stale completion after stopping is suppressed. Queue duration starts at offerFrame, after the host constructs the immutable frame. The host must timestamp camera delivery before copying planes to measure the entire frame-to-result interval. See the live camera demo for both mobile platforms.

offerFrame(frame, frameId: id, maxDuration: const Duration(milliseconds: 2500)) can tighten the soft deadline for that frame. A positive duration is required; a session-level limit remains an upper bound. Omit it to inherit the session limit. Budget exhaustion drains native work and refuses incomplete verification. The demo alternates two short attempts with one full attempt; the SDK default remains a full attempt.

PassportImagePlane copies bytes at construction and exposes a read-only view. PassportImageFrame freezes the plane list and validates dimensions (1–4096 per edge), strides, plane lengths and rotation (clockwise 0/90/180/270 to upright). The host can reuse its original buffers after construction. Frames and queued references are discarded when replaced, drained or stopped; the SDK writes no intermediate image files. Check acceptsFrames before copying planes when inactive. Conversion, rotation and resizing to a maximum 2400-pixel edge run in compute; native reads use RGBA bitmaps through the existing ML Kit/Vision pipeline.

Format	Planes and strides
rgba8888 / bgra8888	One plane, bytesPerPixel = 4, bytesPerRow includes any row padding
yuv420	Three planes in Y, U, V order; preserve each plane's bytesPerRow and bytesPerPixel, including interleaved chroma views
nv21	One contiguous Y + VU plane, even width/height, bytesPerPixel = 1, same bytesPerRow for Y and VU

YUV conversion uses BT.601 limited range. Full-range or other YUV matrices must be converted by the host to RGBA/BGRA first. Do not infer format solely from the OS; use actual camera metadata. Mirrored input must be unmirrored by the host. See camera-plane bridge example; no camera plugin dependency is imposed by the SDK.

For a single memory frame use sdk.recognizeFrame(frame, ...) (throws) or sdk.recognizeFrameResult(frame, ...) (11-field success / 3-field failure JSON). Both accept cancellationToken, maxOcrReads and maxDuration. Path-based APIs remain available. Custom engines can optionally implement PassportFrameTextEngine; path-only custom engines reject memory frames/sessions with INVALID_INPUT and remain usable for file recognition. Custom engines retain responsibility for their own confirmation; read budgets remain unsupported for them.

Session behavior matches the JavaScript SDK:

• start/resume prepare reusable handles, then accept frames. Native model loading can still occur on the first frame; initialize is not a full native-model warm-up guarantee. Session setup is outside per-frame budgets; conversion, preprocessing and recognition are inside them.
• offerFrame returns false when inactive or locked, and throws DISPOSED after disposal. A validated result fires onResult once and locks acceptance until reset; start/resume/stop do not clear this lock. No partial identity fields, cross-frame field merging or identity history are emitted.
• start exclusively claims its SDK instance. Pause, error and completed states retain the claim; direct recognition/disposal and a second session report BUSY. await stop/dispose drains and releases the claim without closing the underlying SDK. reset drains, unlocks and starts again. Dispose is idempotent.
• onState uses PassportScanState: idle (initial property), preparing, searching, recognizing, verifying, completed, paused, stopped, error and disposed. Events contain only state and optional reason; verifying is emitted only for actual opposite-geometry checks. There is no stability/blur-selection detector in this version.
• MRZ_NOT_FOUND, MRZ_INVALID, ABORTED and BUDGET_EXCEEDED return to searching with reason; do not show a modal error for every camera frame. Other errors stop acceptance and call onError with the three-field JSON. Preparation failures also reject start/reset; catch those futures. Callbacks are synchronous; exceptions cannot break cleanup.
• Pause/stop/reset/dispose immediately reject further old-flow frames and suppress stale results, then wait for all in-flight work before settling. They are cooperative, not hard timeouts. The default retains the full 80-read ceiling, required repeat confirmation, all applicable 12 opposite-geometry reads and conflict rejection; a low budget never produces a partial success. Candidate evidence is isolated per frame.

Quick start

import 'dart:convert';
import 'package:eas_passport_ocr/eas_passport_ocr.dart';

// Own one instance for the scan screen or workflow; reuse it for each photo.
final sdk = EasPassportOcr();

Future<String> scanPassport(String localImagePath) async {
  final result = await sdk.recognizeResult(localImagePath);
  if (result.containsKey('code')) {
    // Show result['msg'] and result['suggestion'] to the user.
  } else {
    // Populate your form and ask the user to verify the passport details.
  }
  return jsonEncode(result);
}

// Call when leaving the workflow, after any in-flight scan has finished.
Future<void> closePassportFlow() => sdk.dispose();

Pass a local file path, not an HTTP or file:// URL. The host app handles image selection, camera and photo-library permissions, and any optional cropping. Capture the complete page with both MRZ lines visible.

An instance can be reused for sequential scans. Starting another scan or disposing of the instance while recognition is running reports BUSY. Wait for recognition to finish before calling dispose(). Repeated disposal is safe; scanning with a disposed instance reports DISPOSED.

Cancellation and per-scan budgets (0.2.9)

Both recognize and recognizeResult accept these optional named parameters:

Parameter	Default	Meaning
cancellationToken	None	A PassportOcrCancellationToken; call cancel() to stop scheduling more work. A cancelled token cannot be reset.
maxOcrReads	80	Integer from 1 to 80. Counts every native OCR call across ML Kit, Vision, full pages, rows, segments, and retries.
maxDuration	No time limit	Positive Duration measured from this SDK call, including setup, decoding, preprocessing and OCR.

final token = PassportOcrCancellationToken();
final pending = sdk.recognizeResult(
  localImagePath,
  cancellationToken: token,
  maxOcrReads: 80,
  maxDuration: const Duration(milliseconds: 500),
);

// In a separate cancel/leave-screen callback, when this frame is obsolete:
// token.cancel();

final result = await pending;
// Only now may the same instance scan another frame or be disposed.
// Use a fresh token for the next frame.

The example sets a soft 500 ms budget, not a promise to recognize or return within 500 ms. Active native calls and preprocessing are awaited; they cannot be forcibly interrupted. A cold model or difficult image can exceed the duration while active work drains. No new reads are dispatched after cancellation or an observed deadline, and BUSY remains in effect until draining finishes. Initialized recognizers remain reusable. Future.timeout alone does not cancel an SDK scan.

Cancellation returns ABORTED. Insufficient read/time budget returns BUDGET_EXCEEDED. Both use only code, msg, and suggestion; neither is a diagnosis of glare or damaged images. recognize throws the corresponding PassportOcrException instead. Within an OCR batch, an actual native failure takes precedence over cancellation observed while that batch finishes.

A read budget must cover every required confirmation step. In particular, a natural-row or Vision candidate cannot succeed without all 12 opposite-geometry reads. The SDK checks that the complete verification batch fits before starting it; it may stop with unused reads rather than return a partially checked result. Very small budgets reduce successful recognition. For a stable frame, allow a complete attempt instead of repeatedly cancelling it.

For camera integrations, the host should keep at most one active scan and one newest pending frame. Cancel an obsolete active scan when appropriate, await its completion, then process the latest frame with a fresh token. Do not create unbounded SDK instances or start a scan for every camera frame.

Existing injected PassportTextEngine implementations remain compatible. They support cancellation and duration checks before/after their complete recognizeLines call, but cannot be interrupted internally. Passing maxOcrReads to a custom engine returns INVALID_INPUT, since the SDK cannot measure that engine's internal reads.

Platform setup

Android

• The plugin requires minSdk 21 and compileSdk 36 or later. Use the targetSdk from your Flutter template. If Flutter or another dependency requires higher values, use those values.
• Keep the plugin's bundled-model dependency, com.google.mlkit:text-recognition:16.0.1. Do not replace it with the Play Services variant that downloads models dynamically.

iOS

• Requires a deployment target of iOS 15.5 or later and Xcode 15.3 or later.
• Set the deployment target in both Runner and the Podfile. Exclude unsupported 32-bit architectures as described in the ML Kit plugin documentation.
• The default plugin includes the Latin text model; no additional Chinese-language Pod is needed.
• Version 0.2.8 adds the eas_passport_ocr native Pod. Run flutter pub get, update Pods, and fully rebuild the app after upgrading; hot reload alone cannot register the new native channel.
• If using the camera/gallery example, configure NSCameraUsageDescription and NSPhotoLibraryUsageDescription in Info.plist.

Image OCR with the default engine is not supported on Flutter Web, macOS, Windows, or Linux and returns UNSUPPORTED_PLATFORM. The standalone parsePassportMrz function does not invoke native OCR.

API and results

Recognition

• recognizeResult(path) returns a JSON-serializable Map<String, dynamic> containing either passport fields or a three-field error.
• recognize(path) returns a PassportResult and throws PassportOcrException on failure. Call toJson() on the result or exception to serialize it.
• EasPassportOcr(correctDistortion: false) disables geometric correction and the upscaling, line-box, and orientation retries. Whole-page repeated confirmation still runs.

Successful result

A successful result contains exactly these 11 fields, without a success flag or data wrapper:

Field	Meaning
documentType	Document type from the MRZ.
issuingCountry	Issuing-state code from the MRZ.
surname	Surname in MRZ transliteration.
givenNames	Given names in MRZ transliteration.
passportNumber	Passport number.
nationality	Nationality code from the MRZ.
birthDate	Date of birth, formatted as YYYY-MM-DD.
sex	M, F, or X.
expiryDate	Expiration date, formatted as YYYY-MM-DD.
rawMrz	Two normalized MRZ lines separated by a newline.
isExpired	Expiration status based on the device's local date.

A passport is not considered expired on its expiration date. toJson() recalculates isExpired rather than retaining a stale value. MRZ dates contain two-digit years, so the inferred century must be checked against the original passport.

If you already have OCR text, parse it without invoking the recognition model:

final result = parsePassportMrz('$line1\n$line2');
final expired = result.isExpired;
final data = result.toJson();

Failed result

A failure contains only three string fields: code, msg, and suggestion. Use result.containsKey('code') to detect it. suggestion is a single string, not an array.

The SDK currently returns user-facing error messages in Chinese. Translating this README does not change the runtime language. The following is an example of the actual output:

{
  "code": "MRZ_INVALID",
  "msg": "未能识别清楚，照片可能存在反光、扭曲、缺角或文字截断。",
  "suggestion": "请避开反光、摊平护照，完整拍入资料页及四角，确保底部两行字母和数字清晰、未被截断。"
}

In English, this says that recognition was unsuccessful and the photo may have glare, distortion, missing corners, or cut-off text. It advises avoiding glare, flattening the passport, capturing the entire page and its corners, and keeping both MRZ lines clear and complete.

Code	Meaning
INVALID_INPUT	Invalid image path or input.
UNSUPPORTED_PLATFORM	The default image OCR engine is unavailable on this platform.
BUSY	A recognition operation is already running.
DISPOSED	The instance has been disposed of.
ABORTED	The scan was cancelled; active work has finished.
BUDGET_EXCEEDED	The scan could not complete within its read/time budget.
MRZ_NOT_FOUND	No usable MRZ was found.
MRZ_INVALID	The MRZ could not be reliably confirmed and validated.
OCR_FAILED	Local recognition failed; check the image, permissions, and native engine configuration.

When there is image evidence, messages describe suspected glare, bending or character-size variation, text near an image edge, or insufficient character resolution. When the cause cannot be narrowed down, the message lists glare, distortion, missing corners, and cut-off text as possible causes.

These are conservative diagnostic rules, not a classifier calibrated on a real-world dataset. They do not detect all four page corners or independently classify blur. Text near an image edge does not prove that a physical corner is missing. Image diagnostics run only with the default engine; custom text engines provide text for MRZ validation.

Failure JSON does not contain a success flag, nested error, diagnostics, measurement data, cause lists, unvalidated identity fields, images, or stack traces.

Migrating from 0.1.x: replace checks for a success flag and nested error messages with the three-field schema above. The throwing recognize() API remains available. Dart's PassportOcrException.toJson() and JavaScript's PassportOcrError.toJSON() use the same three-field error schema.

Whole-page recognition and confirmation

The default engine accepts a complete passport page without a manually selected MRZ region. It starts with local ML Kit recognition and repeats reads on normalized and contrast-adjusted images. When needed, it locates the two character rows, preserves natural character proportions, straightens curved baselines, and tries per-character correction. Long rows are scaled to fit the output canvas without cutting off their ends.

On iOS, if the first two ML Kit line variants do not confirm a result, a bounded Apple Vision fast-mode pass reads both natural and per-character corrected rows before continuing the remaining variants. Dictionary correction is disabled. Both engines contribute to the same candidate pool and share the 80-read limit: conflicting valid passport numbers or dates are still rejected, and a Vision result must also pass all 12 alternative-geometry reads. If those early reads remain unconfirmed, a bounded accurate-mode fallback reads the first row intact and the second row in two pixel-preserving segments. This changes recognition context for mixed letters and digits without replacing, inserting, or dropping recognized characters. It shares the same confirmation, conflict checks, and total read budget. Difficult inputs may take more than 500 ms. Android keeps its existing ML Kit path.

Version 0.2.3 incorporates repeated-read confirmation and conflict checks from the JavaScript SDK. Both rows require support from different processing attempts. Before accepting a corrected whole-row result, the engine rereads the second row using another geometric sampling method. Conflicting passport numbers or dates that each pass validation cause rejection.

The SDK does not guess replacements such as O/0, insert missing characters, truncate extra characters, or relax checksum requirements. Stricter confirmation may reject photos accepted by earlier versions. Repeated agreement and passing checksums do not guarantee every field is correct or authenticate a document; users should verify results against the original passport.

Fallbacks include bounded upscaling, rotation, and crops based on ML Kit line boxes. Each scan makes at most 80 native OCR calls and stops early once a result is confirmed. Difficult images can take longer. Local illumination correction and shear compensation can help where character strokes remain visible, but cannot recover information erased by glare or cut off from the image. Arbitrary distortion recovery is not guaranteed.

Decoding, scaling, geometric processing, and encoding derived images run in a background isolate using compute. Full-page copies, crops, and line images are stored in a temporary directory and removed on success or error, without modifying the source image. Forced process termination can prevent cleanup. Do not back up or upload these temporary files.

Custom OCR engine

Implement PassportTextEngine.recognizeLines() and close(), then pass the engine to EasPassportOcr(engine: yourEngine). Return text lines in reading order.

A custom engine is responsible for preprocessing and repeated-read confirmation. The SDK continues to locate, parse, and validate the MRZ and produce the common result schema. It does not read virtual paths supplied to custom engines or perform the default engine's image diagnostics.

Example app and verification

example/lib/main.dart provides a camera/gallery screen and JSON copying. The distribution does not include generated Android/iOS project scaffolding. Create it with:

cd example
flutter create --platforms=android,ios --project-name passport_ocr_example .
flutter pub get
# Complete the platform and permission setup above, then:
flutter run

Run package checks from the package directory:

flutter pub get
flutter analyze
flutter test

Tests use fictional public fixtures, synthetic distorted characters, and native-channel test doubles. They cover the result schema, repeated confirmation, checksum collisions, geometry, rotation and line-box fallbacks, retry limits, instance reuse, in-memory image transport, error drainage, and resource disposal.

Limited iPhone 15 Pro testing has been performed; broad iOS and Android acceptance testing remains pending. Mocked ML Kit responses do not establish real-photo recognition accuracy. Browser results cannot be treated as Flutter results because the engines differ. Test offline on your target devices to verify bundled models, permissions, and recognition quality. The source archive excludes native engines and models, so its size is not the increase in your app's installation size.

Dependencies

• Flutter ML Kit plugin and platform requirements: community maintained, not an official Google Flutter SDK.
• Android ML Kit bundled text recognition.

Third-party dependencies and native ML Kit components are governed by their own terms. This package does not include the Tesseract engine or OCR-B model used by the JavaScript SDK.

Performance and version alignment

Version 0.2.8 removes intermediate image files from the recognition path. Flutter's platform codec handles photo decoding and orientation, with a 2400-pixel longest-edge bound. A background isolate computes row geometry and corrected lines; ML Kit receives RGBA bitmaps directly. Rotation, enlargement and line-box retries reuse the decoded pixels.

Independent native reads use a reusable pool of Latin recognizers. The default is one; set workerCount to an integer from 1 to 4 to control concurrency and memory. More workers consume more memory and are not guaranteed to be faster on every device. Each recognizer handles one image at a time, and completed evidence is merged in a fixed order. Repeated confirmation, all alternative-geometry conflict checks and the 80-read limit remain enabled. Once two valid passport-number/date candidates conflict, the scan fails immediately: additional reads cannot remove that evidence. Difficult or invalid photos may still require many attempts.

final sdk = EasPassportOcr(
  workerCount: 1,
  onPerformance: (timing) {
    // Optional local instrumentation. No photo, file path or OCR text is included.
    final engineMs = timing.total.inMicroseconds / 1000;
    final decodeMs = timing.decoding.inMicroseconds / 1000;
    final preparationMs = timing.preprocessing.inMicroseconds / 1000;
    final nativeMs = timing.recognition.inMicroseconds / 1000;
    final reads = timing.recognitionCalls;
    // Send these numbers to your local benchmark UI if needed.
  },
);

onPerformance is optional and applies to the default native engine. It does not add fields to the result JSON or store/send measurements. Its stages can overlap, so adding stage durations does not give latency. total covers the engine scan; for end-to-end SDK latency, time the complete recognizeResult call with a Stopwatch. Report cold scans separately from scans reusing an instance, use Release/Profile builds, and compare complete results as well as elapsed time. Camera/photo-picker time is outside the SDK call.

A universal 500 ms limit is not established. The iOS accuracy follow-up was tested on an iPhone 15 Pro in Release mode: all four existing test images returned successful results in two rounds, taking 246–397 ms per complete SDK call. Two of these images failed in the earlier ML-Kit-only build. This is a four-image test, not a population accuracy rate or a guarantee for full-resolution camera photos. Only the first image in this batch was the first scan of the process; new SDK instances do not reset system caches.

All 11 fields were consistent between the two final rounds and the preceding prototype. Three images matched the JavaScript reference in all 11 fields; the fourth matched the 10 non-raw fields but differed in rawMrz. That raw-text discrepancy is unresolved against the physical original. Do not describe checksum success or cross-engine agreement as proof of document authenticity or complete transcription accuracy. A later seven-image, three-round iPhone regression also succeeded, preserving all previous results. The new difficult JPEG matched all 11 fields of a clearer image of the same passport, at 1726/1120/1159 ms; this exceeds 500 ms and does not establish general accuracy.

Unit tests and host preprocessing benchmarks do not measure native ML Kit speed or real-photo accuracy. Do not set a 500 ms timeout that abandons verification or returns unchecked fields.

JavaScript and Flutter use matching release numbers. Since 0.2.8, both SDKs validate issuing/nationality codes using an offline CLDR 48 plus MRTD compatibility table. Unknown codes are rejected, never silently replaced. The table includes historical and special codes and must be maintained when codes change; it does not establish document authenticity. Version numbers do not imply equivalent speed or recognition accuracy between engines.