betto_lang_detector 0.1.0-dev.1

A pure-Dart language detection library for Dart and Flutter applications.

betto_lang_detector

A pure-Dart language detection library that works on all Dart and Flutter target platforms — mobile, web, and desktop. No FFI, no native build, no model runtime.

Detects 58 languages using a two-stage pipeline: a near-free Unicode script pre-filter, then a character n-gram model to disambiguate languages that share a script. Designed for coarse, error-tolerant use cases — lexical analyzer selection, document metadata, and tokenizer routing — not high-accuracy NLP-grade classification.

Features

• LanguageDetector.pureDart() — the zero-dependency default, covering 58 languages (the same set betto_lexical's Stopwords enum covers, so the two compose directly with no code-translation layer)
• dominantScript() — cheap, script-only classification (ISO 15924 code) without running the (comparatively more expensive) n-gram stage — the signal you actually need for tokenizer routing
• restrictTo — narrow the candidate language set when you already know the plausible languages; the biggest practical accuracy lever this detector has
• Pluggable via LanguageDetectorBackend for testing or an alternative scoring strategy
• Undetermined as a first-class result — never throws on ordinary input
• Zero runtime dependencies; works on web (no dart:io), mobile, and desktop

Getting started

Add betto_lang_detector to your pubspec.yaml:

dependencies:
  betto_lang_detector: ^0.1.0-dev.1

Run dart pub get (or flutter pub get).

Usage

import 'package:betto_lang_detector/betto_lang_detector.dart';

void main() {
  final detector = LanguageDetector.pureDart();

  switch (detector.detect('Bonjour tout le monde')) {
    case Detected(best: final guess):
      print('${guess.code} (${guess.confidence})'); // fr (1.0)
    case Undetermined():
      print('could not determine a language');
  }

  // Cheap script-only routing, without the n-gram stage:
  detector.dominantScript('こんにちは'); // 'Hira'
  detector.dominantScript('中文'); // 'Hani'

  // restrictTo narrows (and sharpens) the n-gram comparison when you
  // already know the plausible language set:
  final restricted = LanguageDetector.pureDart(restrictTo: {'en', 'de', 'nl'});
  restricted.detect('Guten Morgen'); // Detected(de, ...)
}

Algorithm

Two independent stages:

1. Script pre-filter. A generated Unicode codepoint-range table resolves dominantScript() via binary search. Seven scripts (Bengali, Gujarati, Armenian, Greek, Hebrew, Thai, Hangul) map to exactly one of the 58 languages and short-circuit detection immediately; Han-script text is split into Japanese/Chinese by kana presence rather than script alone.
2. Character n-gram model. For the remaining four scripts (Latin, Cyrillic, Arabic, Devanagari — 49 of the 58 languages), a Cavnar & Trenkle "N-Gram-Based Text Categorization" (1994) model compares the input's own ranked character n-grams (orders 1-5) against each candidate language's reference profile via an "out-of-place" distance.

Confidence is a relative score across the candidate set compared, not a calibrated probability — narrowing restrictTo sharpens it, by design. See docs/spec/README.md for the full algorithm, the 58-language coverage table, and known limitations (short-input accuracy, closely-related Cyrillic Slavic language confusion, the Kurdish script-variant assumption).

Additional information

• Package specification
• Changelog
• Contributing