flutter_lindera_tantivy 2026.10.17

High-performance full-text search Flutter plugin using Tantivy engine with Lindera morphological analysis for Korean, Japanese, and Chinese languages.

flutter_lindera_tantivy

English | 日本語 | 한국어 | 中文

A Flutter FFI plugin that provides high-performance full-text search powered by the Tantivy search engine and Lindera 5.0 morphological analysis (lindera-tantivy v5.0.1 & lindera-analysis v5.0.1). Supports Korean, Japanese (IPADIC/UniDic), and Chinese text processing with embedded dictionaries, character/token filter pipelines, and YAML configuration files.

Features

• 🚀 High-Performance Search: Built on Rust's Tantivy search engine (v0.26) & Lindera (v5.0)
• 🔍 Standalone Morphological Analysis: Tokenize CJK text directly into tokens or detailed POS attributes
• 🧩 Filter Pipelines & YAML Config: Full support for lindera-analysis Character Filters (unicode_normalize, japanese_iteration_mark, mapping, regex) and Token Filters (lowercase, japanese_stop_tags, korean_stop_tags, etc.) as well as lindera.yml configuration files
• 🌏 Multi-Language Support: Korean (Ko-dic), Japanese (IPADIC/UniDic), and Chinese (CC-CEDICT)
• 📱 Cross-Platform: Android, iOS, Linux, macOS, and Windows
• 💾 Flexible Storage: In-memory or disk-based index storage
• ⚡ Native Performance: Direct Rust FFI bindings via flutter_rust_bridge

Supported Languages & Dictionaries

• Korean: Embedded Ko-dic dictionary
• Japanese (IPADIC): IPA dictionary for modern Japanese
• Japanese (UniDic): UniDic dictionary for contemporary written Japanese
• Chinese: CC-CEDICT dictionary

Installation

Add this to your pubspec.yaml:

dependencies:
  flutter_lindera_tantivy: ^2026.10.17

Quick Start

1. Full-Text Search

import 'package:flutter_lindera_tantivy/flutter_lindera_tantivy.dart';

// Initialize Rust library
await RustLib.init();

// Initialize search index with Korean dictionary
initializeSearchIndex(dictionaryType: DictionaryType.korean);

// Add documents
addDocument(
  title: "Flutter Tutorial",
  body: "Learn Flutter full-text search with Lindera",
  metadataJson: '{"category": "tutorial"}',
);

// Search documents
final results = searchDocuments(
  queryStr: "Flutter",
  limit: BigInt.from(10),
);

for (var result in results) {
  print('${result.title}: ${result.score}');
}

2. Morphological Analysis (Tokenization)

import 'package:flutter_lindera_tantivy/flutter_lindera_tantivy.dart';

// Standalone text tokenization
final tokens = tokenizeText(
  dictionaryType: DictionaryType.korean,
  text: "한국어 형태소 분석 테스트",
  mode: TokenMode.normal,
);
print(tokens); // ['한국어', '형태소', '분석', '테스트']

// Detailed tokenization with Part of Speech (POS) tags
final details = tokenizeTextDetailed(
  dictionaryType: DictionaryType.korean,
  text: "한국어 형태소 분석",
  mode: TokenMode.normal,
);

for (var token in details) {
  print('${token.surface} [${token.pos}]: ${token.details.join(", ")}');
}

// Tokenize with lindera-analysis Character & Token Filters
final filtered = tokenizeTextWithFilters(
  dictionaryType: DictionaryType.japaneseIpadic,
  text: "Ｌｉｎｄｅｒａは形態素解析ｴﾝｼﾞﾝです。",
  mode: TokenMode.normal,
  characterFilters: const [
    FilterConfig(kind: 'unicode_normalize', argsJson: '{"kind":"nfkc"}'),
  ],
  tokenFilters: const [
    FilterConfig(kind: 'lowercase', argsJson: '{}'),
  ],
);

API Reference

Morphological Analysis

// Tokenize text into surface string list
List<String> tokenizeText({
  required DictionaryType dictionaryType,
  required String text,
  required TokenMode mode,
})

// Tokenize text into detailed TokenDetail objects
List<TokenDetail> tokenizeTextDetailed({
  required DictionaryType dictionaryType,
  required String text,
  required TokenMode mode,
})

// Tokenize text with character and token filters (lindera-analysis)
List<TokenDetail> tokenizeTextWithFilters({
  required DictionaryType dictionaryType,
  required String text,
  required TokenMode mode,
  required List<FilterConfig> characterFilters,
  required List<FilterConfig> tokenFilters,
})

// Tokenize text using a YAML configuration file (or LINDERA_CONFIG_PATH)
List<TokenDetail> tokenizeTextWithConfig({
  required String configPath,
  required String text,
})

Initialize Index

// In-memory index
String initializeSearchIndex({required DictionaryType dictionaryType})

// Disk-based index
String initializeSearchIndexWithPath({
  required DictionaryType dictionaryType,
  required String indexPath,
})

// Initialize index from a Lindera YAML configuration file
String initializeSearchIndexWithConfig({
  required String configPath,
  String? indexPath,
})

// Initialize index with character and token filters
String initializeSearchIndexWithFilters({
  required DictionaryType dictionaryType,
  required TokenMode mode,
  required List<FilterConfig> characterFilters,
  required List<FilterConfig> tokenFilters,
  String? indexPath,
})

Document Management

// Add single document
String addDocument({
  required String title,
  required String body,
  required String metadataJson,
})

// Add multiple documents
String addDocuments({required List<DocumentInput> documents})

// Update document
String updateDocument({
  required String id,
  required String title,
  required String body,
  required String metadataJson,
})

// Delete document(s)
String deleteDocument({required String id})
String deleteDocuments({required List<String> ids})
String clearAllDocuments()

// Get document count
BigInt getDocumentCount()

Search

List<SearchResult> searchDocuments({
  required String queryStr,
  required BigInt limit,
})

AI Assistant Skills Guide (Claude, Codex, Gemini / Antigravity)

This repository includes pre-configured AI Skills and agent instructions so that AI coding assistants can immediately understand, integrate, and execute workflows with flutter_lindera_tantivy.

Skill Location

• Skill Path: skills/flutter-lindera-tantivy/SKILL.md
• Examples: skills/flutter-lindera-tantivy/examples/

How AI Assistants Use the Skill

Assistant	Integration Method	How It Works
Google Gemini / Antigravity	.agents/skills/flutter-lindera-tantivy/SKILL.md & AGENTS.md	Automatically discovered via progressive disclosure when asking about search, tokenization, or Tantivy in Flutter.
Anthropic Claude (Claude Code)	CLAUDE.md & skills/flutter-lindera-tantivy/SKILL.md	Claude Code CLI reads CLAUDE.md at root and references SKILL.md for recipes and verification commands.
OpenAI Codex / GitHub Copilot	AGENTS.md & .github/copilot-instructions.md	Codex and Copilot consult the repository instructions and skill guide for API patterns and types (BigInt, RustLib.init).

Prompting AI Assistants

You can instruct any AI assistant with prompts like:

• "Use the flutter-lindera-tantivy skill to set up an on-disk full-text search index in my Flutter app."
• "Implement Korean morphological analysis using flutter_lindera_tantivy following the repository skill guide."

License

See LICENSE file.

Repository

https://github.com/JAICHANGPARK/flutter_lindera_tantivy