Wayfinder

Search the current project's configured local OKF bundles. Resolve a configured Profile source with get before read-only validation:

wayfinder get .
wayfinder validate ./knowledge
wayfinder upgrade .
wayfinder graph ./knowledge --output mermaid
wayfinder index
wayfinder search "How do I regain account access?"
wayfinder search "How do I regain account access?" --bundle knowledge

Project search/index discover the nearest wayfinder.json from the current folder and use its applies_to paths. Without one, a conventional knowledge/ directory is supported. Discovery stops at a Git repository boundary. Bundle filters use folder names; ambiguous names require an explicit path. These commands do not fetch Profile sources or install bundles.

Search ranks matches across all selected bundles with one query embedding and one global --limit. Project JSON adds bundles, plus bundle and bundlePath on each hit; text citations include the project-relative bundle path. Source paths, line numbers, status, and relationship context remain available. A missing or stale selected index fails the search, rather than silently returning a subset.

The explicit index <path> and search <path> "<query>" forms retain their single-bundle JSON shape and ignore neighboring configuration. Background indexing (--detach) continues to require an explicit path.

Index and search always use local embeddings. Validation runs the existing OKF and the Profile checks its wayfinder.json selects and retains their output and exit codes. Validation never assesses the judgment a Profile's skill asks for. For projects with direct Git Profile sources, get writes or reuses wayfinder.lock; upgrade refreshes a branch or tag. CLI and MCP validate read only a current lock/cache and preserve the independent OKF result if the Profile source is unresolved. When the selected chain has a matches-generated rule, validate --fix first writes okf's generated indexes, then validates. It never writes a bundle without a selected Profile or one that fails OKF. A bundle with no wayfinder.json above it reports wayfinder/config-missing. graph, index, and search do not resolve Profile sources. See the configuration guide.

Native installation

For indexing and search without a Dart SDK, install the complete runtime from the installation guide. It includes Wayfinder with OKF and Profile validation, the embedding model and required native libraries. The GitHub repository hosts the source, plugin and native release archives.

Dart installation

Requires Dart 3.11.0 or later. Install the development release:

dart pub global activate wayfinder_cli
wayfinder validate ./knowledge

Add the Dart pub cache's bin directory to your PATH if wayfinder is not found. Validation and MCP discovery work without model weights. Indexing and search also require the verified embedding model and native libraries; pub.dev does not distribute these assets with this package. Use the native installer above, or the source preparation and bundle build below, for the complete retrieval installation.

Source development and packaging

From the workspace root:

dart pub get
dart run melos run objectbox:install
dart run melos run wayfinder_embeddings:prepare
dart run wayfinder_cli:wayfinder get examples/bitwild
dart run wayfinder_cli:wayfinder validate examples/bitwild/knowledge
dart run wayfinder_cli:wayfinder index examples/bitwild/knowledge
dart run wayfinder_cli:wayfinder search examples/bitwild/knowledge "How is reporting implemented?"
dart run tool/build_wayfinder.dart --offline

The build creates build/wayfinder/bundle/bin/wayfinder with native libraries and the pinned 25.28 MB Arctic XS model. Distribute the whole bundle/ directory. Model size is only part of the installation size. --offline requires staged or cached model weights; native build hooks have their own cache.

Preparation downloads and verifies model weights. It generates no document embeddings. Runtime inference loads verified local weights through llamadart; there is no model download command or external embedding service. Source runs resolve the prepared embedding package's model/library. Packaged runs resolve their own assets independently of the current working directory.

Embedding lifecycle

Action Embedding work
validate None
First index Fit passages to the tokenizer, embed them and persist a completed snapshot
Repeat index, nothing changed None; returns without loading the model
Repeat index after edits Reuse compatible vectors; encode changed/new inputs and remove deleted passages
index --force Discard the saved index and re-embed every passage
search Reopen the completed snapshot and encode only the query

Titles/headings are part of document embedding inputs. Other metadata and citation-only updates can refresh without inference. Passage shifts can change IDs and cause additional re-encoding. An unchanged index returns without opening the model or staging a database copy. --force rebuilds from scratch into a new generation, which replaces the old one only when it completes. --detach returns at once and indexes in a background process only when the bundle changed; the refresh hooks use it. One background index runs per machine: while one runs, --detach reports it and starts nothing. A background index stops after 30 minutes, or as soon as its bundle directory disappears; index a large bundle for the first time in the foreground.

Oversized-input recovery and warnings

Indexing losslessly subdivides oversized passages, including long separators and identifiers, using the actual tokenizer. If a derived title/heading prefix prevents fitting, indexing retries that original chunk without the prefix. Source files are never edited; fitted fragments reconstruct each pre-fit passage, not the entire document. Metadata and source citations remain available. An irreducible body-only input or a tokenizer/inference error still fails indexing.

Completed and current foreground index results include warnings (an empty array when none). Each warning has code, sourcePath, lineStart, lineEnd, and affectedChunks, counting original chunks once per code and enclosing their line ranges. Codes are oversized_segment_split and embedding_context_omitted; ordinary whitespace splitting is silent. Warnings are sorted by source path, then code.

Human-readable indexing prints warnings to stderr with single-line escaped paths. CLI JSON and MCP return the same structured warnings in their existing payload; MCP still uses one JSON text block. Successful recovery keeps exit status 0. Saved warnings replay on current-index calls without loading the model. Detached launch/current/running acknowledgements keep their status-only shape; once a background index completes, the next foreground call reports its saved warnings. Consumers enforcing a closed JSON response shape must allow the new warnings field.

This changes application index configuration to version 4 (excluding Attested Computation body passages from embedding while retaining their metadata and graph relationships). Run wayfinder index <bundle> once to rebuild even if source bytes are unchanged; subsequent unchanged calls reuse the index. Snapshot version 1 remains readable with additive recovery fields. No bundle migration, Profile release, model change, or ObjectBox schema change is required. New fragment boundaries or omitted context can change retrieval results. Failed rebuilds preserve the previous generation, but search still rejects stale or configuration-incompatible indexes; there is no automatic fallback to old vectors.

Native first-use initialization remains variable. Reusing document vectors does not remove model startup or the first query's inference cost; later launches are faster.

After editing knowledge files, run wayfinder index again. Search detects an absent, stale or incompatible index and reports the indexing command; it does not silently change retrieval methods or update document embeddings.

graph reads the live bundle and projects the ordinary OKF relationship graph, plus each concept's typed relationships entries (engine contract §5.5) beside it: JSON keeps every okf key and adds them as field_edges, and Mermaid and DOT label each one with its relationship name. Search expansion follows the same typed relationships. It does not open the search index or model. --output is json (default), mermaid, or dot. Mermaid and DOT are text for an external preview such as mermaid.live; they are not a rendered picture. Repeatable --type, --path-prefix and --resolution select an induced subgraph, matching okf graph. Load findings print the OKF report and refuse a graph.

The validate, index and search commands accept --output=json. Validate also accepts --output=sarif, a SARIF 2.1.0 log of the same findings for code scanning upload; a location inside the working directory is relative to the WORKINGDIR base the log records, and one outside it is an absolute file URI. A --fix that cannot write a file marks the invocation unsuccessful and records the reason as a tool execution notification. Search also accepts --limit=1..100 (default 5). Query text is one quoted argument. Results include original paths, line ranges, metadata, similarity, context inclusion reasons and link notices. All lifecycle states remain eligible and status is displayed. Ranked passages are candidates; verify the supporting text before answering.

Validation exits 0 for gate PASS, 1 for FAIL and 2 for INCOMPLETE, which covers usage and I/O errors and any run with an error diagnostic. With --output json or --output sarif, a run that stops still prints a result carrying the wayfinder/internal-error diagnostic. Index/search return 0 on success and 2 when they cannot complete, including stale indexes, busy stores and missing model assets. An empty result set is a successful search. Graph returns 0 on success, 1 when load findings refuse a graph, and 2 for usage or I/O failure.

Set WAYFINDER_EMBEDDING_MODEL to select a verified local model file. An invalid explicit setting fails instead of selecting another model.

Native builds check the installed and packaged ObjectBox bytes against pinned platform hashes and include its license and attribution.

MCP server

wayfinder mcp

Configure a local MCP host to launch the packaged executable. For hosts using the common mcpServers configuration shape:

{
  "mcpServers": {
    "wayfinder": {
      "command": "/absolute/path/to/bundle/bin/wayfinder",
      "args": ["mcp"]
    }
  }
}

Launch from the consuming project. With no bundle argument, MCP uses the same project discovery as CLI search/index: all applies_to paths in the nearest wayfinder.json, or the conventional knowledge/ fallback. It reads configuration on each call without resolving Profile sources. No per-query bundle list is needed. Search ranks across these bundles with one query embedding and a global limit; its Markdown output (## Matches and ## Related Context) is the same as project CLI text search, including bundle-aware <path>:<lineStart>-<lineEnd> citations, essential metadata, and resolved local source links. Index refreshes them all. Validate and graph return a bundles array containing each bundle's independent report with name and project-relative path; each validation report retains its own gate and exit_code.

wayfinder mcp <bundle> still binds every tool to that single bundle. With no positional argument, a nonblank WAYFINDER_KNOWLEDGE_DIR also selects one explicit bundle. Unset that variable to search all configured bundles. The process serves stdio; stdout is reserved for JSON-RPC. The tools are validate and index (no arguments), search with a required query and optional limit (1–100, default 5), and graph with optional types, path_prefixes and resolutions. validate, index, and graph return the same JSON as the commands (wrapped per bundle for project validate/graph), while search returns Markdown matching CLI text search, in one text content block. Validation also includes the command's exit_code, findings, diagnostics and gate without treating a completed report as a tool error. graph returns the versioned OKF graph JSON (schema_version: "1"). Mermaid and DOT remain CLI text for an external preview.

Use index explicitly before the first search and after source edits. It saves document embeddings in the same local app-data directory used by the CLI. Discovery and validation work without a model. Search reports missing/stale indexes and never indexes implicitly. The host can require approval for index's local writes. There is no per-call bundle override. Search uses dense retrieval only; a verified local encoder is required. Unsupported request fields, including retired mode, are rejected.

The server keeps one lazily opened encoder across index/search calls and serializes retrieval operations that share it. The CLI delegates MCP retrieval ownership to the server, so wayfinder mcp uses this persistent lifetime. An explicit Dart knowledge factory override keeps the caller's chosen lifetime. Each call still checks source freshness and closes its store. Failed calls evict the encoder; disconnect drains accepted calls and disposes it. The service also retains parsed snapshots in an LRU cache bounded by 64 MiB of UTF-8 JSON text across bundles. Every call rereads the pointer and snapshot text; reuse requires the same canonical index directory, generation, and complete text. Changed, missing, or invalid records, failed retrievals, and disconnect clear the affected entry (or cache on dispose). Larger records bypass the cache. This avoids repeated parsing and graph reconstruction for unchanged snapshots across multi-bundle searches; it retains no open store or search response and skips no source freshness check. One-shot CLI commands do not cache snapshots and still load their own model. Configure the host's tool timeout for first-use startup and the size of the corpus. Overlapping retrieval calls can return a busy error. On disconnect, active operations finish and release their resources; cancellation does not roll back indexing. The server projects the ordinary OKF graph and does not expose upstream OKF's concept-authoring write tools. See ADR-0012.

The current mcp_dart stdio transport defaults to a 10 MiB incoming frame limit; oversized client requests close the transport. Native model startup is variable: a macOS CI run once reached llamadart's 30-second worker timeout and passed on retry. That observation is not a guaranteed startup bound or a permanent fix; allow adequate host time for cold startup and corpus-sized indexing.

Tool argument contracts use ACK through the published ack_mcp_dart ^1.3.0 adapter and the compatible ack ^1.2.0 core. The adapter applies the search default and normalizes integral JSON numbers before the callback. It preserves the advertised nonblank query, 1–100 limit and closed-object constraints.

See ADR-0011 for the MCP adapter's decision rationale.

Agent skills, project setup and updates

wayfinder skills install [--agent=all|claude|agents]
wayfinder skills status
wayfinder setup [<project>] [--bundle=<path>] [--hooks] [--session-hooks]
wayfinder update [--check] [--version=<version>]

Native bundles include the skill family. skills install uses the wayfinder Claude Code plugin when the claude CLI is available, otherwise copies the skills to ~/.claude/skills. It also copies them to ~/.agents/skills for Codex and other Agent Skills clients. It marks its copies and never replaces or removes a skill it did not install. setup adds project-wide wayfinder mcp to a project's .mcp.json and preserves other servers; --bundle=<path> instead binds it to one bundle. To replace an existing single-bundle entry, run wayfinder setup --force, unset WAYFINDER_KNOWLEDGE_DIR, and restart the host. setup --hooks also runs wayfinder index <bundle> --detach after git pulls, checkouts and rebases that change the bundle (--bundle or knowledge by default; .githooks/, enabled with core.hooksPath). The git hooks skip file checkouts and new clones or worktrees. There are no agent Stop hooks: search reports a stale index, and the MCP index tool refreshes it. Rerunning setup --hooks removes the Stop hooks earlier releases added to .claude/settings.json and .codex/hooks.json and keeps every other hook.

setup --session-hooks separately installs context-only SessionStart hooks for Claude, Codex, and Gemini, plus a managed AGENTS.md pointer for Grok, whose startup hooks ignore stdout. The short message prompts loading use-wayfinder and its references to the author, adopt, and assess skills. Install the skills first, then review/trust the local hooks in the client. Startup does no indexing or update checks. See the installation guide.

update reruns the release's verified installer for installer-managed runtimes, then refreshes the skills and plugin. A Dart installation prints its upgrade command instead. The Homebrew formula was removed, so a Homebrew installation is told to reinstall with the install script. Interactive commands also print a one-line stderr notice when a newer stable release exists. That check is the only network request Wayfinder makes on its own: at most once a day it reads the GitHub Releases API and sends no bundle content. It never runs for mcp, when stderr is not a terminal, or when CI or WAYFINDER_NO_UPDATE_CHECK is set.

Local storage

Wayfinder stores an ObjectBox database and snapshot under a hash of the canonical absolute bundle path. Separate worktrees get separate indexes; symlink aliases resolve to the same root. Moving a bundle requires indexing its new location.

  • macOS: ~/Library/Application Support/Wayfinder/indexes/<bundle-key>/
  • Linux: $XDG_DATA_HOME/wayfinder/indexes/<bundle-key>/, defaulting to ~/.local/share/wayfinder/indexes/<bundle-key>/
  • Windows: %LOCALAPPDATA%/Wayfinder/indexes/<bundle-key>/

WAYFINDER_DATA_DIR overrides the app-data root for isolated runs. It must be outside the knowledge bundle. The model is shared by the installation; query vectors and history are not written to disk. The saved snapshot holds a complete copy of the indexed bundle text with its metadata, so wayfinder index places that knowledge in per-user application data on every machine that runs it. Nothing leaves the machine; deleting the index directory removes the copy, and indexing rebuilds it from the original bundle.

An OS lock serializes same-bundle commands. Indexing stages a replacement database and snapshot, then publishes their completed generation atomically. Failed indexing preserves the previous generation; searches still reject it if source files have changed. Successful indexing removes inactive generations. Copying the database temporarily requires space for both generations. Saved generations are decoded and validated once per operation. Indexing returns a typed result internally, with additive JSON warnings and backward-readable snapshot recovery data.

Source-content rechecks detect observed edits; they are not an OS filesystem snapshot. The adapter scores eligible vectors exactly, so work increases with corpus size. There is no background watcher or navigation-file generation.

Verification

# From this package:
dart test
# From the workspace root, after building:
python3 tool/verify_wayfinder.py build/wayfinder/bundle --output=build/wayfinder-checks.json
python3 tool/verify_wayfinder_mcp.py build/wayfinder/bundle --output=build/wayfinder-mcp-checks.json

The native check copies the bundle to a temporary location and uses fresh processes for indexing/search, vector reuse, updates/deletions, missing/corrupt models, cross-process locking and crash-lock recovery. It checks validation without a model and preserves the old index after an indexing failure. The checked-in fixture is generic; it is not a conformant profile template or independent model-quality benchmark. Native CI exercises Linux/macOS.

See ADR-0010 for the application decision and ADR-0009 for model and ranking limitations.

CLI and MCP search share the ACK query, limit, and default contract. The CLI converts integer argument text before validation; MCP accepts JSON integers, including integral JSON numbers such as 2.0. Invalid arguments fail before opening retrieval resources. Whitespace-only queries include Unicode NEXT LINE (U+0085), preserving the CLI rule in MCP as well.

Filter search by frontmatter

wayfinder search "navigation" --tag routing --require-tag mobile --type Guide
wayfinder search "navigation" --description-contains "deep links" --status stable

--tag, --type, --status, and --path-prefix accept comma-separated or repeated values; any value within one field may match. Every --require-tag must match. Different fields combine with AND. Tags, types, and statuses match exact, case-sensitive identifiers; absent status matches stable, and custom statuses are preserved. --title-contains and --description-contains match literal, case-insensitive substrings. Missing or non-string fields do not match text filters. Empty filter values are errors.

Filters apply before ranking and also constrain relationship context. They work with project search, --bundle, and explicit bundle paths. They do not change ranking, write the index, or generate document embeddings. The query is still required. These flags are currently available in the CLI; MCP search retains its existing query/limit contract.

Dense retrieval

Search is dense-only. It opens the verified local encoder for the query and uses a current saved index. A missing model fails; search never falls back to another retrieval method or silently rebuilds the index.

wayfinder search knowledge "permission policy" --limit=5
wayfinder search knowledge "POL-123" --limit=5

These are candidate rankings, not calibrated relevance scores. Existing compatible embedding indexes remain usable; dense-only does not change their model, input identity, snapshot format, or storage schema.

Dense-only migration

Remove CLI --mode, including --mode=dense, and MCP mode from callers. Retired BM25/hybrid requests are rejected explicitly, not reinterpreted as dense. MCP rejects unsupported fields rather than silently discarding them; refresh tool discovery after upgrading. This is a breaking command-contract change. See ADR-0017.

Dart callers remove retrieval mode: and supply the required encoder to the index. Embeddings migration covers removed APIs. Generic reranking and its candidate windows remain available. Chunk preview and snapshot parsing remain model-free; they are not actual search. If eligible saved passages lack compatible vectors, synchronize the index again. If the model cannot open, repair its configured local path or runtime assets before retrying; reindexing alone does not repair a model load.

License

Wayfinder is distributed under the BSD 3-Clause license in LICENSE. Dependencies and separately downloaded models/native libraries retain their own licenses. The repository's OKF Profile materials retain their existing license.

Libraries