tom_build_base #

Part of the Tom framework by al-the-bear. © 2024–2026 Peter Nicolai Alexis Kyaw — BSD-3-Clause, see LICENSE.

Unified CLI framework for workspace traversal, tool definition, pipeline execution, and build configuration.

tom_build_base is the foundation every Tom command-line tool is built on — buildkit, testkit, issuekit, d4rtgen, and the rest. It answers the questions that every workspace tool has to answer and that nobody should have to re-answer: how do I declare commands and options, how do I generate help, how do I find the projects in this workspace, in what order do I process them, and how do I run a pipeline of shell and tool steps over them. A tool author declares what their tool does (a ToolDefinition with commands and an executor per command); tom_build_base supplies how it runs.

This is a deliberately large, single-purpose package. It is also the reason the Tom CLI tools feel like one product rather than a pile of scripts: they share this framework, so they share their navigation flags, their help format, their configuration model, and their end-of-run summaries.

Overview #

A workspace tool is mostly the same machinery wrapped around a small core of tool-specific logic. Without a shared base, every tool reinvents argument parsing, copies a help formatter, writes its own directory walk, and disagrees with its siblings about what --project means. tom_build_base collapses all of that into one framework with four cooperating layers:

Tool definition — ToolDefinition, CommandDefinition, OptionDefinition describe a tool declaratively. From that description the framework derives argument parsing, --help/--version, shell completion, and the standard navigation flags — no imperative wiring per tool.
Execution — ToolRunner takes a definition plus a CommandExecutor per command, parses the arguments, runs the right command across the traversal, and aggregates per-item outcomes into a single ToolResult.
Traversal — BuildBase.traverse (and the higher-level runner) scans the filesystem, detects each folder's natures (Dart project, git repo, buildkit folder…), filters by project/module selectors, orders by dependency build order or git depth, and invokes a callback with a typed CommandContext per match.
Pipelines & configuration — multi-command tools get pipelines, runtime macros, and persistent defines automatically; TomBuildConfig reads the two-tier …_master.yaml / project-level YAML configuration.

The package ships two entry points. package:tom_build_base/tom_build_base.dart is the full surface (tool framework + traversal + config + utilities); package:tom_build_base/tom_build_base_v2.dart is the same modern surface without the few legacy utility exports, for tools that only need the v2 framework.

Installation #

dependencies:
  tom_build_base: ^2.6.0

Or from the command line:

dart pub add tom_build_base

Then import the entry point:

import 'package:tom_build_base/tom_build_base.dart';

Requires Dart SDK ^3.10.4. It depends on args, console_markdown, dcli, glob, path, and yaml. It is a dart:io package — it runs on desktop, server, and CLI hosts, not the web.

Features #

Tool definition #

API	Kind	Purpose
`ToolDefinition`	class	Declares a tool: name, version, mode, commands, features
`CommandDefinition`	class	One command: name, aliases, options, nature requirements
`OptionDefinition`	class	One flag/option/multi-option, with `.flag`/`.option`/`.multi` constructors
`ToolMode`	enum	`singleCommand`, `multiCommand`, or `hybrid`
`NavigationFeatures`	class	Which navigation flags a tool exposes (`projectTool`, `gitTool`, `all`, …)
`CommandListOps`	extension	`without` / `replacing` / `plus` for deriving command lists

Execution #

API	Kind	Purpose
`ToolRunner`	class	Parses args, routes commands, drives traversal, aggregates results
`CommandExecutor`	abstract	The contract: run a command on one folder
`CallbackExecutor`	class	Build an executor from a closure
`SyncExecutor` / `ListExecutor` / `ShellExecutor` / `DartExecutor`	class	Ready-made executors
`ItemResult`	class	Outcome for one folder: success / skipped / failure
`ToolResult`	class	Aggregated run outcome + `renderRunSummary()`

Traversal #

API	Kind	Purpose
`BuildBase.traverse`	static	Scan → detect → filter → order → run a callback per folder
`BaseTraversalInfo`	abstract	Shared traversal config (exclude patterns, test-project toggles)
`ProjectTraversalInfo`	class	Project-mode config (scan path, recursive, build-order, selectors)
`GitTraversalInfo`	class	Git-mode config (modules, inner/outer-first order)
`CommandContext`	class	Per-folder context: path, natures, `getNature<T>()`
`DartProjectFolder` / `GitFolder` / `BuildkitFolder` / `ExtensionFolder`	nature	Detected folder kinds with typed metadata
`FilterPipeline` / `FolderSorter`	class	Project/module filtering and build-order/git-depth ordering

Pipelines, configuration & utilities #

API	Kind	Purpose
`PipelineConfig` / `PipelineExecutor`	class	Multi-step pipelines (`shell`, `shell-scan`, `stdin`, `print`, `{TOOL}`)
`TomBuildConfig`	class	Two-tier `…_master.yaml` + project-level config loading
`HelpGenerator` / `CompletionGenerator`	class	Auto-generated help and shell completion
`yamlToMap` / `yamlListToList` / `toStringList`	function	YAML-node conversion helpers
`MkLinkExecutor` / `createSymLink`	API	Cross-platform symlink creation for tool commands

Quick start #

A complete tool is a ToolDefinition, one CommandExecutor per command, and a ToolRunner to glue them together:

import 'dart:io';
import 'package:tom_build_base/tom_build_base.dart';

const myTool = ToolDefinition(
  name: 'mytool',
  description: 'My custom build tool',
  version: '1.0.0',
  mode: ToolMode.multiCommand,
  features: NavigationFeatures.projectTool,
  commands: [
    CommandDefinition(
      name: 'list',
      description: 'List discovered Dart projects',
      requiredNatures: {DartProjectFolder},
    ),
  ],
);

void main(List<String> args) async {
  final runner = ToolRunner(
    tool: myTool,
    executors: {
      'list': CallbackExecutor(
        onExecute: (context, args) async {
          final dart = context.getNature<DartProjectFolder>();
          print('  ${dart.projectName} v${dart.version}');
          return ItemResult.success(path: context.path, name: context.name);
        },
      ),
    },
  );
  final result = await runner.run(args);
  exit(result.success ? 0 : 1);
}

That ~30-line tool already supports mytool :list, mytool --help, mytool --version, the full --project / --exclude / --scan / --build-order navigation flag set, dependency-ordered traversal, and a consolidated end-of-run summary. --version prints:

mytool v1.0.0

and --help prints the tool description, every global option, and the command list — all derived from the definition, none of it hand-written.

The runnable version of this tool lives in example/tom_build_base_example.dart.

Example projects #

Example	What it shows
`example/tom_build_base_example.dart`	A two-command (`hello`, `list`) `ToolRunner` tool
`tom_build_base_introduction_sample`	A simple single-command build tool, built from one `ToolDefinition` and run over a fixture workspace — the introductory article.
`tom_build_base_advanced_sample`	A nestable, multi-command tool: per-command options, audit/exit codes, sequencing and nested invocation — the advanced article.
`tom_build_base_advanced_analyzer_sample`	A nestable single-command tool whose traversal feeds an analyzer-summary cache (`tom_analyzer_shared`) — the caching article.

Run the local example with:

dart run example/tom_build_base_example.dart --help
dart run example/tom_build_base_example.dart :list

For full worked tutorials, start with the tom_build_base_introduction_sample and progress through the advanced and analyzer-caching samples; the usage sections below and the doc/ guides are the inline reference.

Usage #

Defining a tool #

A ToolDefinition is a const value object. Its mode decides the calling convention:

ToolMode.singleCommand — the tool is one operation (mytool [options]).
ToolMode.multiCommand — the tool dispatches to named commands, invoked with a colon prefix (mytool :build, mytool :clean).
ToolMode.hybrid — supports both.

features selects which standard flags appear. The presets cover the common cases — NavigationFeatures.projectTool (project traversal + recursion), NavigationFeatures.gitTool (git traversal), NavigationFeatures.all, NavigationFeatures.minimal — or construct one to enable exactly the flags you want (jsonOutput, interactiveMode, dryRun, …).

const tool = ToolDefinition(
  name: 'mytool',
  description: 'Demonstrates the definition surface',
  version: '2.0.0',
  mode: ToolMode.multiCommand,
  features: NavigationFeatures.projectTool,
  commands: [
    CommandDefinition(
      name: 'build',
      description: 'Compile each package',
      aliases: ['b'],
      requiredNatures: {DartProjectFolder},
      examples: ['mytool :build', 'mytool :build --project app_*'],
    ),
  ],
);

CommandDefinition.requiredNatures is the filter that makes a command run only where it makes sense: {DartProjectFolder} means "only on folders that are Dart projects". Commands resolve by exact name, alias, or unambiguous prefix — mytool :b and mytool :bui both reach build, and an ambiguous prefix resolves to nothing rather than guessing.

Options #

OptionDefinition has three named constructors for the three kinds of option, and a .usage getter the help generator uses:

const verbose = OptionDefinition.flag(
    name: 'verbose', abbr: 'v', description: 'Verbose output');
const config = OptionDefinition.option(
    name: 'config', abbr: 'c', description: 'Config path', valueName: 'path');
const exclude = OptionDefinition.multi(
    name: 'exclude', description: 'Skip these', valueName: 'pattern');

print(verbose.usage); // -v, --verbose
print(config.usage);  // -c, --config=<path>
print(exclude.type);  // OptionType.multiOption

You rarely define the navigation options yourself — projectTraversalOptions, gitTraversalOptions, and commonOptions are contributed automatically based on the tool's features. Define options only for behaviour unique to your tool.

Deriving a tool from another #

Because a ToolDefinition is immutable, you extend one with copyWith, and the CommandListOps extension (without / replacing / plus) edits the command list functionally. This is how a specialised tool reuses a general one without inheritance:

final superTool = baseTool.copyWith(
  name: 'supertool',
  commands: baseTool.commands
      .without({'clean'})                       // drop a command
      .replacing('build', fasterBuildCommand)   // swap one out, keep its slot
      .plus([shipCommand]),                      // add new ones
);

print(superTool.commands.map((c) => c.name)); // (build, ship)
print(superTool.findCommand('sh')?.name);      // ship  (prefix match)

Running commands #

ToolRunner ties a definition to behaviour. Each command name maps to a CommandExecutor; the simplest is CallbackExecutor, which wraps a closure. The closure receives a CommandContext (the folder and its natures) and the parsed CliArgs, and returns an ItemResult:

final runner = ToolRunner(
  tool: myTool,
  executors: {
    'build': CallbackExecutor(
      onExecute: (context, args) async {
        if (!context.isDartProject) {
          return ItemResult.skipped(
              path: context.path, name: context.name, message: 'not a package');
        }
        // … do the build …
        return ItemResult.success(path: context.path, name: context.name);
      },
    ),
  },
);
await runner.run(args);

For common shapes there are ready-made executors: ShellExecutor (run a shell command per folder), DartExecutor (a Future<bool> function per folder), SyncExecutor (a synchronous callback), and ListExecutor (just enumerate).

Reading results #

Each folder yields an ItemResult — success, skipped (a deliberate, non-failing skip), or failure. ToolResult.fromItems aggregates them, and renderRunSummary() produces the uniform end-of-run block every Tom tool prints:

final result = ToolResult.fromItems([
  ItemResult.success(path: '/a', name: 'a', commandName: 'build'),
  ItemResult.skipped(path: '/b', name: 'b', commandName: 'build', message: 'no changes'),
  ItemResult.failure(path: '/c', name: 'c', commandName: 'build', error: 'compile failed'),
]);

print('success=${result.success} failed=${result.failedCount}');
print(result.renderRunSummary());

Output:

success=false failed=1
=== Skipped ===
  b :build — no changes
1 project(s) skipped.

=== Errors ===
  c :build — compile failed
1 error(s) in 1 project(s).

A skipped item is still a success — it never affects the exit code — but it is reported separately from items that did real work, so a long run ends with one readable account of what was built, what was skipped and why, and what failed.

Traversal without the full runner #

When you want the scanning and ordering machinery but not the CLI layer — say, inside another tool or a test — call BuildBase.traverse directly. You give it a traversal config and a nature filter; it scans, detects natures, filters, orders, and invokes your callback with a typed CommandContext:

await BuildBase.traverse(
  info: ProjectTraversalInfo(
    scan: workspaceRoot,
    recursive: true,
    executionRoot: workspaceRoot,
  ),
  requiredNatures: {DartProjectFolder},
  run: (ctx) async {
    final dart = ctx.getNature<DartProjectFolder>();
    print('${dart.projectName} v${dart.version}');
    return true;
  },
);

Over a workspace containing alpha and beta packages this prints:

alpha v1.2.3
beta v1.2.3

By default ProjectTraversalInfo.buildOrder is true, so packages arrive in dependency order (a package's dependencies before the package itself) computed across all scanned projects — not just the filtered subset — so ordering stays correct even when --project narrows the run. At least one of requiredNatures / worksWithNatures must be set; use {FsFolder} to match every folder.

Natures #

A nature is a capability the framework detects on a folder. One folder can have several — a Dart package inside a git repo is both a DartProjectFolder and a GitFolder. CommandContext exposes them type-safely:

if (ctx.hasNature<DartProjectFolder>()) {
  final dart = ctx.getNature<DartProjectFolder>();      // throws if absent
  print(dart.dependencies.keys);
}
final git = ctx.tryGetNature<GitFolder>();              // null if absent

Nature	Detected when	Carries
`DartProjectFolder`	folder has `pubspec.yaml`	`projectName`, `version`, `dependencies`, `devDependencies`, `pubspec`
`GitFolder`	folder has `.git/`	git repository metadata
`BuildkitFolder`	folder has `buildkit.yaml`	tool configuration presence
`ExtensionFolder`	a VS Code / tool extension folder	extension metadata

DartProjectFolder is hierarchy-aware: a Flutter package is a FlutterProjectFolder and matches DartProjectFolder, so a requiredNatures: {DartProjectFolder} filter catches every Dart project subtype.

Configuration #

Tom tools read configuration from two tiers: a workspace-root master file ({tool}_master.yaml, e.g. buildkit_master.yaml) for shared defaults, and a per-project file (buildkit.yaml) that overrides them. TomBuildConfig.load and TomBuildConfig.loadMaster read these:

# buildkit_master.yaml (workspace root) — shared defaults
navigation:
  scan: .
  recursive: true
  exclude: [.git, build]

mytool:
  verbose: false

# buildkit.yaml (inside a project) — overrides
mytool:
  verbose: true

Pipelines #

Multi-command tools get pipelines for free. A pipeline is a list of steps, each with a prefix that decides how the step runs:

Prefix	Behaviour
`shell <cmd>`	Run a shell command once
`shell-scan <cmd>`	Run the command once per traversed project
`stdin <cmd>`	Run with multiline stdin content
`print <msg>`	Print exactly one resolved message (no shell noise)
`{TOOL} <cmd>`	Delegate to one of the tool's own commands

Pipelines also support runtime macros ($name, defined on the command line and persisted per tool) and persistent defines (key/value pairs in the master and project YAML). See multiws_pipelines_macros_defines.md and modes_and_placeholders.md for the full model.

Architecture #

package:tom_build_base/tom_build_base.dart   (full surface)
package:tom_build_base/tom_build_base_v2.dart (framework only, no legacy utils)
        │
   ┌────┴───────────────── Tool definition ──────────────────┐
   │ ToolDefinition ── commands ─▶ CommandDefinition          │
   │      │                             │                     │
   │   features                      options ─▶ OptionDefinition
   │   (NavigationFeatures)                                   │
   └──────────────┬──────────────────────────────────────────┘
                  │ given to
                  ▼
            ToolRunner ── executors: { name → CommandExecutor }
                  │  parse args → route command → traverse → aggregate
                  ▼
            BuildBase.traverse
                  │ scan ─▶ NatureDetector ─▶ FilterPipeline ─▶ FolderSorter
                  ▼
            CommandContext (path + natures) ─▶ executor ─▶ ItemResult
                  │
                  ▼
            ToolResult.fromItems(...) ─▶ renderRunSummary()

Type	Role
`ToolDefinition`	Declarative description of a tool (the single source of truth)
`CommandDefinition`	One command within a multi-command tool
`OptionDefinition`	One flag/option/multi-option
`ToolRunner`	Parses args, routes to a command, drives traversal, aggregates
`CommandExecutor`	The per-folder behaviour contract (`CallbackExecutor` et al.)
`BuildBase`	Static traversal engine: scan → detect → filter → order → run
`BaseTraversalInfo` / `ProjectTraversalInfo` / `GitTraversalInfo`	Traversal configuration (project vs git mode)
`CommandContext`	Typed per-folder context with nature accessors
`FilterPipeline` / `FolderSorter`	Selection and ordering
`ItemResult` / `ToolResult`	Per-item and aggregated outcomes
`TomBuildConfig`	Two-tier YAML configuration loader

The framework holds no global mutable state across tools: a ToolDefinition is an immutable value, traversal is a pure scan over the filesystem, and a ToolRunner owns only the state of its own invocation.

Ecosystem #

tom_build_base is the build-framework member of the tom_ai/basics foundation layer:

tom_basics — exceptions, logging, the platform seam, and the runtime model.
tom_basics_console — the standalone/server platform implementation.
tom_basics_network — HTTP retry and LAN server discovery.

Downstream, the Tom CLI tools are all tom_build_base tools: buildkit, testkit, issuekit, and the code generators each ship a ToolDefinition and a set of executors and let this package supply everything else. That is the design rule for the workspace — shared CLI infrastructure lives here, never re-implemented in a tool. New capability that a tool needs is added to tom_build_base, published, and then consumed.

Further documentation #

The doc/ folder holds the in-depth guides:

build_base_user_guide.md — the complete user guide and API reference.
cli_tools_navigation.md — the navigation flag model and how traversal selection works.
modes_and_placeholders.md — modes and placeholder resolution.
multiws_pipelines_macros_defines.md — pipelines, multi-workspace runs, macros, and defines.
tool_inheritance_and_nesting.md — deriving tools (copyWith + CommandListOps) and nested tool wiring.
test_coverage.md — the test-coverage map.

See also ../README.md, the tom_ai/basics package map, and example/tom_build_base_example.dart.

Status #

Version: 2.6.25
Tests: an extensive suite under test/ (dart test / testkit :test) covering tool definition, argument parsing, help and completion generation, traversal, filtering and build-order, pipelines, macros and defines, and configuration loading.
Analysis: clean under package:lints (dart analyze — no issues).
Platforms: any Dart runtime with dart:io (desktop, server, CLI).

tom_build_base 2.6.32
tom_build_base: ^2.6.32 copied to clipboard

Metadata

tom_build_base #

Overview #

Installation #

Features #

Tool definition #

Execution #

Traversal #

Pipelines, configuration & utilities #

Quick start #

Example projects #

Usage #

Defining a tool #

Options #

Deriving a tool from another #

Running commands #

Reading results #

Traversal without the full runner #

Natures #

Configuration #

Pipelines #

Architecture #

Ecosystem #

Further documentation #

Status #

← Metadata

Documentation

Publisher

Weekly Downloads

Metadata

Topics

License

Dependencies

More

tom_build_base 2.6.32 tom_build_base: ^2.6.32 copied to clipboard

Metadata

tom_build_base #

Overview #

Installation #

Features #

Tool definition #

Execution #

Traversal #

Pipelines, configuration & utilities #

Quick start #

Example projects #

Usage #

Defining a tool #

Options #

Deriving a tool from another #

Running commands #

Reading results #

Traversal without the full runner #

Natures #

Configuration #

Pipelines #

Architecture #

Ecosystem #

Further documentation #

Status #

← Metadata

Documentation

Publisher

Weekly Downloads

Metadata

Topics

License

Dependencies

More

tom_build_base 2.6.32
tom_build_base: ^2.6.32 copied to clipboard