tom_specs_clitool 0.12.0
tom_specs_clitool: ^0.12.0 copied to clipboard
CLI tools for TomSpecs model analysis, validation, and SOM code generation.
tom_specs_clitool — the TomSpecs generator and gate suite #
Cross-references.
tom_specs_model/doc/som_multiplatform_spec_model.mdis the authority for the Spec Object Model this package generates — naming, versioning and the committed artefacts (SOM §4), what the generator produces (SOM §5), the two access paths (SOM §6) and the generated surfaces themselves (SOM §8).tom_specs_model/doc/som_generator_config.mdowns thetom-spec-object-modelconfig grammar in full;tom_specs_model/doc/som_toolchains.mdowns the per-language build and verify toolchains;tom_specs_model/doc/tom_specs_model_rules.mdowns the model-authoring rules and the structural invariants the static validator here enforces; andtom_specs_model/doc/index.mdowns the§citation convention the gates in this package resolve. This README is the catalogue of what the package ships and how to run it; those documents own what it must produce and why.
CLI tools for TomSpecs model analysis, validation, and SOM code generation.
Where this fits #
TomSpecs is a method for building software from structured specification
documents: a project is written up as a set of typed documents — the
Specification Object Model, or SOM — and the code skeleton is generated from
them. Those documents have to be readable and editable from whatever language a
consuming system happens to be written in, which means the model cannot live
only as Dart classes. tom_specs_clitool is the generation host that closes
that gap: from the single source of truth — the annotated Dart model in
tom_specs_model — it emits, per language, a lossless
meta-data file, DocSpecs and YAML schemas, and a typed document-editing facade
layered over a hand-written generic runtime.
It exists as its own package because the model must not depend on the machinery
that reads it. tom_specs_model is annotated data with no analyzer dependency
and no tools; everything that inspects it — the analyzer-backed reader, the
nine emitter triples, the static validator, the outliner, the JSON exporter and
the four correspondence gates — lives here. So this package sits directly above
the model and below everything else in the TomSpecs tree: the nine
tom_som_<lang>_runtime / tom_som_<lang>_v0 pairs are its output, and the
editor, the reviewer and tom_spec_engine all consume that output rather than
the model itself.
Overview #
The generated component is split into a fixed, hand-written runtime and a generated, typed facade — one project each, per language:
| Layer | Project | Authored | Role |
|---|---|---|---|
| Generic runtime | tom_som_<lang>_runtime |
by hand | Memory representation (SpecDocument), the meta-model "reflection" classes (SpecModel & friends), validation, and YAML/Markdown load-save. No version suffix. |
| Typed facade | tom_som_<lang>_v0 |
generated | Typed, code-completed document-editing API (D00SolutionBlueprint etc.) over the runtime's memory representation. Carries the version suffix (_v0, _v1, …). |
Both expose the same document through two parallel access paths (SOM §6): the type-safe path (the generated classes) and the generic / meta-model path (the runtime + the meta-data file). The typed path is optional ergonomics; the generic path alone can read, write, validate, load, and save any document.
All nine languages are shipped. Every target — Dart, Python, Java,
JavaScript, TypeScript, Go, Rust, C, and C++ — has a hand-written generic
runtime and a generated typed facade. generate_som.dart has a typed emitter, a
generic generator and a meta emitter for each; none is skipped. Dart and Python
are the reference ports; the other seven were ported from them.
The second half of the package is correspondence. A generated tree goes
stale the moment the model moves, and a documentation claim goes stale the moment
what it describes moves — silently, in both cases. So every claim this package
and its documents make is held by something that fails: a model fingerprint for
the nine packages, an invariant-id cross-check for the validator, an
ArgParser cross-check for the entrypoint table below, and three citation gates
for the ids the TomSpecs documents cite inline.
Installation #
Published to pub.dev. As a command-line tool it is normally a dev dependency:
dev_dependencies:
tom_specs_clitool: ^0.2.0
Most use is from a checkout of this workspace rather than from a package consumer, because the generator reads and writes sibling directories:
cd tom_ai/ai_build/tom_specs_clitool
dart pub get
Features #
Generation #
| Capability | Entry point | Produces |
|---|---|---|
| Nine-language SOM emission | bin/generate_som.dart |
tom_som_<slug>_v0 per configured language: typed facade, meta/spec_model.meta.json, schemas/, language-native manifest |
| Serialization-order stamping | bin/stamp_serialization_order.dart |
@SerializationOrder(n) renumbered across the model, 0-based per class in source-declaration order |
| Spec-ops registry | bin/spec_ops.dart |
tom_specs_model/lib/src/generated/spec_ops.g.dart — every class's child slots, shallow clone, yaml scalar, and each projection root's connect: binding |
| DocSpecs + YAML schemas | bin/docspecs_schema.dart, bin/docspecs_yaml_schema.dart |
The schema folders, standalone, for a single model package |
| Registry test suites | bin/generate_registry_tests.dart |
An application's registry_operations_test.dart, registry_screens_test.dart and registry_harness.g.dart, derived from its specification's registries (Phase 5) |
| CodeSpecs area catalogue | bin/codespecs_areas.dart |
codespecs_areas.json — the 27-area catalogue the nine runtimes' extract surface reads |
Model inspection and export #
| Capability | Entry point | Produces |
|---|---|---|
| Class-tree outlines | bin/outliner.dart |
A compact outline of the model from any of the 14 document roots, or from the container root |
| Class-graph export | bin/model_json.dart |
spec_model.json — the resolved graph the editor and reviewer apps browse |
| Static validation | SpecModelValidator (lib/src/validator.dart) |
Field-type rules, @ContentType compatibility, cycle detection, the content: String? override, and all seventeen tom_specs_model_rules.md §10.2 structural invariants |
Gates #
Each fails a build step rather than reporting a warning.
| Gate | Holds | Runs from |
|---|---|---|
| Model freshness | The nine generated packages against a fingerprint of the model source | test/model_freshness_test.dart, default suite |
| Invariant correspondence | tom_specs_model_rules.md §10.2's invariant list against the validator's implementation, in both directions |
test/invariant_correspondence_test.dart |
| Entrypoint options | This README's bin/ table against each entrypoint's own ArgParser, in both directions |
test/entrypoint_options_test.dart |
| Todo citations | Every cited quest-todo id resolves to exactly one open todo | bin/check_todo_citations.dart, test/todo_citations_test.dart |
| Section citations | Every § citation resolves to a real heading under index.md's convention |
bin/check_section_citations.dart, test/section_citations_test.dart |
OE- citations |
Every cited open-ends id has a register row | bin/check_oe_citations.dart, test/oe_citations_test.dart |
| Frozen locks (SCC45) | No package in tom_ai/ai_build — samples and nested fixtures included — resolves a tom_* version behind one already in the pub cache, and no sample declares a chain floor below it. The repo's own copy of the d4rt model guard, run here because this is the suite the repo runs routinely and test/ is not shipped |
test/scc45_resolution_guard_test.dart |
| Nine-plane parity | Every capability the contract names is carried by all nine language runtimes | bin/check_som_parity.dart, test/som_parity_test.dart |
| Release closure | The release-1 package set is dependency-closed | bin/check_release_closure.dart, test/release_closure_test.dart |
| Release drift | Every release member is at its published tree, or says why not | bin/check_release_drift.dart, test/release_drift_test.dart |
| Archive ignores | No release member's pub archive drops a file git tracks because a .gitignore excludes it — asked of dart pub publish --dry-run itself, since git and pub read ignore negations differently |
test/pub_archive_ignores_test.dart |
| Published tests | Every release member's own tests pass on the local tree before a publish (--preflight), and from its pub.dev archive after one |
tool/verify_published_tests.sh (a release step, not a standing gate) |
| Doc coverage | Every measured package is at or above its ratchet floor, and enables public_member_api_docs |
bin/check_doc_coverage.dart, test/doc_coverage_test.dart |
| Formatting | Every governed package is what dart format would write |
bin/check_format.dart, test/format_test.dart |
| API signatures | Every signature a governed package's API summary (doc/api/api_summary_*.md) quotes agrees with the code |
bin/check_api_signatures.dart, test/api_signatures_test.dart |
| Documentation build | Every governed package's dart doc build prints nothing, and no doc-comment link lands in a package the writer did not choose |
bin/check_dartdoc_build.dart, test/dartdoc_build_test.dart |
| Shipped script paths | Every path literal in a shipped example//bin//tool/ script resolves inside its own package |
bin/check_shipped_paths.dart, test/shipped_script_paths_test.dart |
| Spec-tree duplication | The doc lines stated in both spec-tree surfaces, against a committed baseline that may shrink and never grow | bin/check_spec_tree_duplication.dart, test/spec_tree_duplication_test.dart |
| CodeSpecs derivation | A generated CodeSpecs trio against the codespecs_derivation_contract.md §6 checks |
bin/validate_codespecs.dart |
| Spec document | One project's *.docspecs.yaml or rendered *.md on three tiers: decode, DocSpecs schema, instance |
bin/validate_spec.dart |
| Step-folder answers | A review round's answers, comments, open questions and inline marks as answers.md |
bin/extract_answers.dart |
Quick start #
From the tom_specs_clitool package root:
# Regenerate all nine language packages from the annotated Dart model.
dart run bin/generate_som.dart
# → ../tom_som_{dart,python,java,javascript,typescript,go,rust,c,cpp}_v0
# each with lib/ (or src/), meta/spec_model.meta.json, schemas/, manifest
# Re-render the committed outlines, then run the three citation gates.
./tool/regenerate_outlines.sh
# → 16 outlines under ../tom_specs_model/generated-doc/outlines/
# → exits non-zero if any cited todo id, § section or OE- id has gone stale
Both are idempotent: a second run with an unchanged model rewrites the same bytes. Commit the diff.
Usage #
Configuration (tom_som.yaml) #
The generator reads one top-level tom-spec-object-model block. The default
config lives at tom_som.yaml beside this README. Full grammar:
../tom_specs_model/doc/som_generator_config.md; parser:
lib/src/spec_object_model_config.dart.
tom-spec-object-model:
version-label: v0 # optional, default "v0" — suffix on tom_som_<slug>_<label>
output-base: .. # optional, default "." — base for default output roots,
# resolved RELATIVE TO THE CONFIG FILE'S DIRECTORY.
document-roots: # optional — absent/empty ⇒ generate ALL 14 document roots
- D00SolutionBlueprint
- D01CurrentLandscapeAssessment
languages: # required, non-empty
- dart # short form ⇒ default output root
- python
- language: java # map form ⇒ explicit output override
output: custom/java/tom_som_java
- c++ # alias-aware; resolves to slug "cpp"
| Key | Type | Default | Meaning |
|---|---|---|---|
version-label |
String | v0 |
Version suffix on generated project names (_v0, _v1, …). |
output-base |
String | . |
Base dir for default per-language output roots, resolved relative to the config file. The shipped .. places generated projects as siblings of tom_specs_clitool under tom_ai/ai_build/. |
document-roots |
List<String> | all 14 roots | Document root type names to generate; empty/absent means every root. |
languages |
List | (required) | Generation targets, in order. Each entry is a token string or a {language, output} map. |
Language tokens are case-insensitive and alias-aware (js→javascript,
ts→typescript, golang→go, rs→rust, c++/cxx→cpp, py→python).
Each resolves to a package-safe slug used in the project name
tom_som_<slug>_<version-label>. The parser rejects an empty/unknown/duplicate
language, a wrong-typed value, a non-string document-roots entry, or a
document lacking the tom-spec-object-model block.
Running the generator #
# Generate every configured language target from the default config:
dart run bin/generate_som.dart
# Use a specific config and override key paths:
dart run bin/generate_som.dart \
--config tom_som.yaml \
--model ../tom_specs_model \
--runtime ../tom_som_dart_runtime \
--py-runtime ../tom_som_python_runtime
| Option | Default | Purpose |
|---|---|---|
-c, --config |
<clitool>/tom_som.yaml |
The tom-spec-object-model config YAML. |
--model |
<ai_build>/tom_specs_model |
The source model package to read. |
--runtime |
<ai_build>/tom_som_dart_runtime |
Dart runtime package the generated pubspec depends on. |
--py-runtime |
<ai_build>/tom_som_python_runtime |
Python runtime the generated manifest depends on. |
--java-runtime |
<ai_build>/tom_som_java_runtime |
Java runtime the generated manifest depends on. |
--js-runtime |
<ai_build>/tom_som_javascript_runtime |
JavaScript runtime the generated manifest depends on. |
--ts-runtime |
<ai_build>/tom_som_typescript_runtime |
TypeScript runtime the generated file: dep points at. |
--go-runtime |
<ai_build>/tom_som_go_runtime |
Go runtime the generated go.mod replace targets. |
--rust-runtime |
<ai_build>/tom_som_rust_runtime |
Rust runtime the generated Cargo.toml path dep targets. |
--c-runtime |
<ai_build>/tom_som_c_runtime |
C runtime the generated Makefile RUNTIME_DIR targets. |
--cpp-runtime |
<ai_build>/tom_som_cpp_runtime |
C++ runtime the generated Makefile RUNTIME_DIR targets. |
--model-version |
major component of the model version | Override the integer model-version stamp. |
-h, --help |
— | Usage. |
Only the runtime options for languages actually listed in the config's
languages block matter; the rest keep their defaults and are unused.
Version stamp. The integer model-version and the human label are read from
the model's lib/src/version.versioner.dart (version, buildNumber,
gitCommit, buildTime). The buildTime is reused as the meta-data's
generatedAt, so re-running the generator without rebuilding the model produces
a byte-identical tree (idempotent; SOM §4.3). The committed artefacts are the
authoritative output — consumers never run the generator (SOM §4.3).
The generator only writes the module/
lib,meta/,schemas/, and the manifest; it never deletes files. Hand-authoredtest/,example/, andexamples/directories survive regeneration.
Freshness gate. Regenerate after every change to tom_specs_model — the
meta is lossless, so even a reworded doc comment or an added annotation argument
changes all nine packages. This is enforced, not remembered: a canonical run
writes tool/model_surface.stamp.json with a fingerprint of the model it read,
and test/model_freshness_test.dart fails in the default suite when the
model has moved since. See
doc/generating.md
for what the fingerprint covers and what it deliberately ignores. Commit the
stamp together with the regenerated artefacts.
The same run also regenerates tom_specs_model's own
lib/src/generated/spec_ops.g.dart, the SpecClassOps registry — it is
generated from the same model by the same reader, so it goes stale at the same
moment, and one stamp certifies both.
Member serialization order. stamp_serialization_order.dart --package ../tom_specs_model rewrites the model source to pin each member's on-disk
emission order (0-based, per class, source-declaration order). The ordinal flows
through ModelReader → ModelJsonExporter into the meta-data, so every language
serialises members in the authored order. Re-run it after any model edit that
adds, removes, or reorders fields; it is idempotent (old annotations are stripped
and renumbered).
Entrypoints #
Related entrypoints in bin/. Each parses its arguments with package:args
and handles --help and its exit codes itself; the package is
the workspace's recorded exception to building CLIs on tom_build_base,
because these are development gates over this repository and tom_build_base
would widen a release member's dependency closure.
The Options column is the entrypoint's whole input surface bar --help, and
test/entrypoint_options_test.dart holds it against each file's own ArgParser
in both directions — an option added without a mention here fails, and a flag
named here that no longer exists fails. It is enumerated rather than summarised
because the two rows that drifted did so by summarising: one described nine
inputs as "two", and a reader who trusted it ran the tool over four checks it
silently skipped.
| Entrypoint | Purpose | Options |
|---|---|---|
generate_som.dart |
Generate the per-language tom_som_<slug>_<label> projects ("Running the generator" above). |
--config --model --model-version --runtime --py-runtime --java-runtime --js-runtime --ts-runtime --go-runtime --rust-runtime --c-runtime --cpp-runtime |
model_json.dart |
Export the resolved meta-data class graph alone. Refresh either committed asset with --target editor / --target reviewer — the target owns both the path and the version stamp, which the two assets pin differently (tom_specs_model/doc/tom_specs_model_meta_schema.md, "Refreshing the committed assets"). --package + --output is for ad-hoc exports elsewhere, and --model-version / --model-label override the stamp it writes. |
--target --package --output --model-version --model-label |
outliner.dart |
Render a class-tree outline of the model from any document root. --root-type picks the root; the two --show-schema-annotations / --stop-at-detailed-in flags set how much of each class is rendered, and --max-line-length wraps it. |
--package --root-type --output --max-line-length --show-schema-annotations --stop-at-detailed-in |
check_todo_citations.dart |
Check that every quest-todo id cited inline in tom_specs_model/doc and in the cited project READMEs still names exactly one open todo. Exits 1 on a citation of closed or non-existent work, and on a bare stem that several todos answer to. --no-default-readmes scans the doc folder alone; --doc retargets it, --extra adds a file, --doc-folder adds a package doc/ tree (--no-default-doc-folders drops the default ones), --quest names the todo corpora to resolve against and --vocabulary the token allowlist; --verbose lists every resolution. Run by tool/regenerate_outlines.sh and by test/todo_citations_test.dart (see below). |
--doc --extra --default-readmes --doc-folder --default-doc-folders --quest --vocabulary --verbose |
check_section_citations.dart |
Resolve every § citation in tom_specs_model/doc against index.md's citation convention — a bare §N means this document. Exits 1 on a citation that resolves to no heading. The gate is closed: its default corpus is the doc folder, every TomSpecs README (defaultCitedReadmes), every package doc/ folder (defaultCitedDocFolders) and the comments of every citing source tree (defaultCitedSourceRoots — each package's lib/, bin/, test/ and tool/). The last two enumerate roots and discover the files beneath them, so a guide or a source file added tomorrow is gated the day it is written. Comments are read per kind: Dart's /// through a parse (a line scan reads a test fixture's string literal as a doc comment), # in shell/Python/YAML, // in the C family the eight non-Dart SOM planes are written in. --extra <file> adds one more file to that corpus, --doc-folder <dir> adds a package doc/ tree, --doc retargets the doc folder and --source the source roots, and --no-default-readmes / --no-default-doc-folders / --no-default-sources drop any default set; --verbose lists every resolution. |
--doc --extra --default-readmes --doc-folder --default-doc-folders --source --default-sources --verbose |
check_oe_citations.dart |
Resolve every OE- id cited in the editor project, the doc folder and the quest's bookkeeping against the Open-Ends Register (tom_specs_editor_specification.md §22). Exits 1 on a citation with no register row, and on a register that defines one id twice. --root / --register retarget it; --verbose lists every resolution. |
--register --root --verbose |
check_format.dart |
Report every governed Dart file dart format would rewrite, against the committed manifest (tool/format_set.yaml). Shells out to the installed dart format rather than linking package:dart_style: a pinned formatter and the SDK's binary drift apart at every SDK bump, and a gate asserting a different style from the command developers run is worse than none. Reports, never repairs — dart format <path> is the repair, and a gate that rewrote files would edit a branch at a moment nobody chose. Enumerates the files itself so *.b.dart can be excluded: tom_spec_engine's generated D4rt bridge is 973,181 lines and costs five minutes against four seconds for everything else. test/format_test.dart runs it in the default dart test. Exits 1 unformatted, 2 could not run. |
--manifest --container-root |
check_shipped_paths.dart |
Check that every relative path literal in a shipped script of the nine publishable Dart packages resolves onto something that exists inside that package. The defect it exists for: tom_som_dart_v0 published six examples of which three could not run, each reading a document from the unpublished tom_som_conformance; four tool/ scripts and one shipped test/ file had the same fault, and it was found twice by accident before it was found on purpose. Both resolution bases are tried — File('x') resolves against the cwd (the package root under dart run/dart test) and Platform.script.resolve('x') against the script's own directory, and which applies is not visible from the literal — so a literal is accepted when either lands inside the package. Scope is example/, examples/, bin/ and tool/, minus anything a .pubignore takes out of the archive; test/ is deliberately excluded because a workspace tool's tests are legitimately workspace-bound. Comments and import/export directives are skipped. --package repeats to retarget, --container-root relocates, --verbose lists every scan. Exits 1 naming each file, line and literal. test/shipped_script_paths_test.dart runs it in the default dart test. |
--package --container-root --verbose |
check_spec_tree_duplication.dart |
Report every doc-comment line over 45 characters stated in both tom_specs_reviewer/lib and tom_forge/tom_specs_editor/lib, against the committed baseline (tool/spec_tree_shared_docs.txt). The two apps render the same class graph through parallel widget trees, and the hazard is asymmetric: the reviewer is documentation-gated at 100% with public_member_api_docs enabled and the editor is out of documentation scope (tom_specs_documentation_standard.md §6), so a forked explanation is maintained on one side only and drifts on the other with nothing saying so. A ratchet, not a "must be empty" check: some duplication is the intended end state — when a rule moves to tom_som_dart_runtime and both trees cite it, the citation reads the same in both — so demanding zero would push a writer to paraphrase one of the two apart, which is the drift being prevented. The set may shrink and never grow; --record re-records it, and a shrink is reported with the lines to re-record. test/spec_tree_duplication_test.dart runs it in the default dart test. Exits 1 on a new shared line, 2 if either tree is missing. |
--record --baseline --container-root |
check_doc_coverage.dart |
Check public-API dartdoc coverage against the committed manifest (tool/doc_coverage_manifest.yaml), which transcribes the tom_specs_documentation_standard.md §5 bars — 95% for the libraries, tools and SOM runtimes, 90% for tom_specs_reviewer. Parses every lib/**.dart with the analyzer (not a regex: the campaign's own scanner needed twelve corrections before it agreed with the lint) and counts each public declaration and public member, excluding what public_member_api_docs excludes — synthetic members, @override members, and the setter half of a getter/setter pair. Ratchet only: a floor may rise and never fall, and --raise refuses to lower one, reporting a held floor as a regression rather than as approval. Also fails a package that does not enable the lint, and any entry for an exempt (generated tom_som_*_v0) or excluded (tom_specs_editor) package. test/doc_coverage_test.dart runs it in the default dart test. |
--manifest --root --report --raise |
check_api_signatures.dart |
Hold every signature a governed package's API summary quotes to the code (tool/api_signature_set.yaml). Resolves every public library under the package's lib/ (generated *.b.dart bridges aside) with the analyzer and compares each quoted member with its element: that it exists, const / factory / static, its return type, each parameter's kind, type and default. Types are compared structurally, so a typedef, a raw generic or a function type with parameter names is the type it expands to. A fence that does not parse as declarations is a fault unless it parses as statements, which makes it a usage sample. A package joins the set only once it reads zero; --package surveys one first. test/api_signatures_test.dart runs it in the default dart test. Exits 1 on a disagreement, 2 when nothing was compared. |
--manifest --container-root --package |
check_dartdoc_build.dart |
Build each governed package's API documentation (tool/dartdoc_build_set.yaml) and fail on any dart doc warning, and on any link in doc-comment prose into a package that is neither the writer, the SDK, a direct dependency nor re-exported by one — dart doc binds an unresolvable name to whatever package answers, without a warning. Builds four packages side by side, reads the rendered pages, deletes them. A package joins the set only once it is silent; --package surveys one first. test/dartdoc_build_test.dart runs it in the default dart test. Exits 1 on a warning, error or stray link, 2 when a build failed without a verdict or rendered no doc prose. |
--manifest --container-root --package |
check_som_parity.dart |
Report a capability the shared contract names that some language plane does not carry — the question the conformance corpus cannot ask, since it proves the nine ports agree about what they SHARE and is blind to what only one of them has. The vocabulary is the lowerCamelCase tokens of the conformance corpus filtered to what the reference plane (Dart) implements: taking it from Dart's sources alone reported 185 one-plane tokens, almost all local names, and a check whose first run reports 185 things is one nobody runs twice. Matching is by containment over normalised identifiers, which folds enumValueUnknown / ENUM_VALUE_UNKNOWN / EnumValueUnknown together and folds C's prefixed spelling (som_doc_add_list_item contains addlistitem) — without it, 26 of the first run's 31 findings were C being C. Acknowledged asymmetries live in tool/som_parity_exceptions.yaml, and an entry covers only the planes it measured, so the same gap spreading to one more plane goes red. An entry that outlived its gap is reported too. Exits 1 on either. test/som_parity_test.dart runs the same walk in the default dart test. |
--manifest --container-root --verbose |
check_release_closure.dart |
Check that the release-1 package set (tool/release_set.yaml) is dependency-closed: no member reaches, directly or transitively, a workspace package outside the set other than an approved published crossing — and never an excluded package (tom_spec_engine, editor, reviewer, tom_brain_*, tom_assistant*, tom_d4rt/_generator), which is red even if allowlisted. Walks dependencies transitively (continuing through approved crossings), classifies members' dev_dependencies and every dependency_overrides entry, verifies path: deps land on the manifest's directory for the name, and existence-checks the non-Dart source-only members. Exits 1 naming each offending edge. test/release_closure_test.dart runs the same walk in the default dart test run, so a dependency added later goes red rather than passing unseen. |
--manifest --container-root --verbose |
check_release_drift.dart |
Report which release-set members have moved since the commit that introduced their current version: line, and fail on any that are not acknowledged. The anchor is the version bump rather than a release tag, because these packages publish one at a time; the earliest commit that set the string wins, so a version reverted and re-applied still anchors to the first time. Each package's history is read in the repository that actually holds it — tom_core_codespecs is in a different one, and a single container-root git would report no history for it and call that clean. Drift is failed only when unpublished_drift: does not name the package: a gate that failed on any drift would be red from the first edit after every publish. A version no commit ever set is a violation, not a pass. When published_from: records the commit the current version was actually published from (a change landed between the bump and the publish), that commit is the anchor instead; a record for an older version, or a commit outside HEAD's history, is ignored. |
--manifest --container-root --verbose |
stamp_serialization_order.dart |
Re-stamp @SerializationOrder(n) on every model member in source declaration order (SOM §5.2). Run this on tom_specs_model after editing the model (--package), before regenerating; --dry-run reports what it would rewrite. |
--package --dry-run |
validate_spec.dart |
Validate one TomSpecs document — a *.docspecs.yaml or a SOM-rendered *.md — on three tiers that all run every time: decode against the model, the DocSpecs schema of its root over the markdown rendition (completeness), and validateDocument (values). The root is detected from the document, or named with --root; any of the fourteen roots. Several documents are fragments of one document: each is decoded, they are assembled — a place two fragments both write is refused as an overlap — and references are judged on the assembly; --assembled-out writes it. --markdown-out also writes the rendition; --json writes the report in the shape validate_codespecs --json uses (format spec-validation). Exits 0 clean, 1 with findings, 2 on bad usage. See doc/validating_a_document.md. |
--root --assembled-out --markdown-out --json |
extract_answers.dart |
Extract a review round's answers into a step folder's answers.md (SOM §11.9.7). It reads the answered questions and the comments of one or more documents, each a *.docspecs.yaml or a rendered *.md. Files of one root are fragments and are assembled; different roots are grouped in the order given. Each entry has its section-id chain and its line in the Markdown rendition. --open and --marks also list the open questions and the inline marks, --path limits the extraction to a subtree, --counts prints the counts, and -o writes the file. It never writes to a document. Exits 0, 1 on an unreadable document or a malformed annotation, 2 on bad usage. |
--output --path --title --open --marks --counts |
generate_registry_tests.dart |
Generate the registry part of an application's test suite from its specification (tom_specs_project_flow.md Phase 5): from the operation registry (SVOPE with AZREQ and SVOCT), the role catalogue (AZRO with its entitlements), the error codes (ERCEN) and the screens (SCREN) it writes registry_operations_test.dart (a permitted and a refused case per role, a case per custom handler, per error code, per stated idempotency, audit event and rate-limit class), registry_screens_test.dart (each screen rendered as each role, refused actions hidden) and registry_harness.g.dart, the interface the application implements in its own registry_harness.dart with createRegistryTestHarness(). The output is formatted and committed. Generation cross-checks the registries (undeclared error code, unknown role, a screen calling an unknown operation, a person-called operation no screen calls); --check writes nothing and reports a suite the registries no longer give. Exits 0 clean, 1 with findings, 2 on bad usage. |
--out --check --harness-import |
validate_codespecs.dart |
Run the codespecs_derivation_contract.md §6 checks over a generated CodeSpecs project trio. Takes --shared / --client / --server; exits 0 clean, 1 on any violation, 2 on bad usage. The trio is the pass's subject, not its only input: some checks cannot be answered from emitted code alone, and each takes its own corroborating input — --migrations (13), --cs-vocabulary (40, 41: the csFeatureHooks / csFeatureReservedPaths catalogue a @CsFeatureHook and a binding's route are held against) and with --core-source (9), the three paths --regenerated-shared / --regenerated-client / --regenerated-server (31), and --extracts (32–36), the run's generated-doc/codespecs_extracts tree. The extracts answer two different questions: the specification text the comment checks (32, 33, 34) hold the emitted doc comments against, and the routing the self-sufficiency checks (35, 36) compare against the trio's back-links, per §9.6 of codespecs_mapping.md. Give the whole extract tree or none — a partial one understates what the trio was supposed to carry, and check 35 would pass a gap it could not see. Each corroborating input is optional, and an absent one names on stdout the checks it left unrun, so a skipped check never reads as a passed one. --json writes the same result as one versioned JSON document (format codespecs-validation): the inputs, the unrun checks, the excluded areas, and per violation its check number and title, defining section, severity, file, line, the token and area where the check knows them, and the message. --backlinks writes the trio's back-link index instead (format codespecs-backlinks): every declaration with its @CodeSpec id and source tokens, its @DocSpec tokens, locus, file and line — the input for tracing generated code to spec values. The reader parses source text only, so dart compile exe bin/validate_codespecs.dart gives a standalone binary that needs no SDK or package resolution at run time. |
--shared --client --server --migrations --extracts --cs-vocabulary --core-source --regenerated-shared --regenerated-client --regenerated-server --json --backlinks |
codespecs_areas.dart |
Transcribe codespecs_mapping.md §4.1 + §4.4.3 + §4.4.6 (--mapping), plus the Stage-A gate tables of codespecs_prompt.md §4.4.2 and §4.5 (--prompt), into tom_specs_model/generated-doc/codespecs/codespecs_areas.json (--output) — the 27-area catalogue (26 active parts + CE-EN, the domainEnum member kind's extract home) the nine-runtime spec_codespecs_extract surface reads as its input. --check verifies the committed file without writing, and test/areas_catalog_test.dart runs the same comparison. |
--mapping --prompt --output --check |
docspecs_schema.dart |
Emit the DocSpecs schema for one model package (--package, required) into --out-dir, stamped with --version / --label. |
--package --out-dir --version --label |
docspecs_yaml_schema.dart |
Emit the YAML schemas into --out-dir. Split from docspecs_schema.dart above: the two emit different artefacts from different inputs, and one row for both could not say which option belonged to which. |
--out-dir |
spec_ops.dart |
Regenerate tom_specs_model/lib/src/generated/spec_ops.g.dart — the SpecClassOps registry giving every model class its child slots, shallow clone and yaml scalar, plus each projection root's connect: binding. The ad-hoc entry point: generate_som.dart produces the registry as part of the canonical regeneration, so reach for this only to write it elsewhere (--output) or to refresh it alone against a given model (--package) without a nine-language run. |
--package --output |
summaries.dart |
Build an analyzer sdk_summary.sum (and, with --package, a one-off grouped packages.sum) for a single consumer. This is not the producer of tom_specs_editor's scoped summary asset set — that set has one generator, tom_forge/tom_dart_editor_bundler, which also emits the summary_scopes.g.dart helper naming its asset keys. Here it serves --sdk-only, writing into --out-dir (see split_sdk_summary.dart below). |
--package --out-dir --sdk-only |
build.dart |
Build orchestration for the editor app. Its --generate-summaries step invokes the bundler against tom_specs_editor/buildkit.yaml rather than generating the asset set itself, so the assets and the paths the app asks for cannot disagree. --os picks the target platform, --buildkit the config, --model / --editor / --summaries the paths it reads, and --no-flutter-build stops before the Flutter step. |
--os --model --editor --summaries --generate-summaries --buildkit --no-flutter-build |
And in tool/ — thirteen entries: five scripts, one library file two of them
share, and seven data files a gate reads. A script here is a maintained entry
point, not a scratch file: one-shot census and codemod tooling is deleted once
its campaign closes, because a script that reads a shape the model no longer has
is worse than absent — it still looks runnable.
| Entry | Purpose | Re-run when |
|---|---|---|
regenerate_outlines.sh |
The drift-proof batch entry point: renders all 16 committed outlines (DocSpecsProject + D00–D13, plus the compact SolutionBlueprint) into tom_specs_model/generated-doc/outlines/, then runs all three citation gates — check_todo_citations.dart, check_section_citations.dart and check_oe_citations.dart — as blocking steps under set -e. |
Any model-shape change, and any documentation pass. Commit the diff. |
verify_published_tests.sh |
Download each release member's actual archive from pub.dev, dart pub get it in a directory with no workspace siblings, and run its shipped tests. A release step rather than a standing gate: the archive's tests are only meaningful against the archive's dependencies, and a local tree run against published siblings reports the stale-publish drift as a test failure. It therefore refuses to start while check_release_drift.dart is red (--allow-drift overrides). Downloads rather than reconstructs the archive from git ls-files minus .pubignore, because a reconstruction re-implements pub's packaging rules and would hide the untracked-fixture case that motivated this. --strict makes a skip a failure. --preflight asks the same question one step earlier: it runs each member's own dart test on the local tree, with no drift precondition, so a stale shipped test fails before the upload rather than after it. |
--preflight before every dart pub publish; the archive run after a lockstep publish. |
regenerate_api_references.sh |
The API-reference driver over all eighteen SOM packages, running the eight per-language generators into <package>/doc/api/reference/. Its shape is tom_som_conformance/tool/run_all_suites.sh's: per-target selection, and a skip with the reason stated when a toolchain is absent, with --strict turning a skip into a failure. The rendered output is gitignored (som_toolchains.md, "Documentation generation"). |
The documented sources change and a reference is wanted locally. |
provision_doc_generators.sh |
Report which of the eight documentation generators a host carries, and install the only two that are ever a documentation-only install — pdoc and doxygen. Reports by default and installs on --install; --dry-run prints the commands, --markdown emits the som_toolchains.md host table. For pdoc it reports the interpreter behind the tool and checks it for PyYAML and Python >= 3.13, because pdoc imports the module rather than parsing it, and below 3.13 it re-parses the whole module per class — so the version alone cannot tell a working install from one that dies at import or one that never finishes on a facade. |
A host is brought up for documentation generation. |
lib/doc_toolchain_path.sh |
Not an entry point: the toolchain PATH rules the two scripts above share, so the provisioner cannot certify a host the driver then skips. Resolves what a non-interactive shell hides — ~/.cargo/bin and the Go tarball, Dart when it is the SDK bundled with a Flutter checkout, and javadoc from JDK 21 by preference (Homebrew's openjdk@21 is keg-only, so a platform /usr/bin/javadoc shim for another major wins by default). |
A toolchain installs somewhere the rules do not look. |
split_sdk_summary.dart |
Turns assets/sdk_summary.sum into the committed lib/src/sdk_summary/ chunk set that analyzer_bootstrap.dart loads — the only producer of it. Pairs with bin/summaries.dart --sdk-only, which builds the .sum. |
The Dart SDK version or the analyzer major moves (tom_specs_model/doc/som_toolchains.md, "Regenerating after an SDK change"). |
format_set.yaml |
Data, not a script: the committed record of which trees dart format governs, plus the source directories checked in each and the *.b.dart suffix it does not govern. Its header carries the two decisions behind it — the style is the SDK default and deliberately unpinned (Dart 3.7 removed the short style and offers no way back), and generated Dart inside the gate obliges its generator to emit formatted output, which is why generateSpecOpsRegistry runs dart format over what it writes. |
A package joins or leaves the quest. |
doc_coverage_manifest.yaml |
Data, not a script: the committed record of the agreed documentation bar — a floor: and the tom_specs_documentation_standard.md §5 target: per measured package, plus the exempt and excluded lists. Read by bin/check_doc_coverage.dart and test/doc_coverage_test.dart. Its header states the ratchet rule and what the measurement cannot see (tom_specs_documentation_standard.md §5 discounts a comment that restates its identifier; no mechanical check can). |
A sweep raises coverage — via --raise, never by hand. |
api_signature_set.yaml |
Data, not a script: the packages whose API summaries bin/check_api_signatures.dart holds to the code, and why the others are not there. |
A package's summaries read zero — survey with --package first. |
dartdoc_build_set.yaml |
Data, not a script: the packages whose dart doc build must be silent, and why tom_som_dart_v0 is not there yet. |
A package builds silent — survey with --package first. |
release_set.yaml |
Data, not a script: the committed manifest of the release-1 package set — the nine-package Dart chain, the source-only non-Dart runtime directories, the approved published crossings each with its required reason, the unpublished_drift: acknowledgements, and the published_from: record of which commit each member was published from. .pubignored, because it describes this workspace's releases rather than anything a consumer runs. Read by bin/check_release_closure.dart, bin/check_release_drift.dart and both their tests. |
The release scope changes, a crossing is approved, or a member is knowingly left behind its published version. |
spec_tree_shared_docs.txt |
Data, not a script: the committed baseline of doc lines stated in both spec-tree surfaces. Its header states the ratchet rule and what to do when the set grows. Read by bin/check_spec_tree_duplication.dart and test/spec_tree_duplication_test.dart. |
A shared explanation is removed — via --record, in the same commit. |
model_surface.stamp.json |
Data, not a script: the model fingerprint a canonical generate_som.dart run writes, against which test/model_freshness_test.dart checks in the default suite. |
Written by the generator; commit it with the regenerated packages. |
som_parity_exceptions.yaml |
Data, not a script: the committed record of accepted nine-plane asymmetries and known unported capabilities, read by bin/check_som_parity.dart and test/som_parity_test.dart. Its header states the difference between the two kinds, which is the whole point of the file: an ACCEPTED asymmetry is the planes legitimately differing (each language spells a date in its own type system), a KNOWN GAP is a capability that really has not been ported and is written down only so it stops masking the next one. |
A gap is closed, or a new asymmetry is measured and accepted. |
todo_citation_vocabulary.txt |
Data, not a script: the token allowlist that keeps ordinary technical terms from colliding with the discovered todo-id shapes. A token list, never a path list. | A false positive appears — add the one token. |
What is generated (per language) #
Each target lands at <output-base>/tom_som_<slug>_<version-label>. For Dart:
tom_som_dart_v0/
├── pubspec.yaml # depends on tom_som_dart_runtime
├── lib/tom_som_dart_v0.dart # the typed facade
├── meta/spec_model.meta.json # lossless meta-data: every class, member, annotation
├── schemas/ # 14 DocSpecs schema folders (one per document root)
│ ├── solution-blueprint/ … └── transition-rollout-plan/
├── example/ # hand-authored samples (a/b/c) — survives regen
└── test/ # hand-authored generated-tree suite — survives regen
Every other language mirrors this shape — the same meta/, schemas/,
hand-authored samples, and tests — differing only in the facade file and the
language-native manifest that declares the runtime dependency:
| Language | Facade | Manifest | Runtime dependency mechanism |
|---|---|---|---|
| Dart | lib/tom_som_dart_v0.dart |
pubspec.yaml |
path/hosted dep on tom_som_dart_runtime |
| Python | tom_som_python_v0.py |
pyproject.toml |
dep on tom_som_python_runtime |
| Java | src/… |
pom.xml |
dep on tom_som_java_runtime |
| JavaScript | index.js (module) |
package.json |
dep on tom_som_javascript_runtime |
| TypeScript | index.ts (module) |
package.json + tsconfig.json |
file: dep on tom_som_typescript_runtime |
| Go | package source | go.mod |
replace → tom_som_go_runtime |
| Rust | src/lib.rs |
Cargo.toml |
path dep on tom_som_rust_runtime |
| C | header + source | Makefile |
RUNTIME_DIR → tom_som_c_runtime |
| C++ | header + source | Makefile |
RUNTIME_DIR → tom_som_cpp_runtime |
- Meta-data file (
meta/spec_model.meta.json) — the resolved model graph the generic runtime loads. Lossless per SOM §5.3: it carriesmodelVersion(integer major),modelVersionLabel(build stamp),containerRoot, and for every reachable class its name, doc-comment, identity annotations, and for every field its type, nullability, list/enum-ness, render classification, and all annotation arguments. - Typed facade — typed document-editing classes (see Consuming the
generated code below): the
D00SolutionBlueprintroot plus every reachable section, form, list and enum class. - Schemas — the DocSpecs schema and the
*.docspecs.yamlYAML schema, per document root.
Three class counts, and they measure different things. The meta-data
reports classCount: 1254 — the model classes, including the DocSpecsProject
container root. A generation run covers 1253 of them: the container root is
reachable from no @Document, so no run can ever emit it. That figure is what
each Som*GenerationResult.emittedClassCount reports and what
generate_som.dart prints per language, and under a document-roots: filter it
drops further. The Dart facade then declares 3989 classes — the 1253 it can
instantiate plus one small accessor class per form, named
<Owner><Form>Form, giving the form's fields typed getters. That is 2736
accessor classes, and 1253 + 2736 = 3989.
So: quote the meta's classCount for the model's size, emittedClassCount for
what a run covered, and the facade count for the generated Dart surface. The
generated READMEs quote the first and say "model classes" for it.
Consuming the generated code #
The same document is reachable three ways. (Dart shown; Python is the literal
mirror — field.element_type, kind.value, SpecModel.from_json, etc.)
Typed path — tom_som_<lang>_v0
The typed classes are an editing facade over the runtime's SpecDocument.
They are instantiated with the memory root and perform a version check at
construction time (SOM §4.2). Loading/saving is always done through the document,
never through the facade.
import 'package:tom_som_dart_runtime/tom_som_dart_runtime.dart';
import 'package:tom_som_dart_v0/tom_som_dart_v0.dart';
final doc = SpecDocument();
final sbp = D00SolutionBlueprint(doc); // instantiation-time SOM §4.2 check
sbp.content = 'The system vision …'; // typed setter → writes the store
final csa = sbp.currentLandscape; // nested section navigation
final metrics = csa.operationalMetrics; // typed SomList: add()/length/[i]/items
metrics.add().content = 'Order turnaround: 4.2 days.';
sbp.objectModelVersion; // '0.0' — which _vN surface this is
Every typed root exposes static const String modelVersion and an
objectModelVersion getter returning its own major.minor ('0.0' for the
_v0 pre-release surface). The instantiation check compares this against the
document's recorded stamp and throws SomVersionException (Dart) /
SomVersionError (Python) when an older facade is asked to edit a newer
document.
Generic path — tom_som_<lang>_runtime
No typed classes required — address sections by string path:
final doc = SpecDocument();
doc.setContent('SBP/content', 'The system vision …');
// List paths use the field's section-id (the meta-data names it):
final item = doc.addListItem('SBP/currentLandscape/CUOPME-OPER-LST');
doc.setContent('$item/content', 'Order turnaround: 4.2 days.');
doc.content('SBP/content'); // read back
final yaml = SpecDocumentYaml.encode(document: doc, modelVersion: '0.0');
final json = doc.toJson();
Load/save: SpecDocumentYaml.decode / .encode for *.docspecs.yaml, and
SpecDocumentMarkdown for the Markdown route (also the self-contained scanner —
no external DocScanner binding required).
Reflection path — meta-model introspection
Load the package's own meta/spec_model.meta.json and traverse the schema
value-free:
final meta = File('meta/spec_model.meta.json').readAsStringSync();
final model = SpecModel.fromJson(json.decode(meta));
final reflection = SpecReflection(model);
for (final root in model.roots) { /* 14 roots: type, title, sectionId, doc */ }
reflection.fieldsOf('D00SolutionBlueprint'); // each SpecField: kind, type, …
reflection.resolve('SBP/currentLandscape'); // path → model node
This is what lets a consumer modify a document generically and correctly — discover sections, field types, validation, form decomposition, and mapping targets — without compiling against the typed classes.
Versioning rules (SOM §4.2) #
- The generated typed projects carry a version-label suffix
(
tom_som_<slug>_v0). Projects are generated per major version of the model; multiple majors (_v1,_v2, …) can coexist in one codebase. - The generic runtime carries no suffix — one runtime per language, shared across majors.
- A document records the model version it was authored with; the facade's
instantiation check enforces:
- a newer facade may edit older documents of the same major, upgrading the stamp on edit;
- an older facade rejects editing newer documents of the same major;
- different majors are never editable across each other — cross-major is read/convert only.
_vNtrigger. While a major is pre-release the suffix staysv0and the typed surface may change freely between regenerations. Backward-compatibility observation — and the move to_v1— begins when arelease.mdis added to the model major.
Release procedure #
Publishing a release member is a fixed sequence, because each step answers a question the next one depends on:
- What moved.
dart run bin/check_release_drift.dartnames every member whose tree differs from what pub.dev carries; each is acknowledged intool/release_set.yaml'sunpublished_drift:until it is published. - Bump. Raise the version and add the CHANGELOG entry of every member that
moved, and of every member that depends on one in lockstep. When the model
chain moves, regenerate after the bump (
buildkit -v -p . :versionerintom_specs_model, thenbin/generate_som.dart), so the generated packages carry the new version. - Preflight.
tool/verify_published_tests.sh --preflightruns every member's owndart teston the local tree, with the conformance driver (tom_som_conformance/tool/run_all_suites.sh) and this package'sdart testbeside it. A member whose own suite fails is not published: its tests ship in the archive, and nothing else runs them before a consumer does. - Publish.
dart pub publishin each member, each after the members it depends on. - Record. Remove the published members from
unpublished_drift:, and record apublished_from:commit when the publish was made from a later commit than the bump, so the drift gate is green again. - Verify. Once pub.dev serves the new versions,
tool/verify_published_tests.sh --strictruns each archive's shipped tests against hosted dependencies.
Samples #
Each language's runnable samples live in the generated v0 package (they
survive regeneration) and are tabled in that package's own example README:
| Language | Location | Run |
|---|---|---|
| Dart | tom_som_dart_v0/example/ |
dart run example/<file>.dart |
| Python | tom_som_python_v0/examples/ |
python3 examples/<file>.py |
| Java | tom_som_java_v0/ |
see the package's example README |
| JavaScript | tom_som_javascript_v0/ |
node examples/<file>.js |
| TypeScript | tom_som_typescript_v0/ |
tsc && node examples/<file>.js |
| Go | tom_som_go_v0/ |
go run ./examples/<file> |
| Rust | tom_som_rust_v0/ |
cargo run --example <file> |
| C | tom_som_c_v0/ |
make && ./examples/<file> |
| C++ | tom_som_cpp_v0/ |
make && ./examples/<file> |
Each provides the same triplet — a_typed_access, b_generic_document,
c_reflection_metadata (the three access paths above) — building the same
document shape so the three paths visibly converge across every language.
The three citation gates #
The TomSpecs documents cite each other, the quest's todos and the editor's
open-ends register by inline id. Every such citation decays silently: what
it points at is renamed, archived or deleted, and the sentence goes on reading
as if it were still true. Three commands close that, and each is also a test, so
the corpus is held at zero violations in the default dart test run.
Todo citations. A quest-todo id is cited to say who owns an open question.
check_todo_citations.dart resolves every backticked id-shaped token in
tom_specs_model/doc — plus every TomSpecs README and every package doc/
folder (defaultCitedReadmes + defaultCitedDocFolders, the same closed sets
the section gate holds) — against
the active, archived and deleted todo files of the quests those documents cite
(defaultCitedQuests — tom_specs and tom_core).
Four things about it are deliberate:
- Id shapes are discovered, not enumerated. The stems come from the todo
files themselves, so a citation of another quest's corpus resolves rather
than going invisible — and a series nobody remembered to add to a list is
still checked. The price of a shape rule is that ordinary technical terms
collide with it;
tool/todo_citation_vocabulary.txtis that price, paid one token at a time. It is a token list, never a path list. - Two exemptions, both inline, so an exemption travels with the line rather
than blanket-exempting a file.
<!-- todo-cite: provenance -->allows a closed citation only when the same line also cites an open todo — the legitimate case is a provenance note ("<landed prerequisite>landed — restated by<open todo>"), where the raiser is history and what it raised is open. The example is written schematically on purpose: an earlier version named a real pair, and each todo that closed had to re-point it at the next one, which then closed in its turn. A<!-- todo-cite: history -->standing alone on its own line exempts the whole document, for a changelog. - A citation must name exactly one todo. Ids are
<stem>_<datecode>-<slug>and several campaigns renumber their follow-ups from 1 per prompt, using the date code to tell them apart — so a stem can carry many todos, and a bare-stem citation names all of them. That is theambiguousverdict, a violation whatever the statuses behind it: answering with the first open record would make a stem whose older todo closed and whose newer one is open read as healthy forever, while a reader following it lands on either. The report names every id the stem matched, because the fix is always to write more of the id — and a citation carrying the date code resolves exactly, against ids equal to it or extending it at a-or_boundary, so a truncated date code reads as the typo it is. Neither exemption applies: both excuse a citation of finished work, and ambiguity is not a question about status. - It checks citations, not claims. That a cited id still exists and is open is mechanical; that what the document says about it is still true is a semantic judgement and stays with a human reading pass.
The check runs from two places, because a citation goes stale from two sides: a
documentation pass trips it through tool/regenerate_outlines.sh, and a todo
archive trips it through test/todo_citations_test.dart, which runs in the
default dart test.
Section citations. The same documents cite each other's sections
far more often than they cite todos, and check_section_citations.dart resolves
those. index.md owns the convention; this is its decision procedure. A bare
§N means this document — that carve-out is what makes the rule decidable at
all, since intra-document self-reference is how the documents overwhelmingly
cite. A document name overrides it in exactly five ways: standing in front of the
citation (across a soft line wrap, and as the tail of a markdown link), standing
behind it (§N of <file>.md), inherited within a run (§N / §M / §K),
document-map table-row scope, and its transpose table-column scope — a
column headed `<file>.md` § governs the numbers beneath it. Every
illustration here is written with metavariables: a real section number in an
unqualified example would be read by the checker as a citation of a section of
this README, which has none.
Three things about it are deliberate:
- Resolution is exact.
§N.M.Kresolves only against a headingN.M.K, never through its parent. The relaxation was tried and hides real defects: it excuses a hundred citations belonging to another document, and nothing a parser can see separates those from the genuine case of a citation naming a numbered rule inside a section. So such a citation says so in words — "rule 6 of §N.M" — and a citation written as a section number has to be one. - What counts as a citation is decided by the id's shape, a dotted number or
an upper-case symbolic id (
PF-FLW-OVE), not by an exception list.§oneof,§itemand the convention's own metavariable§Nfall out for free, and a document that coins a new section-type name is covered on the day it is written. - The three narrow clauses are narrow on purpose. The run joiner admits only
separators and joining words, so a name mentioned two clauses back cannot vouch
for an unrelated citation; table-row scope fires only when the row's first cell
holds a document reference and nothing else, because a table that cites
another document in column one and its own sections in column two is not a
document map; and a table-column header must carry a trailing
§, which is what makes it say the column holds sections rather than merely mention a file.
Five verdicts come out — self, crossDocument, dangling, wrongSection,
unverifiable — of which only dangling and wrongSection fail the run. A
citation naming a document outside the scanned corpus is unverifiable, not a
defect: the checker cannot see the file, which is not the same as the citation
being wrong.
One exemption, and it is narrow by construction. A file that documents this
convention has to exhibit its syntax, and an exhibit is a metavariable rather
than a reference — nothing a parser can see tells the two apart, because they are
deliberately identical. Where a metavariable will not serve, because the point
being made is about particular numbers, the line carries
<!-- section-cite: exhibit … -->, listing the ids it exhibits without the
§ sign. The marker is line-scoped and id-scoped: it excuses exactly those
ids on exactly that line, and there is deliberately no document-level blanket,
since the file the mechanism exists for also carries real citations two lines
away. Markers inside fences are inert, and an id no citation on the line uses is
reported as a stale exemption and fails the run — an exemption that excuses
nothing is the register decaying in the other direction.
test/section_citations_test.dart fixes the rule against hand-written fixtures
and closes the gate: its last tests hold the live corpus at zero violations
and zero stale exemptions. The corpus is the doc folder, every TomSpecs
README (defaultCitedReadmes), every package doc/ folder
(defaultCitedDocFolders) and the citing source trees' /// comments
(defaultCitedSourceRoots) — which is also what
check_section_citations.dart scans by default, so the command and the gate
cover the same files and neither list can drift from the other. Anti-vacuity is
asserted per root rather than in aggregate, so a root that resolved to an
empty directory cannot hide behind the five that did not. Pass
--no-default-readmes / --no-default-sources to narrow the scan.
A Dart source is held to the same convention with one difference that decides
its whole house style: a source file declares no sections, so it is scanned
with an empty set of its own headings and the bare-§N carve-out has nothing to
resolve against. Every citation in source must therefore name its document.
OE- citations. The third gate differs from the other two in where the
citing side lives: OE- ids are cited from shipped source — comments in
tom_specs_editor's lib/ and test/, notes in its pubspec.yaml and
buildkit.yaml — as durable handles on seams and drop-in points, and they
resolve against the Open-Ends Register in tom_specs_editor_specification.md
§22. That register was once a pair of quest documents, and consolidating the
editor specification deleted them without folding them in; 71 citations across
16 ids went on resolving to nothing until someone tried one, and three
deferred.tom_specs.md entries paid for it by restating in prose what an id
meant. check_oe_citations.dart makes the next such deletion fail a build step.
Two things about it are deliberate:
- A row defines its id by carrying it in the row's first inline-code span,
and nowhere else. One place to read, and no id can be defined by accident in
running prose. In the register's own document that first token is therefore
suppressed as a citation — but every other mention in the file, including the
OE-a row's own prose cross-references, is checked normally. - It runs in one direction only: cited → defined. An id is allocated once and never reused, so the register deliberately keeps rows nothing cites any more — a retired row is what reserves its number. Checking the reverse direction would turn that invariant into a failure.
test/oe_citations_test.dart fixes the parse and match rules against fixtures
and closes the gate: its last test holds the live corpus at zero violations
over the same roots the command scans by default (defaultCitingRoots).
Architecture #
tom_specs_clitool
├── bin/ 15 entrypoints — see "Entrypoints" above
├── tool/ batch scripts, their shared lib, the data files gates read
└── lib/src/
├── model_reader.dart analyzer-backed read of tom_specs_model → ModelClass graph
├── validator.dart the STATIC tier: the model's own annotations
├── meta_tree.dart the resolved graph → the language-neutral meta tree
├── som_<lang>_generator ┐ nine emitter triples, one per language:
├── som_<lang>_emitter ├─ generator = project + manifest + packaging
├── som_<lang>_meta_emit… ┘ emitter = typed facade · meta emitter = meta/
├── packaging.dart the generated README / doc / manifest boilerplate
├── docspecs_*_generator DocSpecs and YAML schema emission
├── spec_ops_generator the SpecClassOps registry for tom_specs_model
├── serialization_order the @SerializationOrder re-stamper
├── outline_writer.dart the class-tree outline renderer
├── model_json_exporter the class graph the editor and reviewer browse
├── model_freshness.dart the model-source fingerprint behind the stamp
├── {todo,section,oe}_citations.dart · release_closure.dart · pub_archive_ignores.dart · entrypoint_options.dart
└── codespecs/ the CodeSpecs derivation-contract checks
tom_specs_model ──ModelReader──▶ ModelClass graph ──▶ meta tree
│ │
validator ◀─┘ ├──▶ 9 × typed emitter
outliner ◀─┘ └──▶ 9 × meta emitter
| Type | Responsibility |
|---|---|
ModelReader |
The one analyzer-backed read of tom_specs_model. Everything downstream works on its ModelClass graph, so no other component parses Dart. |
SpecModelValidator |
The static tier — whether the model's own annotations are well-formed. Runs at generation time only; the instance tier (a concrete document's values) is each runtime's validateDocument. |
MetaTree |
The language-neutral resolved graph. Every one of the nine languages derives from this, which is why one freshness stamp covers all nine. |
Som<Lang>Generator / Emitter / MetaEmitter |
The nine emitter triples: project + manifest, typed facade, and meta/spec_model.meta.json respectively. |
writeFacadePackaging (packaging.dart) |
The generated README.md / readme_howtointegrate.md / manifest boilerplate each tom_som_*_v0 ships. renderFacadeReadme emits the whole tom_specs_documentation_standard.md §2 template, drawing on three inputs it keeps strictly apart (tom_specs_documentation_standard.md §4.2): the shared body for what is the same in all nine, a required PackagingDescriptor field for each per-language nuance — never a branch on the language — and the emitted meta/spec_model.meta.json for what the model decides, so the document-roots table cannot drift from the package it describes. Hand-written doc/ folders survive it. |
SpecOpsGenerator |
spec_ops.g.dart — every class's snapshot/serialization contract, keyed by section id rather than Dart field name, plus each projection root's connect: binding. |
ModelJsonExporter |
The spec_model.json class graph the editor and reviewer apps browse. |
ModelSurfaceFingerprint (model_freshness.dart) |
Fingerprints the model source, never the generated output — fingerprinting the output would let a stale package match a stamp taken over its own stale self. |
TodoCitations / SectionCitations / OeCitations |
The three inline-citation resolvers, each with a closed default corpus so the command and the test cover the same files. |
ReleaseClosure |
The dependency-closure walk over tool/release_set.yaml. |
DocCoverage |
The public-API dartdoc coverage walk over tool/doc_coverage_manifest.yaml. |
EntrypointOptions |
Holds this README's bin/ table against each entrypoint's ArgParser, in both directions. |
Ecosystem #
tom_specs_core (the annotations the model is written in)
│
▼
tom_specs_model (the annotated Dart model — data only)
│ read once by ModelReader
▼
▶ tom_specs_clitool ◀ (this package: read · validate · emit)
│ emits
┌──────────────────────┼──────────────────────┐
▼ ▼ ▼
tom_som_dart_v0 tom_som_python_v0 … tom_som_cpp_v0 (9 typed facades)
│ │ │
▼ ▼ ▼
tom_som_dart_runtime tom_som_python_runtime … (9 hand-written runtimes)
│
├──▶ tom_spec_engine ──▶ tom_forge/tom_specs_editor
└──▶ tom_specs_reviewer
Further documentation #
TomSpecs subject matter — the authorities this package implements:
| Document | Authority for |
|---|---|
| index.md | The catalogue of the whole TomSpecs document set, and the § citation convention the gates here resolve |
| som_multiplatform_spec_model.md | The nine-language generation, the metadata tree, the generated/runtime/meta classes, serialization, schema generation, the embedded validator and language packaging |
| som_generator_config.md | The tom-spec-object-model config block — languages, output roots, version label, document roots |
| som_toolchains.md | The per-language build and verify toolchains, and running the Dart host with no installed SDK |
| tom_specs_model_meta_schema.md | The on-disk schema of the emitted spec_model.meta.json, its two version stamps, and the committed-asset refresh procedure |
| tom_specs_model_rules.md | The model-authoring rules, the seventeen structural invariants the validator enforces, and the outliner |
| codespecs_derivation_contract.md | What bin/validate_codespecs.dart checks, and why each corroborating input is needed |
| codespecs_mapping.md | The parts catalogue bin/codespecs_areas.dart transcribes |
This package — its own guides:
| Guide | Covers |
|---|---|
| doc/index.md | The catalogue of this package's documentation |
| doc/generating.md | The canonical regeneration, stamping, the ad-hoc entry points and the freshness stamp |
| doc/inspecting_the_model.md | The outliner, the JSON exporter, and reading an outline's notation |
| doc/gates.md | Running each gate, reading a citation failure, and why the scan sets are closed |
| doc/api/api_summary_index.md | The per-module API summaries |
Siblings — packages you will reach for next:
| Package | What it is |
|---|---|
| tom_specs_model | The annotated Dart model this package reads — and the home of all TomSpecs subject-matter documentation |
| tom_specs_core | The annotation library the model is written against |
| tom_som_dart_runtime | The hand-written generic Dart runtime the emitted facade layers over |
| tom_som_dart_v0 | The generated typed Dart facade — the reference output of this generator |
| tom_som_conformance | The cross-language harness that holds all nine ports to the same behaviour |
| tom_spec_engine | The scripting and agent plane built on the generated Dart pair |
Status #
Version 0.11.0, published to pub.dev.
1364 tests, all passing except the release-drift test in the window
between a model edit and its publish. The skips are toolchain-gated — a suite that
needs a language runtime this host does not have reports as skipped with the
reason stated rather than as a red suite. Run them with dart test or
testkit :test.