terradart_migrate 0.35.0

HCL → Dart migrator library behind terradart migrate. Turns Terraform modules into TerraDart Stacks via the curated catalogs.

terradart_migrate

The HCL → Dart migrator library behind terradart migrate (#80). Reads a Terraform module through terradart_hcl and rewrites what the curated TerraDart factories cover as a Dart Stack, leaving the rest in a Terraform sidecar.

Status

Alpha — same expectations as the rest of TerraDart (pin versions, read release notes). The library migrates a Terraform source tree (*.tf and *.tf.json, through terradart_hcl) into a Dart package with a leftover sidecar per directory. It reads files only: no Terraform run, no state access, nothing written into the source tree or outside --out. The user command is terradart migrate in terradart_cli. This package stays a library: the migration manifests and migrateModule / migrateTree are what the command and the gates call. The terradart-migrate executable still runs, and prints that it is deprecated.

Install

Migration runs before a Dart project exists, so install the user CLI globally:

dart pub global activate terradart_cli
terradart migrate --version

terradart lands in ~/.pub-cache/bin (add it to your PATH if dart pub global activate says so). It needs a Dart SDK — the one the migrated package needs anyway.

From a checkout: dart run bin/terradart.dart migrate --help in packages/terradart_cli.

Guide: terradart.dev — Migrating from HCL

CLI

terradart migrate --dir infra --out infra_dart

terradart migrate --dir scans module directories (every directory holding .tf / .tf.json files; hidden directories are skipped) and their roles are inferred: a directory a module block's ./ or ../ source points at is a child (migrated in child-module mode), everything else a root, and roots sharing a parent directory are environment siblings. --roots and --env-dirs override the inference. Nothing under --dir is written; --out must be empty unless --force is given.

The output is one Dart package:

Path	Content
pubspec.yaml, bin/infra.dart	lockstep pins; dart run bin/infra.dart synthesizes every Stack
lib/<dir>_stack.dart	one Stack per module directory (dev → DevStack)
lib/<dir>_module.dart	one typed ModuleCall wrapper per local module directory a module block calls (modules/cloud_run → CloudRunModule), from its variable and output blocks
lib/env.dart	with --merge-envs: the Env enum, one member per environment root
tf-out/<dir>/	each module's Terraform directory, mirroring the source tree so source = "../modules/x" keeps resolving: main.tf.json (written by synth) next to the sidecar files, plus terraform.tfvars, *.auto.tfvars and .terraform.lock.hcl copied from the source (other *.tfvars are listed for -var-file)
tf-out/<dir>/terradart_leftover.tf	resources, data sources, module calls the Stack cannot express, and moved blocks whose target stays, in Terraform, verbatim, each with its reason
tf-out/<dir>/backend.tf, variables.tf, locals.tf, outputs.tf	the terraform settings, variables, locals and outputs the Stack does not own
MIGRATION.md	the report: every module, every kept block with its reason and file, warnings, and how the environment roots differ

A single-module --dir synthesizes into tf-out/ directly. A directory where nothing translates — no curated resource, no known provider and no module call it can express — gets no Stack and stays Terraform: its sidecar files are its whole output. A root that only calls modules is a Stack with providers: [], since the child modules pin what they use. --json prints the report as JSON. Exit codes follow sysexits: 64 usage, 65 unreadable input, 73 output not empty.

--report runs the migration in memory and writes nothing (no --out; --dir defaults to the current directory): every resource / data type with how many blocks translate, how many stay in Terraform and why, the factory each maps to or not in any catalog, and the module calls whose source is outside the tree. --json prints it as JSON. MigrationCoverage.of(project) is the library form.

--merge-envs folds each group of environment siblings into one Stack instead of one per root:

// lib/app_stack.dart
enum Env {
  dev(path: 'dev', assetsName: 'app-dev-assets', backendBucket: 'app-dev-tfstate', backendPrefix: 'infra/dev'),
  prod(path: 'prod', assetsName: 'app-prod-assets', backendBucket: 'app-prod-tfstate', backendPrefix: 'infra/prod', isProd: true);

  const Env({required this.path, required this.assetsName, required this.backendBucket, required this.backendPrefix, this.isProd = false});

  final String path;
  final String assetsName;
  final String backendBucket;
  final String backendPrefix;
  final bool isProd;
}

final class AppStack extends Stack {
  AppStack({required this.env})
    : super(providers: [const GoogleProvider()],
            backend: GcsBackend(bucket: env.backendBucket, prefix: env.backendPrefix)) {
    final region = variable<String>('region');
    add(GoogleStorageBucket(
      'assets',
      name: .literal(env.assetsName),   // "app-dev-assets" / "app-prod-assets"
      location: region,
    ));
    if (env.isProd) {
      add(GoogleStorageBucket(
        'backups',
        name: .literal('app-prod-backups'),
        location: region,
      ));
    }
  }

  final Env env;
}

Every top-level argument the roots write differently — a resource argument, a module call input, a variable default or description, a provider argument, a backend argument — becomes a constant on the generated Env enum, typed as the argument takes it (an enum-valued argument becomes a typed member, BucketStorageClass.nearline, and lib/env.dart imports its barrel). The enum also carries each environment's path (tf-out/<path>) and a flag per group of blocks only some roots declare. dart run bin/infra.dart writes every environment; --env dev writes the ones of that name. A guarded local another guarded block reads is declared outside the if, null in the other environments (final backups = env.isProd ? add(...) : null;), and the reading block tests it (if (env.isProd && backups != null)).

The merge is refused — leaving one Stack per root, with the reason in MIGRATION.md — when the roots differ in anything the enum cannot hold: a value that is not a plain scalar (a nested block, a list, a reference, an interpolated string), a sensitive variable's default (never copied into Dart), different providers or backends, blocks declared in a different order, or a root where nothing translates. What merging never changes is the plan: tool/migrate_fixture_gates.dart proves the merged Stack synthesizes, per environment, exactly the JSON one Stack each did.

--lift-workspace turns terraform.workspace into a workspace parameter on the Stack, so dart run bin/infra.dart --workspace prod synthesizes for one workspace by name: a bare reference becomes .literal(workspace), a template around it becomes Dart interpolation (.literal('orders-$workspace')), and one inside a list or map becomes the value. A template mixing it with another reference stays a Terraform expression, with a warning. Off by default: the synthesized JSON then names a workspace instead of leaving ${terraform.workspace} for terraform workspace select — faithful for the workspace it names, and only that one.

Child-module mode registers providers without configuration (synth emits only required_providers), turns variable into variable<T>(...) and output into addOutput, and keeps provider configurations or a backend found in the module in the sidecar. The root's module call becomes addModule(...), whose source keeps pointing at the child's directory in the mirrored tf-out/ tree, so plan addresses keep their module.<name>. prefix. Each root plans with No changes: terradart plan --env dev synthesizes the package and plans tf-out/dev.

Library

import 'dart:io';

import 'package:terradart_hcl/terradart_hcl.dart';
import 'package:terradart_migrate/terradart_migrate.dart';

void main() {
  final module = loadTfModule(Directory('infra/dev'));
  final result = migrateModule(module, name: 'dev');

  result.files['lib/dev_stack.dart'];            // final class DevStack extends Stack { ... }
  result.files['bin/infra.dart'];                // synth entry point
  result.files['pubspec.yaml'];                  // lockstep pins on terradart_core + the provider packages used
  result.files['tf-out/terradart_leftover.tf'];  // the sidecar: what stays in Terraform, verbatim
  print(result.report.renderText());             // what became Dart, what stays in Terraform and why

  // A whole tree, as the CLI does it:
  final project = migrateTree(scanModuleTree(Directory('infra')), name: 'infra');
  project.files;   // every Stack and module wrapper, bin/infra.dart, pubspec.yaml, tf-out/**/sidecars, MIGRATION.md
  project.copies;  // tfvars and lockfiles to copy next to each main.tf.json
}

Translation is resource-atomic. A resource whose arguments all translate becomes a curated factory call; one untranslatable argument keeps the whole block in Terraform, listed in report.kept with the reason — nothing is dropped silently. A kept block the Stack still reads is declared with addExternalBlock('<address>'), so synth accepts the reference to a block the sidecar holds. Resource addresses are preserved (localName is the Terraform name), so a migrated Stack plans with No changes once the leftover blocks sit beside its main.tf.json.

What translates (the conversion rules of #655):

• literals (.literal(...), ${ / %{ re-escaped), enum members from the manifest (.postgres15), typed nested helpers (single, repeated, exactly-one-of variants), opaque passthrough maps;
• a reference in an argument that names another resource (RefTo<C>) as x.ref (x.ref.pinned('attr') when it reads another attribute than the argument emits); other references to migrated resources and data sources as typed x.id (or TfRef.attribute<T>(...) when the wrapper has no getter), var.x as .variable, module.x.out as a TfRef on the call (see below), a bare terraform.workspace as .workspace(), everything else — templates, function calls, conditionals, local.x — verbatim as .expression on any TfArg-typed argument (string, number, bool, enum, list or sensitive), the variables inside it declared like references;
• depends_on, lifecycle and timeouts (const TfTimeouts(create: Duration(minutes: 30), ...)), terraform.required_version, backend "gcs" | "local" | "s3" — a partial configuration (backend "gcs" {}, for terraform init -backend-config) included — provider blocks of the five providers (and time; default_tags, assume_role, ignore_tags and endpoints included for aws, user_project_override included for google / google-beta, credentials dropped; an argument the class does not model keeps that configuration in the sidecar, registered with addExternalProvider) — aliased ones included, registered as final googleEuProvider = addProvider(GoogleProvider(alias: 'eu', ...)) and selected per resource as provider: googleEuProvider (provider = google-beta on a GA type works the same way; a child that only receives the alias registers addConfigurationAlias(const GoogleProvider(alias: 'eu'))) — variable blocks as variable<T>(...) handles (type derived from T, spelled out as type: .object({...}) where Dart cannot say it), single-attribute outputs as addOutput;
• module calls: a call into a local directory of the tree uses the typed wrapper generated from that module's variable and output blocks (CloudRunModule('cloud_run_bff', source: '../modules/cloud_run', name: .literal('app-bff')), bff.serviceName), everything else a bare ModuleCall with source / version verbatim and an untyped inputs map, whose values spell out TfArg.literal(...) because an Object? value has no type to resolve a dot shorthand against; module.x.out reads like a resource attribute, and the call is ordered with the blocks around it;
• a literal count / for_each unrolled into one resource per instance (google_pubsub_topic.t[0] → google_pubsub_topic.t_0, google_pubsub_topic.t["eu"] → google_pubsub_topic.t_eu): count.index / each.key / each.value substituted, every reference in the module — indexed, splat or bare — pointed at the new addresses (in blocks that stay in Terraform too), and a moved entry per instance (addMoved) so the plan shows moves only; the module's own moved blocks follow their targets into the Stack;
• blockers, always with a reason: types outside every catalog, a count / for_each that is not a literal (its instances cannot be known without evaluating it) or one on a module call (whose instances are addressed module.x[0]), dynamic / provisioner, a timeouts key that is not a Terraform operation or whose value is not a duration string, a provider = x.alias the module does not configure (a child module that does not configure it declares configuration_aliases instead; a provider block inside the child stays in Terraform, with the resource that selects it), an argument with no Dart parameter — an input the called module does not declare included, a non-literal source, an expression inside a typed collection (a List<int> element, say) or on a bare non-TfArg parameter, a sensitive literal (never copied), a depends_on on a resource that stays in Terraform.

Round-trip gate

tool/migrate_roundtrip_gates.dart migrates every quickstart's synth output back to Dart, analyzes the generated Stacks and re-synthesizes them: synth(migrate(synth(S))) == synth(S), byte-for-byte after JSON canonicalization. Strict examples must round-trip completely; tool/migrate_roundtrip_debt.yaml ratchets the reasoned exceptions. It runs in tool/agent_verify.sh (full mode) and as the CI migrate round-trip gate job.

Moved gate

tool/migrate_moved_gates.dart is the acceptance check for count / for_each unrolling: it migrates test/fixtures/moved_state/ — a count resource, a for_each resource, references to their instances, an output over them and a moved block of its own — synthesizes the Stack, puts the fixture's state.json (a terraform.tfstate of the indexed instances as Terraform recorded them) next to the synth output, and runs terraform plan -refresh=false: the plan must be moves only, nothing created, changed or destroyed. It runs in tool/agent_verify.sh (full mode) and as the CI migrate moved gate job.

tool/migrate_fixture_gates.dart is the end-to-end acceptance: it migrates the tree fixtures config_tree/ (two environment roots over six local modules, which migrates completely — module calls included), real_plan_src/ (a root with a child) and lanes/ (one module per provider, the mixed pairs, and an aliased child), analyzes and synthesizes the generated package, and runs terraform validate in every directory of the mirrored tf-out/ tree (agent_verify.sh full mode, CI migrate fixture gate). A child that only declares configuration_aliases is validated through its root. It then migrates config_tree/ again with --merge-envs and requires that the one ConfigTreeStack(env: ...) synthesizes, per environment, exactly the JSON the two separate Stacks did — the proof that folding the roots together changes the Dart and nothing else. test/golden/ pins config_tree, its merged form and real_plan_src; UPDATE_GOLDENS=1 dart test test/golden_test.dart regenerates those.

Migration manifests

One MigrateManifest per curated catalog, generated into lib/src/manifest/ by terradart wrap --migrate-manifest from the same inputs (schema IR, override YAML, emitted wrapper source) the factories are generated from — so the recipe cannot drift from the generated Dart API:

Constant	Package	Source lane
googleMigrateManifest	terradart_google	hashicorp/google
googleBetaMigrateManifest	terradart_google_beta	hashicorp/google-beta
appwriteMigrateManifest	terradart_appwrite	appwrite/appwrite
cloudflareMigrateManifest	terradart_cloudflare	cloudflare/cloudflare
awsMigrateManifest	terradart_aws	hashicorp/aws

Each manifest lists, per curated factory (MigrateEntry), how every constructor slot maps back to Terraform (MigrateSlot: a TfArg scalar, an enum, a typed nested helper, a sealed exactly-one-of choice, an opaque passthrough, or manual with a reason), the package-wide helper-class and enum tables those slots refer to by name, and the output-attribute getters. allMigrateManifests lists the five in lookup precedence (a google_* type is looked up in terradart_google before terradart_google_beta), then the hand-written terradart_time manifest (time_sleep, lib/src/manifest/time.dart); findMigrateEntry and MigrateManifest.entryFor resolve a Terraform type.

The runtime types are hand-written (lib/src/migrate_manifest.dart); the *.g.dart values are regenerated by the five terradart wrap --check lanes (tool/agent_verify.sh, CI wrap_check) and fail the gate when stale. Shapes the generator cannot derive are recorded as manual and gated by terradart lint-override (migrate-shape-underivable) with tool/migrate_manifest_debt.yaml as the reasoned escape hatch.

import 'package:terradart_migrate/terradart_migrate.dart';

void main() {
  final hit = findMigrateEntry('google_pubsub_topic', CatalogKind.resource)!;
  print('${hit.manifest.package}: ${hit.entry.className}'); // terradart_google: GooglePubsubTopic
  for (final slot in hit.entry.slots) {
    print('${slot.tfName} -> ${slot.dartName} (${slot.kind.name})');
  }
}

Development

Source lives under packages/terradart_migrate/; publish.yml publishes it to pub.dev with the rest of the workspace. Regenerate a manifest with the matching wrap lane, e.g. for GA google:

cd packages/terradart_codegen && dart run bin/terradart.dart wrap \
  --provider hashicorp/google \
  --source test/fixtures/wrap/source \
  --output ../terradart_google/lib/src \
  --migrate-manifest ../terradart_migrate/lib/src/manifest/google.g.dart

See tool/providers.yaml for the other lanes' coordinates.