matrix library

The matrix of apps that the continuous integration of SMF generates with smf create and analyzes with Flutter: every app that the contract harness builds for a set of modules, and the apps with every module, of which CI also builds a covering for Android and iOS and starts it on devices, one app for each job of the plan of CI; those apps once more for the other values of the mode options of the roles, which CI only analyzes and tests; the apps whose tests must fail, which show that the tests can fail; and the versions of Flutter that its nightly run checks them with.

It serves the repository of SMF, and its API may change in any release.

Classes

EveryModuleAppPerProvider
One app with every module for each provider of role: the first of the apps that has it, which has the first providers of the other roles that fit with it.
EveryModuleApps
The apps with every module that createEveryModuleApps generates: those of selection, each of them by default, among the apps with every module, or, with withoutExternalSteps, among those without the modules whose steps need an external service (see everyModuleAppsOf).
EveryModuleSelection
Which apps with every module a run of the matrix takes, of those that everyModuleAppsOf gives, one for each combination of the providers of the roles that take one: a covering of the combinations (EveryModuleCombinations), one app by its name (NamedEveryModuleApp), or one app for each provider of a role (EveryModuleAppPerProvider).
FlutterRelease
A stable release of Flutter for a FlutterSystem.
FlutterVersion
A version of Flutter that the nightly run of CI checks the apps with.
MatrixApp
An app of the matrix: the modules to ask for, which name every module of the app so that no question is left, and the options of its roles.
MatrixAppTest
Tests that the matrix adds to the apps it generates and runs with flutter test: the files of a directory for the apps that appliesTo accepts.
MatrixAppTests
The tests that a matrix adds to its apps, and the roles whose contract they must check with every provider.
MatrixCiBuild
What builds an app in its CI, for one environment; see MatrixCiCommands.
MatrixCiCommands
What the CI of an app runs; see MatrixCi.
MatrixCommands
The commands that runMatrix and createEveryModuleApps run for each app, and where they write what happens, which tests of the matrix may replace; each left null is the real one.
MatrixEnv
How the apps of a matrix get the values that their code reads, which the developer of an app fills in after generating it, such as the public key of a service: the file of an app with them, and the toolArguments with which the commands of the flutter tool that build, run or test an app give it those values.
MatrixExpectedFailure
A test of an app of the matrix that must fail, and a reason that its first failure must give, such as a test of a role in an app whose provider of the role has a known bug.
MatrixFailingApp
An app whose tests must fail, and how: of the apps with every module of the registry modules, one for each combination of the providers of the roles that take one, the one with the providers, such as an app with a provider of a role that has a known bug and the modules that the tests of the role need, whose tests of the role must fail as failures expect, and whose other tests must pass; see runFailingApps.
MatrixMocks
A function among the files of a MatrixAppTest that sets up the mocks of the platform side of what the module of the tests runs in an app (see MatrixAppTest.mocks).
MatrixSelection
The apps of a matrix that a run of runMatrix checks: every app of the matrix, with each of its apps with every module and each of those for the other values of the mode options of its roles (modeAppsOf), unless told otherwise.
MatrixShard
One of the count shares of the apps of a run of the matrix that jobs of CI check side by side, the one numbered index, from 1: the apps at the positions whose remainder after division by count is index - 1, so each app is in exactly one share and the apps of a share keep their order.
MatrixStartProbe
A function among the files of a MatrixAppTest, in their directory integration_test/, that a check on a device runs once the first screen of the app settled (see MatrixAppTest.startProbe).
MatrixToolOptions
The options of a matrix tool that choose the apps of its run, which come before its directory: --combinations <pairwise|3-wise|all>, the apps with every module that a run checks or generates, a pairwise covering by default; --app <name>, only the app with every module of that name, with --every-module or --create, or the app that --add-app-tests adds tests to, which was generated as that app; and --shard <index>/<count>, a share of the apps that a run checks.
NamedEveryModuleApp
The app with every module whose case of the contract harness is name, such as every module (riverpod), as a job of CI takes its app of the plan (see matrixPlanOf).

Enums

EveryModuleCombinations
A covering of the apps with every module: the apps that have, between them, every tuple of strength providers of different roles that one of the apps has, of the roles that take one and have several providers among the apps, and every module that one of the apps has; all of the apps without a strength.
FlutterSystem
A system that the jobs of CI with Flutter run on, with the architecture of its runners.

Constants

appsPerShard → const int
The number of apps that a job of CI checks at most in a shard of a matrix: about 20 minutes, as generating, analyzing and testing an app with Flutter takes 45 to 50 seconds on Linux.
maxEveryCombination → const int
The number of the apps with every module, one for each combination of the providers of the roles that take one, and one more for each such combination and each combination of the other values of the mode options (modeAppsOf), that a plan with every combination checks all of, rather than a 3-wise covering of them.
startAnswersFile → const String
The path in an app of the answers of CI that the probes of its tests ask for (MatrixStartProbe.answer), one on each line in the order of the tests, which addAppTests writes next to startProbesFile: the script of CI that starts the app on a device runs only those of its answers.
startBuildArgumentsFile → const String
The path in an app of the arguments of the flutter tool that build the app with its values, one on each line, which addAppTests writes next to startProbesFile (see MatrixEnv.toolArguments): the script of CI that starts the app on a device builds it with them.
startProbesFile → const String
The path in an app of the list of the probes of its tests, which addAppTests writes for the tests that run them (MatrixAppTest.readsStartProbes): startProbes, the name of the tests of each probe and the probe, in the order of the tests.
startReleaseArgumentsFile → const String
The path in an app of the arguments of the flutter tool that a build of the app as a release takes too, one on each line, which addAppTests writes next to startProbesFile: the script of CI that starts the app on a device builds a release app with them, and a debug app without them, so that it names no flavor of the app.

Functions

addAppTests(TestedApp target, List<MatrixAppTest> tests, {String? envFile, List<String> buildArguments = const [], List<String> releaseBuildArguments = const [], FileSystem fileSystem = const LocalFileSystem()}) → List<String>
Copies the files of tests into the generated app of target, with the placeholders of the files filled, writes the files that the tests generate for the app next to them, and returns the paths of the files in the app; see MatrixAppTest.directory and MatrixAppTest.generatedFiles.
addAppTestsTo(TestedApp target, List<MatrixAppTest> tests, {MatrixEnv? env, List<String> releaseBuildArguments = const [], CiEnvValues? envValues, MatrixFlutter flutter = _flutter, FileSystem fileSystem = const LocalFileSystem()}) → Future<(int, String)>
Adds tests to the generated app of target, an app that smf create generated outside the matrix, such as one that CI starts on a device: copies their files with addAppTests, which fills only {{app_name}} in them, and adds their dev dependencies with flutter pub add through flutter, if they have any. It runs neither the analysis nor the tests, which the caller runs where it needs them.
appTestCommands(List<MatrixAppTest> tests, {List<String> toolArguments = const []}) → List<List<String>>
The flutter commands that run tests in an app once their files are in it: flutter pub add of their dev dependencies, if they have any; flutter analyze, since the tests follow the rules of the analysis of the app too; and flutter test, which runs every test of the app, with the toolArguments of the app, which give it its values (see MatrixEnv.toolArguments). flutter analyze takes no such argument.
appTestsDirectoryOf(String package) → Future<String>
The directory app_tests of the package package, next to its lib/ in the configuration of the packages of this program: the directory of the files of the MatrixAppTests that the package keeps, each in a directory of its own.
appTestsFor(MatrixApp app, List<MatrixAppTest> named, List<MatrixAppTest> all) → List<MatrixAppTest>
The tests of all to add to app, an app of the matrix that smf create generated outside it, such as one that CI starts on a device, for the tests named: those, and, if one of them runs the probes of the tests of the app (MatrixAppTest.readsStartProbes), such as the start check, each other test of all with a probe (MatrixAppTest.startProbe) that applies to app, in the order of all. So the start check goes through the roles of the app, whichever they are, without the caller naming the tests of the roles.
appTestsReport(List<MatrixAppTest> tests, {required List<SmfModule> modules, required Map<ModuleId, String> packages, required Future<List<MatrixApp>> apps(), MatrixEnv? env, FileSystem fileSystem = const LocalFileSystem()}) → Future<List<Map<String, Object>>>
What the checks of the repository need to know of each of tests, as JSON, which the matrix tools print with --app-tests --json (tools/app_tests_test.dart):
createEveryModuleApps(List<SmfModule> modules, {required String directory, required String name, List<String> options = const [], Map<String, String?> roleOptions = const {}, EveryModuleApps apps = const EveryModuleApps(), MatrixCommands commands = const MatrixCommands()}) → Future<int>
Generates in directory the apps with every module of modules with roleOptions, for CI to build them for Android and iOS and to start them on devices: each as the MatrixApp.packageName of name, such as start_app and start_app_riverpod, with the options of CI and then options, such as --org com.example. It analyzes and tests none of them, which runMatrix does.
everyModuleAppsOf(List<SmfModule> modules, {Map<String, String?> roleOptions = const {}, bool withoutExternalSteps = false}) → Future<({List<MatrixApp> apps, List<ContractResult> failed})>
The apps with every module of modules: those that the contract harness builds for each combination of the providers of the roles that take one, each with as many modules as one app can have (see ContractHarness.casesOfAll), with their MatrixApp.everyModuleWith. The apps and the options of their roles are those of matrixOf.
expectedFailureProblems(String report, List<MatrixExpectedFailure> expected, {String? directory}) → List<String>
The problems of a run of flutter test --reporter json in an app whose tests of expected must fail, and whose other tests must pass; report is what the run wrote. A problem shows a test file by its path in the app when the report has it in directory, the directory of the app.
flutterVersionsMatrix(List<FlutterVersion> versions) → Map<String, Object>
The matrix of the nightly workflow for versions: the FlutterVersion.toJson of each.
flutterVersionsOf(Map<FlutterSystem, Map<Version, FlutterRelease>> releases, {required VersionConstraint flutter, required VersionConstraint dart}) → List<FlutterVersion>
The versions of Flutter that the nightly run of CI checks the apps with, the oldest first: every stable release for Linux among releases (see stableReleasesOf) whose Flutter flutter allows and whose Dart dart allows, such as those of matrixSdkConstraints.
matrixCommand(List<String> arguments, String directory) → List<String>
The command that the matrix runs for flutter with arguments in directory: flutter with them, but for flutter analyze in a directory whose path has letters beyond ASCII, where flutter analyze of Flutter 3.44 and 3.47 fails on every system (https://github.com/flutter/flutter/pull/191377). There it is dart analyze --fatal-infos, which reports the same issues.
matrixOf(List<SmfModule> modules, {Map<String, String?> roleOptions = const {}, EveryModuleSelection everyModuleApps = EveryModuleCombinations.all}) → Future<({List<MatrixApp> apps, List<ContractResult> failed})>
The apps of the matrix of modules: every app that the contract harness builds for them, which covers every module and every provider with each subset of the roles it uses, and the apps with every module, one for each combination of the providers of roles that take one (see everyModuleAppsOf).
matrixPlanOf(List<SmfModule> modules, {bool everyCombination = false, Role<Object>? native, Map<String, String?> roleOptions = const {}}) → Future<({Map<String, Object> plan, List<String> problems})>
The plan of the jobs of CI that check the matrix of modules with roleOptions, which the matrix tools print with --plan, or its problems: the cases of the contract harness that failed, which leave the plan empty. The plan has, as JSON:
matrixSdkConstraints(List<SmfModule> modules) → Future<({VersionConstraint dart, VersionConstraint flutter})>
The versions of Flutter and of Dart that every app of the matrix of modules allows, as matrixOf finds its apps: the intersection of the SDK constraints of their pubspec.yamls, which the pipeline merges from the PubspecContribution.environments of the modules of each app, and which smf create checks the Flutter SDK against before it generates anything. A case of the contract harness with errors has no app in the matrix, and an app whose pubspec has no constraint allows every version.
modeAppsOf(List<SmfModule> modules, {Map<String, String?> roleOptions = const {}}) → Future<({List<MatrixApp> apps, List<ContractResult> failed})>
The apps with every module of modules for the other values of the mode options of their roles (see RoleOption.mode): each app of everyModuleAppsOf once more for every combination of the values of the mode options of its roles, but the combination of the first value of each, which is that app itself (see ContractHarness.casesOfModes).
printMatrixPlan(List<SmfModule> modules, List<String> options, {Role<Object>? native, void out(String line)?, void err(String line)?}) → Future<int>
Prints the plan of modules (see matrixPlanOf) as one line of JSON, for --plan of a matrix tool with options, which may be --every-combination, and returns the exit code: 0; 1, when the cases of the contract harness failed, with their problems instead; and 64, with the usage, for other options. out gets the plan, by default the standard output, and err the problems and the usage, by default the standard error.
runAppTests(GeneratedApp generated, MatrixApp app, List<MatrixAppTest> tests, {MatrixEnv? env, MatrixFlutter flutter = _flutter, FileSystem fileSystem = const LocalFileSystem()}) → Future<(int, String)>
Adds tests to generated, the app of the matrix app, with addAppTests, writes the values that they give the app into the file of env (see MatrixAppTest.envValues), and runs the appTestCommands with flutter until one fails, flutter test with the tool arguments of env for app; returns its exit code and the output up to it, which says which command failed, or 0 and the output of all. A problem of the files of the tests or of their values is a failure too, with the exit code 1, such as a value that the tests give without env.
runCiCommands(String directory, List<String> commands, {Map<String, String> environment = const {}, String? artifact}) → Future<(int, String)>
Runs commands one after another, each with bash -c, in a clone of a commit of the app in directory, as a job of CI runs them on a runner, and removes the clone. Returns 0 and the output, or the exit code of the first command that fails and the output up to it, whose last line names the command.
runCiOf(MatrixApp app, {required String directory, required MatrixCi ci, bool build = false, MatrixCommands commands = const MatrixCommands()}) → Future<int>
Runs the commands of the CI of app, which smf create generated in directory as that app of the matrix, such as an app with every module of the plan of a job of CI, in a clone of it (runCiCommands): those of each push, and with build then the build, with the values of MatrixCiBuild.variables, in a clone of its own, since a build starts from a repository without .env, which the checks may have written.
runFailingApps(List<MatrixFailingApp> apps, {required String directory, MatrixAppTests appTests = const MatrixAppTests([]), MatrixEnv? env, MatrixCommands commands = const MatrixCommands(), MatrixFlutter flutter = _flutter, FileSystem fileSystem = const LocalFileSystem()}) → Future<int>
Generates each of apps in directory, with the options of CI, adds to it the tests of appTests that apply to it, and runs them: its expected failures must fail as they expect, and its other tests must pass; see runFailingAppTests. Returns the exit code: 0 if the contract harness builds each app without errors (MatrixFailingApp.check), smf create generates it with every module and every step that the options of CI do not leave for later, the app with its tests has no issue, and its tests fail as expected; 1 otherwise.
runFailingAppTests(GeneratedApp generated, MatrixApp app, List<MatrixAppTest> tests, List<MatrixExpectedFailure> failures, {MatrixEnv? env, MatrixFlutter flutter = _flutter, FileSystem fileSystem = const LocalFileSystem()}) → Future<(List<String>, String)>
Adds tests to generated, the app of the matrix app, with their dev dependencies and the values that they give the app in the file of env, and runs them with flutter test --reporter json and the tool arguments of env through flutter once the app with the tests passes flutter analyze: the tests of failures must fail as they expect, and the other tests of the app must pass (see expectedFailureProblems).
runMatrix(List<SmfModule> modules, {required String directory, MatrixAppTests appTests = const MatrixAppTests([]), MatrixSelection selection = const MatrixSelection(), MatrixEnv? env, MatrixCi? ci, MatrixCommands commands = const MatrixCommands()}) → Future<int>
Generates every app of the matrixOf of modules in directory, with the options of CI, analyzes each with flutter analyze, and, in an app that some of the tests of appTests apply to, adds them and runs every test of the app with flutter test. Returns the exit code: 0 if every app was generated with every module and every step that the options of CI do not leave for later, has no issue and passes its tests, each of the tests applies to some app, they check the contract of their roles with every provider (see MatrixAppTests.roleProblems), and none of them reads the name or the options of an app (see MatrixAppTests.modeProblems); 1 otherwise.
stableReleasesOf(Object? json, FlutterSystem system) → Map<Version, FlutterRelease>
The stable releases of Flutter for system in json, its list of releases (FlutterSystem.releases), by version: those for the architecture of system whose version is plain, such as 3.44.2, and not v1.12.13+hotfix.9. The Dart of a release is the version that starts its dart_sdk_version, such as 3.10.0 of 3.10.0 (build 3.10.0-290.4.beta).
testRunSummary(String report, {String? directory}) → List<String>
What a run of flutter test --reporter json wrote in report, for a log: its lines that are no events, such as what flutter says before the tests, and a line for each test that it shows, with its result, its test file, by its path in the app when it is in directory, and its name, followed by the first lines of the message of its first failure if it did not pass; see expectedFailureProblems.

Typedefs

CiEnvValues = ({bool mask, String path})
A file of values of CI for the file of the values of an app, such as .env, which addAppTestsTo writes into the app: its path, and whether the values are masked as they are read (mask), as in a job of GitHub Actions.
MatrixAnalyze = Future<(int, String)> Function(String directory)
Analyzes the app in directory and returns the exit code and output of the analyzer.
MatrixCi = MatrixCiCommands? Function(MatrixApp app)
The commands of the CI of an app of a matrix, or null for an app without CI. The tool of a matrix gives it to its runs from the role of CI, which they know nothing of.
MatrixCiRunner = Future<(int, String)> Function(String directory, List<String> commands, {String? artifact, Map<String, String> environment})
Runs commands of CI in a clone of the app in a directory, as runCiCommands does; returns the exit code of the first command that fails and the output up to it, or 0 and the output of all.
MatrixCreate = Future<int> Function(List<String> arguments, void onCreated(GeneratedApp app))
Generates the app of smf create with arguments and gives it to onCreated; returns the exit code.
MatrixEnvKeyOwner = ({ModuleId? module, String? role})
Who gives an app a key of its values (MatrixEnv.keys): the module with the id module, or else the template of the role with the id role, since this library knows no role.
MatrixFlutter = Future<(int, String)> Function(List<String> arguments, String directory)
Runs flutter with arguments in directory and returns the exit code and the output.
MatrixTest = Future<(int, String)> Function(GeneratedApp generated, MatrixApp app, List<MatrixAppTest> tests, {MatrixEnv? env})
Adds tests to generated, the app of the matrix app, with the values that they give it in the file of env, and runs them with flutter test, with the tool arguments of env; returns the exit code of the first command that fails and the output up to it, or 0 and the output of all.
TestedApp = ({MatrixApp? app, GeneratedApp generated})
An app that tests go into: the app that smf create generated (generated), and the app of the matrix that it was generated as (app), null for an app that it generated outside the matrix.

Exceptions / Errors

FlutterVersionsException
A problem of the versions of Flutter that flutterVersionsOf finds.
MatrixAppTestException
A problem of the files of MatrixAppTests, which addAppTests finds before it copies them into an app.