build_modules 2.7.0

  • Readme
  • Changelog
  • Installing
  • 91

Module builders for modular compilation pipelines.
Build Status Issues related to build_modules Pub Package Version Latest Dartdocs Join the chat on Gitter

This package provides generic module builders which can be used to create custom compilation pipelines. It is used by build_web_compilers and build_vm_compilers package which provides standard builders for the web and vm platforms.

Usage #

There should be no need to import this package directly unless you are developing a custom compiler pipeline. See documentation in build_web_compilers and build_vm_compilers for more details on building your Dart code.

Module creation #

The overall process for emitting modules follows these steps:

  1. Emit a .module.library asset for every .dart library containing a description of the directives (imports, exports, and parts), a list of the dart: imports used, and a determination of whether the library is an "entrypoint". Entrypoint libraries are either those which are likely to be directly imported (they are under lib/ but not lib/src/) or which have a main. Only the libraries that can be imported (they aren't part files) will have an output. This step is mainly separated out for better invalidation behavior since the output will change less frequently than the code. The outputs from this step are non differentiated by platform.
  2. Emit package level .meta_module assets. This is an aggregation of the .module.library information with a first run at creating a module structure. The output depends on the algorithm, described below. The outputs from this step are specific to a platform and will indicate which libraries contain unsupported dart: imports for a platform. Conditional imports will also be resolved to a concrete dependency based on the platform. In this step the dependencies of modules references the imported library, which may not match the primary source of the module containing that library.
  3. Emit package level .meta_module.clean assets. This step performs an extra run of the strongly connected components algorithm so that any libraries which are in an import cycle are grouped into a single module. This is done as a separate step so that it can be sure it may read all the .meta_module results across packages in a dependency cycle so that it may resolve import cycles across packages. As modules are merged due to import cycles the new module becomes the union of their sources and dependencies. Dependencies are resolved to the "primary source" of the module containing the imported library.
  4. Emit .module assets which contain a filtered view of the package level meta-module specific to a single module. These are emitted for the "primary source" of each module, as well as each library which is an entrypoint.

Module algorithms #

Fine #

The fine-grained module algorithm is straightforward - any dart library which can be it's own module, is it's own module. Libraries are only grouped into a module when there is an import cycle between them.

The .meta_module step filters unsupported libraries and does no clustering. Import cycles are resolved by the .meta_module.clean step.

Coarse #

The coarse-grained module algorithm attempts to cluster libraries into larger modules without bringing in more code than may be imported by downstream code. It assumes that libraries under lib/src/ won't be imported directly outside of the package. Assuming that external packages only import the libraries outside of lib/src/ then no code outside of the transitive imports will be brought in due to module clustering.

The .meta_module step performs strongly connected components and libraries which are in an import cycle are grouped into modules since they can't be ordered otherwise. Within each top level directory under the package (lib/, test/, etc) the libraries are further bundled into modules where possible. Each "entrypoint" library is always kept in it's own modules (other than in the case of import cycles). Non entrypoint libraries are grouped where possible based on the entrypoints that transitively import them. Any non-entrypoint which is only transitively imported by a single entrypoint is merged into the module for that entrypoint. Any non-entrypoints which are imported by the same set of entrypoints are merged into their own module. The "primary source" for any module is always the lowest alpha-sorted source, regardless of whether it is an entrypoint. As libraries are merged the module becomes the union of their sources and dependencies.

2.7.0 #

  • Add support for an environment variable BUILD_DART2JS_VM_ARGS which can be used to supply Dart vm arguments for the dart2js processes.

2.6.3 #

  • Keep cached deserialized module instances in more cases. This may improve performance of incremental builds in watch mode.
  • Deprecated: The package specific unsupported module whitelist option provided by computeTransitiveDependencies. The only known uses are being removed.
  • Allow analyzer version 0.39.x.

2.6.2 #

Republish of 2.6.0 with the proper min sdk contraint.

2.6.1 #

Bug fix for issue #2464 #

Ignore the trackUnusedInputs option that was added in 2.6.0.

This option will be respected again in the next release which will have the proper minimum sdk constraint.

2.6.0 #

Add support for dependency pruning to the KernelBuilder. This should greatly improve the invalidation semantics for builds, meaning that less code will be recompiled for each edit you make.

This is not enabled by default but can be enabled by passing trackUnusedInputs: true to the KernelBuilder constructor.

2.5.0 #

  • Add an option to skip the unsupported module check for modules in specified packages.

2.4.3 #

  • Allow analyzer version 0.38.0.

2.4.2 #

  • Support the latest release of package:json_annotation.

2.4.1 #

  • Require non-empty output from kernel build steps.

2.4.0 #

  • Allow overriding the target name passed to the kernel worker.

2.3.1 #

  • Allow analyzer version 0.37.0.

2.3.0 #

  • Add a hasMain boolean to the ModuleLibrary class.
    • This is now used instead of isEntrypoint for determining whether or not to copy module files for application discovery.
  • Fix computeTransitiveDeps to do error checking for the root module as well as transitive modules.

2.2.0 #

  • Make the librariesPath in the KernelBuilder configurable.
  • Fixed bug where the configured dart SDK was ignored.

2.1.3 #

  • Skip compiling modules with kernel when the primary source isn't the primary input (only shows up in non-lazy builds - essentially just tests).

2.1.2 #

  • Output additional .module files for all entrypoints to ease discovery of modules for compilers.

2.1.1 #

  • Allow build_config 0.4.x.

2.1.0 #

  • Make using the incremental compiler in the KernelBuilder configurable.

2.0.0 #

Breaking Changes #

  • Remove the merge method from Module and replace with a static Module.merge. Module instances are now immutable.
  • Remove jsId, and jsSourceMapId from Module.
  • DartPlatform no longer has hard coded platforms, and its constructor is now public. Anybody is now free to create their own platform.
  • Removed the platform specific builder factories from the builders.dart file.
    • Packages that want to target compilation for a platform should create their own builder factories.
  • Removed completely the analyzer based builders - UnlinkedSummaryBuilder and LinkedSummaryBuilder.
    • All backends should now be using the KernelBuilder instead.
  • Removed the default module builders for each supported platform. These must now be created by the packages that want to add compilation targeting a specific platform.
    • This will help reduce asset graph bloat caused by platforms that you weren't actually targeting.

Improvements #

  • Update the kernel worker to pass input digests, along with --reuse-compiler-result and --use-incremental-compiler.
  • Increased the upper bound for package:analyzer to <0.37.0.

1.0.11 #

  • Allow build_config 0.4.x.

1.0.10 #

  • Fix a performance issue in the kernel_builder, especially for larger projects.

1.0.9 #

  • Allow configuring the platform SDK directory in the KernelBuilder builder.

1.0.8 #

  • Don't follow dart.library.isolate conditional imports for the DDC platform.

1.0.7+2 #

  • Update dart2js snapshot arguments for upcoming SDK.

1.0.7+1 #

  • Fix broken release by updating dart2js worker and restricting sdk version.

1.0.7 #

  • Explicitly skip dart-ext uris during module creation.
    • Filed Issue #2047 to track real support for native extensions.
  • Run workers with mode detachedWithStdio if no terminal is connected.

1.0.6 #

  • Improved the time tracking for kernel and analyzer actions by not reporting time spent waiting for a worker to be available.

1.0.5 #

  • Increased the upper bound for package:analyzer to <0.36.0.

1.0.4 #

  • Update to package:graphs version 0.2.0.
  • Use dartdevc --kernel instead of dartdevk.

1.0.3 #

  • Increased the upper bound for package:analyzer to <0.35.0.

1.0.2 #

  • Support the latest package:json_annotation.

1.0.1 #

  • Increased the upper bound for package:analyzer to '<0.34.0'.

1.0.0 #

Breaking Changes #

  • The Module constructor has an additional required parameter isSupported, which indicates if a module is supported on that module's platform.

0.4.0 #

Improvements #

  • Modules are now platform specific, and config specific imports using dart.library.* constants are supported.

Breaking Configuration Changes #

  • Module granularity now has to be configured per platform, so instead of configuring it using the build_modules|modules builder, you now need to configure the builder for each specific platform:
targets:
  $default:
    build_modules|dartdevc:
      options:
        strategy: fine

The supported platforms are currently dart2js, dartdevc, flutter, and vm.

Breaking API Changes #

  • Output extensions of builders have changed to include the platform being built for.
    • All the top level file extension getters are now methods that take a platform and return the extension for that platform.
  • Most builders are no longer applied by default, you must manually apply them using applies_builders in your builder.
  • Most builder constructors now require a platform argument.

0.3.2 #

  • Module strategies are now respected for all packages instead of just the root package.
  • Can now mix and match fine and coarse strategies at will, even within package cycles (although this may cause larger modules).
  • Removed analyzer dependency.

0.3.1+1 #

  • Support package:json_annotation v1.

0.3.1 #

  • Change the default module strategy for the root package to coarse.

0.3.0 #

Improvements #

  • Updated dart2js support so that it can do multiple builds concurrently and will restart workers periodically to mitigate the effects of dart-lang/sdk#33708.
  • Increased the upper bound for the sdk to <3.0.0.

Breaking Changes #

  • Removed the kernelSummaryExtension, and renamed the KernelSummaryBuilder to KernelBuilder. The new builder can be used to create summaries or full kernel files, and requires users to give it a custom sdk.
  • Changed metaModuleCleanBuilder to read .module.library files which are produced by the moduleLibrayBuilder. Clients using the automatically generated build script will get this automatically. Clients which have manually written build scripts will need to add it.

0.2.3 #

  • Update to the latest package:scratch_space and don't manually clear it out between builds. This provides significant speed improvements for large projects.

0.2.2+6 #

  • Support the latest package:build_config.

0.2.2+5 #

  • Updated the missing modules message to highlight that the error is likely due to a missing dependency.

0.2.2+4 #

  • Support package:analyzer 0.32.0.

0.2.2+3 #

  • Fix a race condition where we could fail to find the modules for some dependencies.

0.2.2+2 #

  • Fix an issue where modules were unnecessarily being built with DDC. #1375.

0.2.2+1 #

  • Fix an issue with new files causing subsequent build failures #1358.
  • Expose MetaModuleCleanBuilder and metaModuleCleanExtension publicly for usage in tests and manual build scripts.

0.2.2 #

  • Clean up .module and summary files from the output and server.

  • Add new ModuleBuilder strategies. By default the coarse strategy is used for all non-root packages and will create a minimum number of modules. This strategy can not be overridden. However, for the root package, the fine strategy will be used which creates a module for each strongly connected component. You can override this behavior by providing coarse to the strategy option.

    Example configuration:

    targets:
      $default:
        builders:
          build_modules|modules:
            options:
              strategy: coarse
    

0.2.1 #

  • Give a guaranteed reverse dependency order for Module.computeTransitiveDependencies

0.2.0+2 #

0.2.0+1 #

0.2.0 #

  • computeTransitiveDependencies now throws a MissingModulesException instead of logging a warning if it discovers a missing module.

0.1.0+2 #

  • Fix a bug with the dart2js workers where the worker could hang if you try to re-use it after calling terminateWorkers. This only really surfaces in test environments.

0.1.0+1 #

  • Fix a bug with imports to libraries that have names starting with dart.

0.1.0 #

  • Split from build_web_compilers.

Use this package as a library

1. Depend on it

Add this to your package's pubspec.yaml file:


dependencies:
  build_modules: ^2.7.0

2. Install it

You can install packages from the command line:

with pub:


$ pub get

with Flutter:


$ flutter pub get

Alternatively, your editor might support pub get or flutter pub get. Check the docs for your editor to learn more.

3. Import it

Now in your Dart code, you can use:


import 'package:build_modules/build_modules.dart';
  
Popularity:
Describes how popular the package is relative to other packages. [more]
94
Health:
Code health derived from static analysis. [more]
100
Maintenance:
Reflects how tidy and up-to-date the package is. [more]
70
Overall:
Weighted score of the above. [more]
91
Learn more about scoring.

We analyzed this package on Nov 21, 2019, and provided a score, details, and suggestions below. Analysis was completed with status completed using:

  • Dart: 2.6.1
  • pana: 0.12.21

Platforms

Detected platforms: Flutter, other

Primary library: package:build_modules/build_modules.dart with components: io, isolate, build.

Maintenance suggestions

The package description is too short. (-20 points)

Add more detail to the description field of pubspec.yaml. Use 60 to 180 characters to describe the package, what it does, and its target use case.

Maintain an example. (-10 points)

Create a short demo in the example/ directory to show how to use this package.

Common filename patterns include main.dart, example.dart, and build_modules.dart. Packages with multiple examples should provide example/README.md.

For more information see the pub package layout conventions.

Dependencies

Package Constraint Resolved Available
Direct dependencies
Dart SDK >=2.5.0 <3.0.0
analyzer >0.35.0 <0.40.0 0.39.1
async ^2.0.0 2.4.0
bazel_worker ^0.1.20 0.1.23
build >=1.2.0 <2.0.0 1.2.2
build_config >=0.3.0 <0.5.0 0.4.1+1
collection ^1.0.0 1.14.12
crypto ^2.0.0 2.1.3
glob ^1.0.0 1.2.0
graphs ^0.2.0 0.2.0
json_annotation >=1.2.0 <4.0.0 3.0.0
logging ^0.11.2 0.11.3+2
meta ^1.1.0 1.1.8
path ^1.4.2 1.6.4
pedantic ^1.0.0 1.8.0+1
scratch_space ^0.0.4 0.0.4+1
Transitive dependencies
_fe_analyzer_shared 1.0.0
args 1.5.2
charcode 1.1.2
checked_yaml 1.0.2
convert 2.1.1
csslib 0.16.1
fixnum 0.10.11
front_end 0.1.29
html 0.14.0+3
js 0.6.1+1
kernel 0.3.29
node_interop 1.0.3
node_io 1.0.1+2
package_config 1.1.0
pool 1.4.0
protobuf 1.0.1
pub_semver 1.4.2
pubspec_parse 0.1.5
source_span 1.5.5
stack_trace 1.9.3
string_scanner 1.0.5
term_glyph 1.1.0
typed_data 1.1.6
watcher 0.9.7+13
yaml 2.2.0
Dev dependencies
a
b
build_runner ^1.0.0
build_test ^0.10.9
build_vm_compilers ^1.0.0
test ^1.6.0