build_native 0.0.11+1

build_native #

Pub License

Compile native extensions with package:build, using the system compilers.

Windows building is not supported YET.

About #

This is a 2-step build process:

  1. Build *.{c,cc,cpp} files to .o.
  2. Link files into a shared library.

Ultimately, to build everything and run a Dart script, you will just need to run pub run build_runner run <script name>.

The goal of this package is to use existing infrastructure to build native extensions.

As an added note, this has not been tested on Linux, but it is developed on Mac, and the two platforms compile extensions almost exactly the same way.

Usage #

build_native requires only very little configuration. However, this means that it can only serve one purpose: invoking the compiler on the user's system.

Command Line Usage #

package:build_native ships with a few commands that can make the native extension experience a bit easier to bear:

Verifying the Environment #

To ensure that the system has a compatible compiler available, and that the necessary executables are in the PATH to build extensions, run:

pub run build_native doctor

Generating Extension Boilerplate #

Creating native extensions for any language can tend to involve a lot of boilerplate.

To quickly scaffold a new native extension, run:

pub run build_native scaffold

Source Files #

Also, because of the nature of package:build, each input can only create one output. For *.c, *.cc, and *.cpp, files, the system compiler is invoked to create a .o file. To override this, set the CC environment variable when compiling C, or the CXX environment variable when compiling C++.

Files named *.macos.{c,cc,cpp} will only build on Mac. The same applies to linux and windows. This can be used to apply platform-specific settings in your build.

Master Build File #

To perform linking, include an lib<extension_name>.build_native.yaml file in the directory where the extension should be built.

It should contain a list of source files to link together.

Note that these should all be asset ID's.

The simplest example, libsample_extension.build_native.yaml:

sources:
  - example|src/sample_extension.cc

See example/ for more.

The most common of the supported options:

flags:
  - "-O2"
sources:
  - example|sample_extension.cc
  - example|sample_extension.macos.cc # Will only be included on MacOS; ignored elsewhere
include:
  - some_dir
  - some_other_package|lib/some_file.h # If passing an asset id, you must use a filename.
link:
  - curl
  - readline
  - some_other_package|lib/libsome_extension.build_native.yaml
define:
  foo: bar
  DEBUG:

Platform-Specific Options #

To specify options that should only apply to a given platform, add a <extension_name>.<platform_name>.macos.build_native.yaml file, for example, foo_bar.macos.build_native.yaml. This will be merged into the main options.

Platforms available: macos, windows, linux.

By providing this as the PLATFORM environment variable, you can override this.

Disallowing a Platform #

If you know your library will certainly never build on a given system, you can explicitly disallow it, instead of forcing users to first download dependencies before a build failure:

disallowed_platforms:
  - linux
  - macos
  - windows

Third-Party Dependencies #

Unless you are a maniac and actually intend to write by hand every line of C/C++ code by hand, you might eventually need to pull in some source code from the Internet, so that it can be built alongside your program. Specify the names of dependencies in the third_party section of your configuration:

third_party:
  git: git://some/repo/here
  link:
    - lib # Directories to link against; relative paths.
  include:
    - include  # Directories to include from; relative paths.
  sources: # Source files to compile
    - src/main.c
    - src/b/c/d.c

Specifying a Subdirectory #

You can optionally specify a subdirectory against which to search for external build systems, include directories, source files, and the like:

third_party:
    curl:
      path: src/some/dir/where/everything/really/is

External Build Systems #

package:build_native can automatically detect configuration and build external projects based on the following files:

  • CMakeLists.txt - if present, triggers a CMake build on the system.
  • Makefile - if present, triggers a GNU make build (nmake on Windows).
  • configure - if present, it is executed via sh, followed by a make build (nmake on Windows).
  • configure.ac or configure.in - if present, triggers autoreconf, configure, then make (nmake on Windws).

You can specify a target in your dependency, which will be passed to make or CMake.

Note: If your aim is cross-platform builds, I personally recommend using CMake. Opting for GNU Make can easily shut out Windows users, which many libraries might not want. (Think node-gyp, which has abysmal support for Windows.)

Explicitly Linking a File #

In some cases, especially when using an external build system, a dependency might emit a shared library; in such a case, the OS needs to know where to find it. package:build_native is not able to discern this automatically, so specify the names of any possible dynamic libraries the external build might create:

mysql_connector:
    git: ...
    libraries:
      - libmysqlcppconnector8.1.dylib
      - libmysqlcppconnector8.1.so
      - libmysqlcppconnector8.1.dll

Disabling Automatic Builds #

In the case that you don't want to automatically to auto-detect build configuration in an external project, make sure you pass a sources array to its configuration.

Note: You can pass ["none"], and no sources will be built, as well as disabling auto-build.

From the Web #

To require an archive from the Internet:

third_party:
  curl:
    # All of these formats are supported:
    url: https://curl.haxx.se/download/curl-7.60.0.zip
    url: https://curl.haxx.se/download/curl-7.60.0.tar
    url: https://curl.haxx.se/download/curl-7.60.0.tar.gz
    url: https://curl.haxx.se/download/curl-7.60.0.tar.bz2
    url: https://curl.haxx.se/download/curl-7.60.0.tar.xz
    url: https://curl.haxx.se/download/curl-7.60.0.tar.lz
    md5: "some-hash-here" # Recommended if distributing on Pub, for security reasons.
    sha256: SHA256 hashes are also supported

From Git #

To require from Git:

third_party:
  http_parser:
      git: https://github.com/nodejs/http-parser.git
      commit: "some-hash"
      branch: master
      tag: some tag
      path: foo/bar
      include:
        - include
      sources:
        - src/main.c
        - src/b/c/d.c

Always cloned with --depth 1.

Windows #

Save yourself a hassle by running the build within the Visual Studio Developer Command Prompt.

Regardless, executables like cl.exe and link.exe should be available. Otherwise:

To enable a 64-bit toolset: https://docs.microsoft.com/en-us/cpp/build/how-to-enable-a-64-bit-visual-cpp-toolset-on-the-command-line

vcvarsall might be contained in: C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build

Unix #

On Unix, if you some error like this:

fatal error: bits/c++config.h: No such file or directory

Then run:

sudo apt-get install -y gcc-multilib g++-multilib

0.0.11 #

  • Begin working on Windows support.
  • Fixes for supporting *.g.cpp, etc.

0.0.9+11 #

  • Cover a case where file modes in an archive are null.

0.0.9+10 #

  • Fix decoding of .lz/.xz.

0.0.9+9 #

  • Resolve third party link directories against the build directory.

0.0.9+8 #

  • Don't build shared libraries to /dev/stdout, but to a file instead.

0.0.9+7 #

  • Allow malformed UTF8 sequences.

0.0.9+6 #

  • Patch a bug in how 3rd-party dependencies with sources were compiled.

0.0.9+5 #

  • Support autoconf, and check for it in doctor.
  • Check for cmake.
  • When using libraries from a 3rd-party dependency, pass the library's directory as an -L flag.

0.0.9+4 #

  • Build external dependencies into a separate third_party_build dir.

0.0.9+3 #

  • 644 => 420 in decimal...

0.0.9+2 #

  • Only chmod files without mode 644.
  • Support SHA1 hashes.

0.0.9+1 #

  • Re-enable builder.

0.0.9 #

  • Use C++ 11 on Unix systems.
  • Support decompression of .xz and .lz via package:lzma.
  • Support SHA256 checksum verification.

0.0.8 #

  • Use otool and install_name_tool to ensure that output libraries on MacOS know where to find dependencies.

0.0.7+6 #

  • Fix a bug in which third-party includes were not processed.

0.0.7+5 #

  • Don't manually git checkout if no branch/tag/commit is specified.
  • Change third_party deps from package|x -> package.x; this seems to appease CMake.

0.0.7+4 #

  • Log every program execution in [CONFIG].

0.0.7+3 #

  • Fix a small bug in how platform-specific options were discovered.

0.0.7+2 #

  • thirdPartyBuilder and libraryBuilder should only access the master build file.

0.0.7+1 #

  • Errors in doctor should print in red!

0.0.7 #

  • Added command-line utilities, for an easier experience.

0.0.6 #

  • Allow third-party libraries with sources to build their own static libraries.
  • Allow linking against the outputs of other packages.
  • Allow including headers from other packages.
  • Allow projects to explicitly disallow platforms.

0.0.5 #

  • Update the README, etc. to reflect on the fact that we are no longer using CMake.
  • Added the thirdPartyBuilder, which enables users to pull in external sources.
  • Enabled includes and linking against third_party dependencies.

0.0.4 #

  • Return to using the user's system to build object files. Hooray for incremental builds!
  • Split out object file-building functionality into a much cleaner API.

0.0.3 #

  • Update SDK constraints, dependencies, etc., to ensure the package installs!
  • Finalize decision to build to cache.

0.0.2 #

  • Use scratch_space to deal with temp files.
  • Split into build_native and example.
  • Use CMake.

0.0.1 #

  • Build individual object files separately.
  • Use a config-based approach to link libraries.
  • Windows support is now broken, but will be added again soon.

Use this package as an executable

1. Install it

You can install the package from the command line:


$ pub global activate build_native

2. Use it

The package has the following executables:


$ build_native

Use this package as a library

1. Depend on it

Add this to your package's pubspec.yaml file:


dependencies:
  build_native: ^0.0.11+1

2. Install it

You can install packages from the command line:

with pub:


$ pub get

with Flutter:


$ flutter pub get

Alternatively, your editor might support pub get or flutter pub get. Check the docs for your editor to learn more.

3. Import it

Now in your Dart code, you can use:


import 'package:build_native/build_native.dart';
  
Popularity:
Describes how popular the package is relative to other packages. [more]
0
Health:
Code health derived from static analysis. [more]
86
Maintenance:
Reflects how tidy and up-to-date the package is. [more]
70
Overall:
Weighted score of the above. [more]
40
Learn more about scoring.

We analyzed this package on Jul 22, 2019, and provided a score, details, and suggestions below. Analysis was completed with status completed using:

  • Dart: 2.4.0
  • pana: 0.12.19

Platforms

Detected platforms: Flutter, other

Primary library: package:build_native/build_native.dart with components: io, build.

Health issues and suggestions

Document public APIs. (-1 points)

3 out of 3 API elements have no dartdoc comment.Providing good documentation for libraries, classes, functions, and other API elements improves code readability and helps developers find and use your API.

Fix lib/src/platform_type.dart. (-3.93 points)

Analysis of lib/src/platform_type.dart reported 8 hints, including:

line 26 col 7: DO use curly braces for all flow control structures.

line 28 col 7: DO use curly braces for all flow control structures.

line 30 col 7: DO use curly braces for all flow control structures.

line 32 col 7: DO use curly braces for all flow control structures.

line 34 col 7: DO use curly braces for all flow control structures.

Fix lib/src/models/config.g.dart. (-3.45 points)

Analysis of lib/src/models/config.g.dart reported 7 hints, including:

line 11 col 8: Don't type annotate initializing formals.

line 12 col 7: Don't type annotate initializing formals.

line 13 col 7: Don't type annotate initializing formals.

line 14 col 7: Don't type annotate initializing formals.

line 15 col 7: Don't type annotate initializing formals.

Fix lib/src/compiler/windows.dart. (-2.48 points)

Analysis of lib/src/compiler/windows.dart reported 5 hints:

line 15 col 3: The class 'Future' was not exported from 'dart:core' until version 2.1, but this code is required to be able to run on earlier versions.

line 15 col 10: The class 'Stream' was not exported from 'dart:core' until version 2.1, but this code is required to be able to run on earlier versions.

line 67 col 3: The class 'Future' was not exported from 'dart:core' until version 2.1, but this code is required to be able to run on earlier versions.

line 94 col 3: The class 'Future' was not exported from 'dart:core' until version 2.1, but this code is required to be able to run on earlier versions.

line 94 col 10: The class 'Stream' was not exported from 'dart:core' until version 2.1, but this code is required to be able to run on earlier versions.

Fix additional 4 files with analysis or formatting issues. (-3.49 points)

Additional issues in the following files:

  • lib/src/compiler/unix.dart (4 hints)
  • lib/src/common.dart (1 hint)
  • lib/src/third_party/url_dependency.dart (1 hint)
  • lib/src/third_party_builder.dart (1 hint)

Maintenance issues and suggestions

Support latest dependencies. (-10 points)

The version constraint in pubspec.yaml does not support the latest published versions for 1 dependency (build_config).

Maintain an example. (-10 points)

Create a short demo in the example/ directory to show how to use this package.

Common filename patterns include main.dart, example.dart, and build_native.dart. Packages with multiple examples should provide example/README.md.

For more information see the pub package layout conventions.

Package is pre-v0.1 release. (-10 points)

While nothing is inherently wrong with versions of 0.0.*, it might mean that the author is still experimenting with the general direction of the API.

Dependencies

Package Constraint Resolved Available
Direct dependencies
Dart SDK >=2.0.0-dev <3.0.0
angel_serialize ^2.0.0 2.2.3+2
archive ^2.0.0 2.0.10
args ^1.0.0 1.5.2
build ^1.0.0 1.1.5
build_config ^0.3.0 0.3.2 0.4.1
cli_util ^0.1.3 0.1.3+2
collection ^1.0.0 1.14.11
convert ^2.0.0 2.1.1
crypto ^2.0.0 2.0.6
front_end ^0.1.2 0.1.20
io ^0.3.0 0.3.3
lzma ^0.4.0 0.4.0+2
path ^1.0.0 1.6.2
quiver ^2.0.0 2.0.3
recase ^2.0.0 2.0.1
scratch_space ^0.0.3 0.0.3+2
source_span ^1.0.0 1.5.5
system_info ^0.1.0 0.1.1
yaml ^2.0.0 2.1.16
Transitive dependencies
analyzer 0.37.0
angel_model 1.0.3
async 2.3.0
charcode 1.1.2
csslib 0.16.1
file_utils 0.1.1
fixnum 0.10.9
glob 1.1.7
globbing 0.2.0 0.3.0
html 0.14.0+2
json_annotation 2.4.0
kernel 0.3.20
logging 0.11.3+2
matcher 0.12.5
meta 1.1.7
package_config 1.0.5
pedantic 1.8.0+1
pool 1.4.0
pub_semver 1.4.2
pubspec_parse 0.1.4
quiver_hashcode 2.0.0
stack_trace 1.9.3
string_scanner 1.0.4
term_glyph 1.1.0
typed_data 1.1.6
watcher 0.9.7+12
Dev dependencies
angel_serialize_generator ^2.0.0
build_runner ^1.0.0

Admin