test_runner 0.2.16+1

  • Readme
  • Changelog
  • Installing
  • 0

Dart Test Runner #

The Dart Test Runner (DTR) is a command line Test Runner for Dart test files. DTR will automatically detect and run all the tests in your Dart project in the correct environment (VM or Browser).

Installation and usage #

DTR is available for download on the Pub Package Manager

To install DTR use this command:

pub global activate test_runner

To run DTR use this command from within the root of your Dart project:

pub global run test_runner

Alternatively you can add the pub cache bin directory to your PATH: ~/.pub-cache/bin. Then you will be able to simply use (-c gives a nice colored output):

run_tests -c

For a list of options and to learn more use:

run_tests --help

Result #

Here is an example of output from the Dart test runner:

bash> run_tests

Checking Dart binaries...
Dart binaries OK.

Looking for Dart project in "./"...
Found project "test-runner".

Looking for test suites...
Found 5 test suites (3 Standalone VM, 2 Dartium).

Checking browser binaries...
Browser binaries OK.

Running all tests...
Test suite passed: /vm_ok_test.dart
Test suite passed: /subdir/vm_in_subdir_ok_test.dart
Test suite failed: /vm_fail_test.dart
Detailed results of test suite vm_fail_test.dart:
│ FAIL: QuickSort
│   Expected: equals [1, 2, 3, 4, 5] ordered
│     Actual: [3, 5, 2, 4, 1]
│      Which: was <3> instead of <1> at location [0]
│   ...
│ FAIL: Partition
│   Expected: <1>
│     Actual: <0>
│   ...
Test suite passed: /browser_ok_test.dart
Test suite passed: /browser_ok_with_html_test.dart


TIP: use the -c option to get a nice colored output

The exit code will be:

  • If all tests passed: 0
  • If at least one test has failed: 1
  • Incorrect command line argument (e.g. missing pub or dart2js, incorrect project path...): 2
  • If no test files were found in the project: 3

Test files detection and conventions #

Your tests have to follow certain conventions to be reliably detected by DTR. Please make sure that:

  • Your tests files are suffixed with _test.dart
  • Each test file contains a main() that runs all your unit tests.

Depending on the environment into which your test runs there are additional requirements listed below.

Each test file is considered a "test suite". If a test suites does not complete in 240 seconds it is aborted.

Standalone VM tests #

Standalone VM tests are tests that can be run from the command line using 'dart'. The executable of the test needs to return an exit code of 0 if all tests were successful and 1 or more if there was an error.

NOTE: Typically if you wrote your Standalone VM tests using the unittest package you should be all set.

Browser tests #

Browser tests are tests that need to be ran in a browser environment such as dartium.

Browser tests will be executed in a headless version of Chromium for Dart called Content Shell. Content Shell is not the same thing as Dartium. If you don't have Content Shell installed read the specific section about it.

If all tests have passed your test needs to print PASS\n ("PASS" followed by a line break).

You can provide an HTML file for a Browser Test. The HTML file needs to have the same base name (for my_test.dart use my_test.html).

You don't have to write an HTML file associated to your browser tests. The Dart test runner will automatically use a default HTML file and run your Browser tests in it if you didn't provide a custom one.

DTR will automatically detect if a test file needs to be ran inside a Browser if there is no associated HTML file.

NOTE: Typically if you wrote browser tests using the unittest package you should be all set as the Dart test runner will automatically and transparently set an appropriate test Configuration and will import packages/unittest/test_controller.js into the HTML page.

Tools and environment #

DTR runs on Linux, Mac OS X and Windows. DTR also needs the following tools installed:

  • Content Shell: A headless version of Dartium. Needed to run browser tests. Not the same thing as Dartium. Make sure you read the specific section about it
  • Dart SDK: Especially the pub command which will be used to run and serve tests and dart2js which is used to detect browser tests.

Ideally make sure that these tools are available in your PATH as pub, dart2js and content_shell (with a .bat or .exe extension in Windows). You can also specify the path to these tools executables with --content-shell-bin, --pub-bin and --dart2js-bin.

Content Shell #

Content Shell is a stripped down version of Dartium/Chromium and it has the ability to run headlessly. This is not the same thing as Dartium and if you haven't already you likely need to install it.

For step-by=step instructions on how to install Content Shell on different operating systems read CONTENT_SHELL_INSTALL_INSTRUCTIONS.md.

Options and examples #

Usage #

Generic usage of DTR:

run_tests [options] [<project-or-tests>...]

Where <project-or-tests> is the path to the project root/test folder or a list of path to individual test files to run. If omitted all tests of the current project will be discovered and ran.

Options #

--content-shell-bin: Path to the Content Shell executable. If omitted "content_shell" from env is used.

--pub-bin: Path to the Pub executable. If omitted "pub" from env is used.

--dart-bin: Path to the Dart executable. If omitted "dart" from env is used.

--dart2js-bin: Path to the dart2js executable. If omitted "dart2js" from env is used.

--skip-browser-tests: Skips all browser tests. Useful when browser binaries like content_shell are not available.

--max-processes: Maximum number of processes that will run in parallel. Defaults to "auto" which depends on the number of processors available. Otherwise an integer is expected.

--disable-ansi: Disables the special ANSI character used in the console output for things like dynamic line updating and color. This is activated automatically on Windows.

-c or --color: Prints the output in color in a shell.

-v or --verbose: Prints all tests results instead of just the summary.

-h or --help: Print usage information.

Examples #

Runs all unit tests of the project in the current directory with a colored output:

run_tests -c

Runs the specified two unit test files:

run_tests test/my_first_test.dart test/my_second_test.dart

Runs all unit tests of the Dart project located at ~/my_project/:

run_tests ~/my_project/

Runs all tests of the project located at ~/my_project/ and use ~/dartium/content_hell as the Dartium executable.

run_tests --content-shell-bin ~/dartium/content_shell ~/my_project/

Test using Docker #

The test environment can sometimes be tricky to setup (for instance Content Shell in Linux). As an alternative we are providing a Docker image that can be used to run your tests. There are also options to automatically run tests of Pub packages or GitHub repos.

For more details read docker/README.

License #

BSD 3-Clause License. See LICENSE file.

Changelog #

This file contains highlights of what changes on each version of the Dart Test Runner.

Version 0.2.16 #

Retry to run pub serve on another port if the port is already in use.

Version 0.2.15 #

  • Now using dart instead of pub run when running VM tests.
  • Added the --dart-bin command line option to allow setting the dart executable.
  • Making sure that pub serve is killed at the end.
  • Fixed bug where the stdError of Browser tests was not logged.

Version 0.2.14 #

Some bug fixes:

  • Fixed bug where --skip-browser-tests was having the opposite effect as intended.
  • Fixed a display bug with --skip-browser-tests.

Version 0.2.13 #

Added a --disable-ansi option which disables the use of special ANSI characters like dynamic line updating and color.

Version 0.2.12 #

Detect if pub get has not been ran on the project previously and if not it gets automatically ran.

Version 0.2.11 #

Minor command line output formatting modifications:

  • Condensed the output of the test detection step to 1 line instead of 3.
  • Now printing the warning "Dartium tests will be skipped!" in Orange instead of red when the --skip-browser-tests flag is used.

Version 0.2.10 #

  • Option --max-processes is now a new option: 'auto' which is now the default and will set the number of max processes depending on the number of processors on the machine.

Version 0.2.9 #

  • Now limiting the number of tests and tests analysis processes that can run in parallel.
  • Added option --max-processes to allow setting the max number of processes running concurrently.
  • Now showing the progress when detecting tests.
  • Changed the option name to specify the path to the dart2js executable from --dart2js to --dart2js-bin for consistency.

Version 0.2.8 #

  • Added windows support.

Version 0.2.7 #

  • Small fix so that browser tests in sub-directories work.

Version 0.2.6 #

  • Small fix to the browser test detection.

Version 0.2.5 #

  • If a test suite does not complete in 240 seconds it is aborted.

Version 0.2.4 #

  • If no browser tests are detected content_shell won't be needed.
  • Added a --skip-browser-tests that will skip all browser tests and won't require content_shell.

Version 0.2.3 #

  • Tweaked command line output. Now show failed test suite details by default.

Version 0.2.2 #

  • Exit code 3 if no test files were found.

Version 0.2.1 #

  • Fixes to have the test runner work with pub global activate and pub global run.
  • Moved a lot of the code under lib.

Version 0.2.0 #

  • Now using Code Generation to make sure unittests are ran in a correct environment. Basically we set the unittest configuration.

Version 0.1.0 #

  • Initial version that can run both Standalone VM and Web tests with basic output.

Use this package as an executable

1. Install it

You can install the package from the command line:

$ pub global activate test_runner

2. Use it

The package has the following executables:

$ run_tests
$ test_runner

Use this package as a library

1. Depend on it

Add this to your package's pubspec.yaml file:

  test_runner: ^0.2.16+1

2. Install it

You can install packages from the command line:

with pub:

$ pub get

Alternatively, your editor might support pub get. Check the docs for your editor to learn more.

3. Import it

Now in your Dart code, you can use:

import 'package:test_runner/test_runner.dart';
Describes how popular the package is relative to other packages. [more]
Code health derived from static analysis. [more]
Reflects how tidy and up-to-date the package is. [more]
Weighted score of the above. [more]
Learn more about scoring.

This package is not analyzed, because it is discontinued.

The package version is not analyzed, because it does not support Dart 2. Until this is resolved, the package will receive a health and maintenance score of 0.

Maintenance issues and suggestions

Make sure dartdoc successfully runs on your package's source files. (-10 points)

Dependencies were not resolved.


Package Constraint Resolved Available
Direct dependencies
Dart SDK >=1.0.0 <2.0.0