Dart Test Runner #
The Dart Test Runner (DTR) is a command line Test Runner for Dart test files. DTR will automatically detect and run all the tests in your Dart project in the correct environment (VM or Browser).
Installation and usage #
DTR is available for download on the Pub Package Manager
To install DTR use this command:
pub global activate test_runner
To run DTR use this command from within the root of your Dart project:
pub global run test_runner
Alternatively you can add the pub cache
bin directory to your PATH:
~/.pub-cache/bin. Then you will be able to simply use (
-c gives a nice
For a list of options and to learn more use:
Here is an example of output from the Dart test runner:
bash> run_tests Checking Dart binaries... Dart binaries OK. Looking for Dart project in "./"... Found project "test-runner". Looking for test suites... Found 5 test suites (3 Standalone VM, 2 Dartium). Checking browser binaries... Browser binaries OK. Running all tests... Test suite passed: /vm_ok_test.dart Test suite passed: /subdir/vm_in_subdir_ok_test.dart Test suite failed: /vm_fail_test.dart Detailed results of test suite vm_fail_test.dart: ┌──────────────────────────────────────────────── │ FAIL: QuickSort │ Expected: equals [1, 2, 3, 4, 5] ordered │ Actual: [3, 5, 2, 4, 1] │ Which: was <3> instead of <1> at location  │ ... │ FAIL: Partition │ Expected: <1> │ Actual: <0> │ ... │ 0 PASSED, 2 FAILED, 0 ERRORS └──────────────────────────────────────────────── Test suite passed: /browser_ok_test.dart Test suite passed: /browser_ok_with_html_test.dart Summary: 1 TEST SUITE(S) FAILED. 4 TEST SUITE(S) PASSED.
TIP: use the
-c option to get a nice colored output
The exit code will be:
- If all tests passed:
- If at least one test has failed:
- Incorrect command line argument (e.g. missing
dart2js, incorrect project path...):
- If no test files were found in the project:
Test files detection and conventions #
Your tests have to follow certain conventions to be reliably detected by DTR. Please make sure that:
- Your tests files are suffixed with
- Each test file contains a
main()that runs all your unit tests.
Depending on the environment into which your test runs there are additional requirements listed below.
Each test file is considered a "test suite". If a test suites does not complete in 240 seconds it is aborted.
Standalone VM tests #
Standalone VM tests are tests that can be run from the command line using 'dart'. The executable of the test needs to return an exit code of 0 if all tests were successful and 1 or more if there was an error.
NOTE: Typically if you wrote your Standalone VM tests using the unittest package you should be all set.
Browser tests #
Browser tests are tests that need to be ran in a browser environment such as dartium.
Browser tests will be executed in a headless version of Chromium for Dart called Content Shell. Content Shell is not the same thing as Dartium. If you don't have Content Shell installed read the specific section about it.
If all tests have passed your test needs to print
PASS\n ("PASS" followed by a
You can provide an HTML file for a Browser Test. The HTML file needs to
have the same base name (for
You don't have to write an HTML file associated to your browser tests. The Dart test runner will automatically use a default HTML file and run your Browser tests in it if you didn't provide a custom one.
DTR will automatically detect if a test file needs to be ran inside a Browser if there is no associated HTML file.
NOTE: Typically if you wrote browser tests using the
unittest package you should be all
set as the Dart test runner will automatically and transparently set an
Configuration and will import
packages/unittest/test_controller.js into the HTML page.
Tools and environment #
DTR runs on Linux, Mac OS X and Windows. DTR also needs the following tools installed:
- Content Shell: A headless version of Dartium. Needed to run browser tests. Not the same thing as Dartium. Make sure you read the specific section about it
- Dart SDK: Especially the
pubcommand which will be used to run and serve tests and
dart2jswhich is used to detect browser tests.
Ideally make sure that these tools are available in your PATH as
content_shell (with a
.exe extension in Windows).
You can also specify the path to these tools executables with
Content Shell #
Content Shell is a stripped down version of Dartium/Chromium and it has the ability to run headlessly. This is not the same thing as Dartium and if you haven't already you likely need to install it.
For step-by=step instructions on how to install Content Shell on different operating systems read CONTENT_SHELL_INSTALL_INSTRUCTIONS.md.
Options and examples #
Generic usage of DTR:
run_tests [options] [<project-or-tests>...]
<project-or-tests> is the path to the project root/test folder or a list
of path to individual test files to run. If omitted all tests of the current
project will be discovered and ran.
--content-shell-bin: Path to the Content Shell executable. If omitted
"content_shell" from env is used.
--pub-bin: Path to the Pub executable. If omitted "pub" from env is used.
--dart-bin: Path to the Dart executable. If omitted "dart" from env is used.
--dart2js-bin: Path to the dart2js executable. If omitted "dart2js" from env
--skip-browser-tests: Skips all browser tests. Useful when browser binaries
like content_shell are not available.
--max-processes: Maximum number of processes that will run in parallel.
Defaults to "auto" which depends on the number of processors
available. Otherwise an integer is expected.
--disable-ansi: Disables the special ANSI character used in the console
output for things like dynamic line updating and color. This
is activated automatically on Windows.
--color: Prints the output in color in a shell.
--verbose: Prints all tests results instead of just the summary.
--help: Print usage information.
Runs all unit tests of the project in the current directory with a colored output:
Runs the specified two unit test files:
run_tests test/my_first_test.dart test/my_second_test.dart
Runs all unit tests of the Dart project located at
Runs all tests of the project located at
~/my_project/ and use
~/dartium/content_hell as the Dartium executable.
run_tests --content-shell-bin ~/dartium/content_shell ~/my_project/
Test using Docker #
The test environment can sometimes be tricky to setup (for instance Content Shell in Linux). As an alternative we are providing a Docker image that can be used to run your tests. There are also options to automatically run tests of Pub packages or GitHub repos.
For more details read docker/README.
BSD 3-Clause License. See LICENSE file.
This file contains highlights of what changes on each version of the Dart Test Runner.
Version 0.2.16 #
Retry to run
pub serve on another port if the port is already in use.
Version 0.2.15 #
- Now using
pub runwhen running VM tests.
- Added the
--dart-bincommand line option to allow setting the
- Making sure that
pub serveis killed at the end.
- Fixed bug where the stdError of Browser tests was not logged.
Version 0.2.14 #
Some bug fixes:
- Fixed bug where
--skip-browser-testswas having the opposite effect as intended.
- Fixed a display bug with
Version 0.2.13 #
--disable-ansi option which disables the use of special ANSI
characters like dynamic line updating and color.
Version 0.2.12 #
pub get has not been ran on the project previously and if not it
gets automatically ran.
Version 0.2.11 #
Minor command line output formatting modifications:
- Condensed the output of the test detection step to 1 line instead of 3.
- Now printing the warning "Dartium tests will be skipped!" in Orange instead of red when the --skip-browser-tests flag is used.
Version 0.2.10 #
--max-processesis now a new option: 'auto' which is now the default and will set the number of max processes depending on the number of processors on the machine.
Version 0.2.9 #
- Now limiting the number of tests and tests analysis processes that can run in parallel.
- Added option
--max-processesto allow setting the max number of processes running concurrently.
- Now showing the progress when detecting tests.
- Changed the option name to specify the path to the
Version 0.2.8 #
- Added windows support.
Version 0.2.7 #
- Small fix so that browser tests in sub-directories work.
Version 0.2.6 #
- Small fix to the browser test detection.
Version 0.2.5 #
- If a test suite does not complete in 240 seconds it is aborted.
Version 0.2.4 #
- If no browser tests are detected
content_shellwon't be needed.
- Added a
--skip-browser-teststhat will skip all browser tests and won't require
Version 0.2.3 #
- Tweaked command line output. Now show failed test suite details by default.
Version 0.2.2 #
- Exit code
3if no test files were found.
Version 0.2.1 #
- Fixes to have the test runner work with
pub global activateand
pub global run.
- Moved a lot of the code under
Version 0.2.0 #
- Now using Code Generation to make sure unittests are ran in a correct environment. Basically we set the unittest configuration.
Version 0.1.0 #
- Initial version that can run both Standalone VM and Web tests with basic output.
Use this package as an executable
1. Install it
You can install the package from the command line:
$ pub global activate test_runner
2. Use it
The package has the following executables:
$ run_tests $ test_runner
Use this package as a library
1. Depend on it
Add this to your package's pubspec.yaml file:
dependencies: test_runner: ^0.2.16+1
2. Install it
You can install packages from the command line:
$ pub get
Alternatively, your editor might support
Check the docs for your editor to learn more.
3. Import it
Now in your Dart code, you can use:
Describes how popular the package is relative to other packages. [more]
Code health derived from static analysis. [more]
Reflects how tidy and up-to-date the package is. [more]
Weighted score of the above. [more]
This package is not analyzed, because it is discontinued.
The package version is not analyzed, because it does not support Dart 2. Until this is resolved, the package will receive a health and maintenance score of 0.
Maintenance issues and suggestions
dartdoc successfully runs on your package's source files. (-10 points)
Dependencies were not resolved.
|Dart SDK||>=1.0.0 <2.0.0|