The coverage rate threshold on CI tools is a common approach to encourage developers to write tests. However, judging a pull request solely based on the coverage rate of the entire project is not always fair, particularly when it comes to significant refactor tasks that may naturally reduce the coverage rate.
This package adopts a different approach to analyze test coverage: it specifically focuses on checking the lines added in the pull request. The coverage rate is then calculated by dividing the number of uncovered new lines by the total number of new lines.
In addition, you have the option to set thresholds that will cause tests to fail on a CI tool. Furthermore, this package can also generate a report highlighting the uncovered code, making it easier to identify areas that require additional tests.
There are two ways to install the
pull_request_coverage package. Since it is a binary package, you can activate it from the command line by using the command
dart pub global activate pull_request_coverage. This will allow you to use it as a standalone program on your command-line interface (CLI). Alternatively, you can add it to the
dev_dependencies section of your project's
This package requires two pieces of information to analyze the test coverage:
lcov.infofile, which is generated by running the
flutter test --coveragecommand
- A diff between the current branch and the target branch, generated using the
Generating the lcov.info file
There is a known issue with the
flutter test --coverage command. It may not report untested files. There is a workaround for it, described in
Run the following command to generate the
flutter test --coverage
If you want to analyze a Dart project rather than Flutter, use the coverage package
To check the PR's code, pull_request_coverage needs a diff between its branch and the target one. The diff is read from the
You can pipe the
pull_request_coverage using bash's
| operator, like this:
git diff repository/main | flutter pub run pull_request_coverage
If you activate
dart pub global activate, you can invoke it directly:
git diff repository/main | pull_request_coverage
See Example tab to check an output example out
There are two ways to configure the execution of
pull_request_coverage: using CLI arguments or a YAML config file. Using a YAML file is recommended for large teams as the settings can be shared and used consistently across CI scripts. This eliminates the need for each developer to manually copy and paste the arguments to analyze coverage locally.
pull_request_coverage looks for the
./pull_request_coverage.yaml file to load its settings. You can override this by using the
Both methods support the same settings, and if a setting is provided through both CLI arguments and the YAML file, the value from the CLI arguments will take precedence.
git diff origin/main | flutter pub run pull_request_coverage --maximum-uncovered-lines 5 --ignore '/lib/di/**','**/gen.dart' --ignore-lines "^.*@override.*$"
YAML config file
Default values are indicated in parentheses.
- lcov-file (
coverage/lcov.info): Path to the lcov.info file generated by the
flutter test --coveragecommand.
- config-file (
pull_request_coverage.yaml): Path to the YAML settings file.
minimum-coverage : Specifies the minimum coverage rate required for the tests to pass.
maximum-uncovered-lines : Specifies the maximum number of uncovered lines allowed before the tests fail.
lines-and-rate): when both,
maximum-uncovered-lines, are specified, the
approval-requirementdetermines the conditions for passing the tests.
lines-and-rate: both conditions must be met to pass the tests.
lines-or-rate:only one of the conditions must be met to pass the tests.
ignore: list of files that should be ignored, using the widely-known Bash glob syntax. The cumulative count of ignored lines from these files will be shown in the report for statistical purposes only.
ignore-lines: A list of regular expressions used to filter lines in the source code. The cumulative count of ignored lines will be shown in the report for statistical purposes only.
true) : Ignores file paths that end with
add-to-known-generated-files: list of glob matchers to extend the list specified in
ignore, the lines matched by these patterns will be completely excluded from the report.
Check example out to see how those params can change the output
cli): Specifies the output format.
- cli: this output format is designed to be read on the command-line interface (CLI).
- markdown: this output is formatted using Markdown syntax, which is useful for displaying the output in a pull request comment, posted by a bot, for example.
diff): Specifies the Markdown output format (see example page).
- diff: Uses diff syntax to highlight the uncovered lines.
- dart: Uses Dart syntax to display the code with an appropriate color scheme, adding a comment at the end of uncovered lines.
true): Prints the file path of each fully covered file as a celebratory message =)
true): When set to
true, the source code of the uncovered lines will be printed in red font color, making it easier to identify the missing tests. If this parameter is set to
false, only the file path will be shown in the log.
true): By default,
pull_request_coverageutilizes a colorful font to highlight uncovered lines. You can disable this feature by setting this parameter to
false. Please note that this option is only available when the output mode is set to
true) : Enable or disabled emojis in the output .
2): Specifies the number of digits to display after the decimal point in the coverage rate.
fully-tested-message : Replace the final table report with a custom message when there are no untested lines
Under the hood
pull_request_coveragereads the diff from stdin. In certain cases, the stdin stream may never close, causing the analysis to become stuck. By default, if no data is received within one second,
pull_request_coverageassumes that it has reached the EOF.
none): internal log level. Possible values are
- 0 - Tests passed.
- 1 - Tests failed (only when thresholds are set).
- 255 - Execution has failed and tests were not executed.