pull_request_coverage 0.0.1

A CLI tool to verify the test coverage of a pull request only, ignoring the rest of the project.

This is a CI tool that analyzes the coverage rate of a pull request, ignoring the lines that were not changed in the PR.

Motivation

The coverage rate threshold on CI tools is a common approach to encourage developers to write tests and keep improving the whole project's quality. Unfortunately, judging a pull request coverage by analyzing the coverage of the entire project is not always fair, especially on big refactor tasks, witch may naturally decrease the coverage rate.

This package tries a different approach to analyse the coverage rate. We only analyse lines that have been added on the pull request. The coverage rate will be calculated by dividing the amount of uncovered new lines by the amount of new lines.

You can set thresholds to make tests fail on a CI tool. This package can also print those lines that were not covered, making it easier to identify the missing tests.

Installing

Add this line to your package's pubspec.yaml under the dev_dependencies section:

dev_dependencies:
  pull_request_coverage:

You should specify the version to avoid breaking changes

Usage

This package uses two information to generate its report:

• A lcov.info file, generated by the flutter test --coverage command
• A diff between the current branch and the main one, generated by the git diff command

Generating the lcov.info file

There is a known issue with the flutter test --coverage command. It may not report untested files. There is a workaround for it, described in this issue

Run the following command to generate the coverage/lcov.info file:

flutter test --coverage

Running pull_request_coverage

To check the PR's code, this packages needs a diff between the pull request and the target branch of the repository. The package reads the diff from the STDIN input.

You can feed the STDIN using bash's | operator, like this:

git diff repository/main | flutter pub run pull_request_coverage

Output example:

lib/screen/fantasy_button.dart is fully covered 🎉 (+10)
lib/screen/fantasy_list.dart has uncovered lines 👀  (+133)
 58 : +      );
 59 : +    }
 60 : +
 ⬤  : +    return Scrollbar(
 ⬤  : +      child: AzListView(
 ⬤  : +        data: fantasyItems,
 64 : +        indexBarAlignment: Alignment.centerLeft,
 ⬤  : +        itemCount: fantasyItems.length,
 66 : +        indexBarHeight: double.infinity,
 67 : +        indexBarOptions: const IndexBarOptions(
 68 : +          hapticFeedback: true,
 72 : +            fontWeight: FontWeight.w500,
 73 : +          ),
 74 : +        ),
 ⬤  : +        itemBuilder: _buildTile,
 76 : +      ),
 77 :      );
 78 :    }
After ignoring excluded files, this pull request has:
    ✪ 256 new lines, 5 of them are NOT covered by tests. But....it's enough to pass the test 😉
    ✪ 98.046875% of coverage. This is above the limit of 95.0%

Lines set with a ⬤ symbol are the uncovered ones.

Example with thresholds set and not showing uncovered lines:

git diff repository/main | flutter pub run pull_request_coverage --minimum-coverage 95 --maximum-uncovered-lines 5 --hide-uncovered-lines

Exit code

Code	Description
0	All images are properly arranged
1	Inconsistencies were found
255	Execution has failed and tests were not executed

Options

Parameter	default	mandatory	description
lcov-file	coverage/lcov.info	no	lcov file path
exclude-suffix	.g.dart,.pb.dart,.pbenum.dart, .pbserver.dart,.pbjson.dart	no	Exclude all paths that start with those suffixes, separated by commas
exclude-prefix		no	Exclude all paths that start with those prefixes, separated by commas
minimum-coverage		no	Fail the test if the coverage rate doesn't reach this minimum value
maximum-uncovered-lines		no	Fail the test if the the total of uncovered lines is less than this value

Flags	description
hide-uncovered-lines	Do not print the source code that it is not covered by tests