gherkin_integration_test 0.0.2+1

A gherkin integration test framework based on flutter's official integration_test package.

Gherkin Integration Test

This package is based on the Behaviour Driven Development (BDD) language called Gherkin. This language enables us as developers to design and execute tests in an intuitive and readable way. For people who have a little less experience with development, these tests are also easy to understand because the syntax is very similar to English.

Most Gherkin tests look something like this:

Feature: This feature shows an example

    Scenario: It shows a good example
      Given we start without an example
      When an example gets created
      Then the example should explode

In this same manner we have built our framework, we have the following classes at our disposal:

• IntegrationTest
• IntegrationFeature
• IntegrationScenario
• IntegrationExample
• IntegrationStep (abstract)
  • Given
  • When
  • Then
  • And
  • But

From top to bottom, each class can contain a number of the class below it (one to many). A test may contain multiple features which in turn may contain multiple scenarios. Scenarios can then (optionally) run different examples in which they perform a series of steps.

🥼 Basic testing knowledge

---

• Before continuing this guide make sure you have basic testing knowledge regarding writing unit tests in Flutter. The following resource is a great place to start:

  An introduction to integration testing

• Be sure to have a look at the expect library of the flutter_test package. You don’t have to know every method that’s available but it’s good to have seen most methods at least once so you know what’s possible. This library should be used to assert outcomes of your tests.

  expect library - Dart API

• Then specifically for integration tests continue to explore the methods that are available to you through the WidgetTester. Most useful ones (tapping, entering text, waiting for animations to finish) you will learn by practice but it’s good to have seen this reference at least once before writing your first test to get a feel for what’s possible.

  WidgetTester class - flutter_test library - Dart API

• Last but not least, we use the mockito package to mock services when needed, check out their page and specifically how to stub and how the @GenerateMocks([]) annotation works.

  mockito | Dart Package

🛠 Implementation

---

Start by creating a test class that inherits the IntegrationTest class. Then create a constructor that takes no arguments but does call the superclass with a description and (for now) an empty list of features.

@GenerateMocks([])
class DummyIntegrationTest extends IntegrationTest {
  DummyIntegrationTest()
      : super(
          description: 'All integration tests regarding dummies',
          features: [],
        );
}

📲 Features

---

In the features list we can now define our first IntegrationFeature. We give it a name and (for now) an empty list of scenarios.

@GenerateMocks([])
class DummyIntegrationTest extends IntegrationTest {
  DummyIntegrationTest()
      : super(
          description: 'All integration tests regarding dummies',
          features: [
            IntegrationFeature(
              description: 'Saving of dummies',
              scenarios: [],
            ),
          ],
        );
}

🤝 Scenarios

---

Now it's time to think about what kind of scenarios might occur in your test. Often this is already well thought out in advance when preparing a ticket. For this example we will use ’a successful save’ and ‘an unsuccessful save’ as possible scenarios. We use the IntegrationScenario class to create both scenarios and place them in the empty list. We again pass in a description and this time an empty list of steps.

@GenerateMocks([])
class DummyIntegrationTest extends IntegrationTest {
  DummyIntegrationTests()
      : super(
          description: 'All integration tests regarding dummies',
          features: [
            IntegrationFeature(
              description: 'Saving of dummies',
              scenarios: [
                IntegrationScenario(
                  description: 'Saving a good dummy should succeed',
                  steps: [],
                ),
                IntegrationScenario(
                  description: 'Saving a bad dummy should fail',
                  steps: [],
                ),
              ],
            ),
          ],
        );
}

🐾 Steps

---

Now comes the good part. For each scenario, we may define steps. We have access to Given, When, Then, And and But. All of these steps do basically the same thing in the background, but by using them correctly, you learn to plan, work out and execute your tests in an intuitive and proper BDD way.

Each step requires a description and a callback. The callback for the IntegrationTests looks as follows and grants access to the following parameters:

/// Callback used to provide the necessary tools to execute an [IntegrationStep].
typedef IntegrationStepCallback<T extends IntegrationExample?> = FutureOr<dynamic> Function(
  WidgetTester tester,
  Log log, [
  T? example,
  IntegrationTestWidgetsFlutterBinding? binding,
  Object? result,
]);

• WidgetTester tester
  • Class that programmatically interacts with widgets and the test environment (directly from Flutter’s integration_test package).
• Log log
  • Class that allows for subtle logging of steps information in your tests.
• IntegrationExample? example
  • Optional ‘Scenario Outline’ examples that we’ll get to later, in short these are different inputs for the same scenario so you can run / cover different variations of one scenario.
• IntegrationTestWidgetsFlutterBinding? binding
  • Optional binding element that’s retrieved after starting an integration test and that you may pass through at different levels (most commonly when initialising the IntegrationTest and passing it as an argument). This may be used to take screenshots for example.
• Object? result
  • Each step is able to optionally return a value, may this be the case then this value is available to you in the next step as a result.

Setting up the success scenario may look like this:

@GenerateMocks([])
class DummyIntegrationTest extends IntegrationTest {
  DummyIntegrationTest()
      : super(
          description: 'All integration tests regarding dummies',
          features: [
            IntegrationFeature(
              description: 'Saving of dummies',
              scenarios: [
                IntegrationScenario(
                  description: 'Saving a good dummy should succeed',
                  steps: [
                    Given(
                      'We are at the dummy screen',
                      (tester, log, [example, binding, result]) {
                        // TODO(you): Go to dummy screen
                      },
                    ),
                    When(
                      'We save a dummy',
                      (tester, log, [example, binding, result]) {
                        // TODO(you): Save dummy
                      },
                    ),
                    Then(
                      'It should succeed',
                      (tester, log, [example, binding, result]) {
                        // TODO(you): Verify success
                      },
                    ),
                  ],
                ),
                IntegrationScenario(
                  description: 'Saving a bad dummy should fail',
                  steps: [
                    // TODO(you): Implement fail steps
                  ],
                ),
              ],
            ),
          ],
        );
}

While this may perfectly fit our testing needs there are a couple functionalities at our disposal that give our tests extra power:

• IntegrationExample
• setUp and tearDown methods

🏆 Bonus GherkinSteps

---

• GivenWhenThen
  • For when you can’t be bothered to create and use the separate step functionality regarding the ‘Given’, ‘When’ and ‘Then’ steps. This allows you to write the entire test in one step.
• WhenThen
  • For when you can’t be bothered to create and use the separate step functionality regarding the ‘When’ and ‘Then’ steps. This allows you to combine both steps into one.
• Should
  • For when you feel like using steps is not your style. This step defines the entire test in one ‘Should’ sentence.

🧪 Examples

---

Let’s continue with our test demonstrated above. For the succeeding scenario of ‘saving a good dummy should succeed’ we’re going to add some examples. Each example will get run through the specified steps in your scenario and each example will be accessible through the example parameter. Let’s start with adding an example where we specify the platform and the current connection status.

@GenerateMocks([])
class DummyIntegrationTest extends IntegrationTest {
  DummyIntegrationTest()
      : super(
          description: 'All integration tests regarding dummies',
          features: [
            IntegrationFeature(
              description: 'Saving of dummies',
              scenarios: [
                IntegrationScenario(
                  description: 'Saving a good dummy should succeed',
                  examples: [
                    IntegrationExample(
                      description: 'Platform is iOS, Connection status is online',
                      values: [Platform.iOS, Connection.online],
                    ),
                    IntegrationExample(
                      description: 'Platform is Android, Connection status is online',
                      values: [Platform.android, Connection.online],
                    ),
                    IntegrationExample(
                      description: 'Platform is iOS, Connection status is offline',
                      values: [Platform.iOS, Connection.offline],
                    ),
                    IntegrationExample(
                      description: 'Platform is Android, Connection status is offline',
                      values: [Platform.android, Connection.offline],
                    ),
                  ],
                  steps: [
                    Given(
                      'We are at the dummy screen',
                      (tester, log, [example, binding, result]) {
                        // TODO(you): Go to dummy screen
                      },
                    ),
                    When(
                      'We save a dummy',
                      (tester, log, [example, binding, result]) {
                        // TODO(you): Save dummy
                      },
                    ),
                    Then(
                      'It should succeed',
                      (tester, log, [example, binding, result]) {
                        // TODO(you): Verify success
                      },
                    ),
                  ],
                ),
                IntegrationScenario(
                  description: 'Saving a bad dummy should fail',
                  steps: [
                    // TODO(you): Implement fail steps
                  ],
                ),
              ],
            ),
          ],
        );
}

So for each example:

• 'Platform is iOS, Connection status is online'
• 'Platform is Android, Connection status is online'
• 'Platform is iOS, Connection status is offline'
• 'Platform is Android, Connection status is offline'

It will now run each step (Given, When, Then) inside the 'Saving a good dummy should succeed' scenario. You may now access the example values in the following way:

Given(
  'We are at the dummy screen',
  (tester, log, [example, binding, result]) {
    final Platform platform = example.firstValue();
    final Connection connection = example.secondValue();
  },
),

🧸 Custom Examples

---

It’s also possible to create your own IntegrationExample like this:

class CustomExample extends IntegrationExample {
  CustomExample({
    required this.platform,
    required this.connection,
  });

  final Platform platform;
  final Connection connection;
}

Your IntegrationStep will automatically recognise the type of your example if you specify it as a generic argument for your IntegrationScenario like this:

class ExampleScenario extends IntegrationScenario<CustomExample> {}

🏗 setUpOnce, setUpEach, tearDownOnce, tearDownEach

---

Another handy way to empower your tests is by using one of several setUp and tearDown methods. Each class has access to these methods and will run them in sort of the same way:

• setUpEach - will run at the START of EACH IntegrationScenario under the chosen class (may be specified in IntegrationTest, IntegrationFeature or IntegrationScenario itself).
• tearDownEach - will run at the END of EACH IntegrationScenario under the chosen class (may be specified in IntegrationTest, IntegrationFeature or IntegrationScenario itself).
• setUpOnce - will run ONCE at the START of chosen class (may be specified in IntegrationTest, IntegrationFeature or IntegrationScenario itself).
• tearDownOnce - will run ONCE at the END of chosen class (may be specified in IntegrationTest, IntegrationFeature or IntegrationScenario itself).

Using the methods may look a bit like this:

@GenerateMocks([])
class DummyIntegrationTest extends IntegrationTest {
  DummyIntegrationTest()
      : super(
          description: 'All integration tests regarding dummies',
          setUpOnce: () async {
            await AppSetup.initialise(); // Runs once at the start of this test.
          },
          tearDownOnce: () async {
            await AppSetup.dispose(); // Runs once at the end of this test.
          },
          features: [
            IntegrationFeature(
              description: 'Saving of dummies',
              tearDownEach: () {
                // TODO(you): Reset to dummy screen at the end of each scenario or example in this feature.
              },
              scenarios: [
                IntegrationScenario(
                  description: 'Saving a good dummy should succeed',
                  setUpEach: () {
                    // TODO(you): Reset dummy status at the start of this scenario or each example in this scenario.
                  },
                  examples: // etc, rest of code

Now to run these tests all you have to do is add the DummyIntegrationTests to your main test function and hit run. In this example we would like to use the IntegrationTestWidgetsFlutterBinding in our tests so let’s add that to the constructor as well.