adaptive_golden_test

A Flutter package to generate adaptive golden files during widget tests.

This package is in beta. Use it with caution and file any potential issues here.

Example

Features

Use this package in your test to:

  • Generated golden files during test for different devices.
  • Load fonts.
  • Set window sizes and pixel density.
  • Await for images rendering.
  • Render Physical and system UI layers.
  • Render a keyboard during tests.
  • Set a preferred OS to run the tests.
  • Configure a difference tolerance threshold for files comparison.

Getting started

Add adaptive_golden_test to your dev dependencies

At the root of your test folder create a flutter_test_config.dart file with a testExecutable function.

Future<void> testExecutable(FutureOr<void> Function() testMain) async {
  await testMain();
}

See the official doc.

Usage

To render custom fonts:

  • Add your fonts to your app assets folders.
  • Add your fonts to your flutter assets.
flutter:
  fonts:
  - family: Roboto
    fonts:
      - asset: fonts/Roboto-Black.ttf
...
  • In your flutter_test_config, call loadFonts().
Future<void> testExecutable(FutureOr<void> Function() testMain) async {
  TestWidgetsFlutterBinding.ensureInitialized();
  await loadFonts();
  await testMain();
}

Alternatively you can load fonts from a separate package by specifying its name and path:

await loadFontsFromPackage(
  package: Package(
    name: 'my_theme_package',
    relativePath: '../theme',
  ),
);

Setup devices to run test on

Define a set of device variant corresponding to your definition of done.

final defaultDeviceConfigs = {
  Devices.ios.iPhoneSE,
  Devices.ios.iPhone12,
  Devices.ios.iPad,
  Devices.linux.laptop,
  Devices.android.pixel4,
  Devices.android.samsungGalaxyS20,
};

Use the AdaptiveTestConfiguration singleton to set variants.

Future<void> testExecutable(FutureOr<void> Function() testMain) async {
  TestWidgetsFlutterBinding.ensureInitialized();
  AdaptiveTestConfiguration.instance
    ..setDeviceVariants(defaultDeviceConfigs);
  await loadFonts();
  await testMain();
}

(Optional) Allow a differences threshold in golden files comparators

Source : The Rows blog Different processor architectures can lead to a small differences of pixel between a files generated on an ARM processor and an x86 one. Eg: a MacBook M1 and an intel one.

We can allow the tests to passe if the difference is small. To do this, add setupFileComparatorWithThreshold() to your flutter_test_config.

Future<void> testExecutable(FutureOr<void> Function() testMain) async {
  TestWidgetsFlutterBinding.ensureInitialized();
  AdaptiveTestConfiguration.instance
    ..setDeviceVariants(defaultDeviceConfigs);
  await loadFonts();
  setupFileComparatorWithThreshold();
  await testMain();
}

(Optional) Enforce a Platform for the test to run on

Different OS render golden files with small differences of pixel. See the flutter issue.

You can configure AdaptiveTestConfiguration singleton to make tests throw if they are run on an unintended platform.

Future<void> testExecutable(FutureOr<void> Function() testMain) async {
  TestWidgetsFlutterBinding.ensureInitialized();
  AdaptiveTestConfiguration.instance
    ..setEnforcedTestPlatform(TargetPlatform.macOS)
    ..setDeviceVariants(defaultDeviceConfigs);
  await loadFonts();
  setupFileComparatorWithThreshold();
  await testMain();
}

As an alternative you can use Alchemist.

Write a test

Use testAdaptiveWidgets function. It take a callback with two arguments, WidgetTester and WindowConfigData.

WindowConfigData is a data class that describes a devices. It's used as a test variant.

void main() {
  testAdaptiveWidgets(
    'Test description',
    (tester, variant) async {},
  );
}

Wrap the widget you want to test with AdaptiveWrapper.

await tester.pumpWidget(
  AdaptiveWrapper(
    device: variant,
    orientation: Orientation.portrait,
    isFrameVisible: true,
    showVirtualKeyboard: false,
    child: const App(),
  ),
);

Use the WidgetTester extension expectGolden to generate golden files.

await tester.expectGolden<App>(variant);

A basic test should looks like this:

void main() {
  testAdaptiveWidgets(
    '$App render without regressions',
    (tester, variant) async {
      await tester.pumpWidget(
        AdaptiveWrapper(
          device: variant,
          orientation: Orientation.portrait,
          isFrameVisible: true,
          showVirtualKeyboard: false,
          child: const App(),
        ),
      );

      await tester.expectGolden<App>(variant);
    },
  );
}

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Special thanks

I want to thank previous developers for their work:

  • BAM: 100 people company developing and designing multiplatform applications with React Native and Flutter using the Lean & Agile methodology.

Maintainer

👤 Martin Jablečník

Show your support

Give a ⭐️ if this project helped you!

Libraries

adaptive_golden_test