puppeteer 1.9.0

Puppeteer in Dart #

A Dart library to automate the Chrome browser over the DevTools Protocol.

This is a port of the Puppeteer Node.JS library in the Dart language.

pub package Build Status Coverage Status

What can I do? #

Most things that you can do manually in the browser can be done using Puppeteer! Here are a few examples to get you started:

  • Generate screenshots and PDFs of pages.
  • Crawl a SPA (Single-Page Application) and generate pre-rendered content (i.e. "SSR" (Server-Side Rendering)).
  • Automate form submission, UI testing, keyboard input, etc.
  • Create an up-to-date, automated testing environment. Run your tests directly in the latest version of Chrome using the latest JavaScript and browser features.

Api #

Examples #

Launch Chrome #

Download the last revision of chrome and launch it.

import 'package:puppeteer/puppeteer.dart';

main() async {
  // Download the Chromium binaries, launch it and connect to the "DevTools"
  var browser = await puppeteer.launch();

  // Open a new tab
  var myPage = await browser.newPage();

  // Go to a page and wait to be fully loaded
  await myPage.goto('https://www.github.com', wait: Until.networkIdle);

  // Do something... See other examples
  await myPage.screenshot();
  await myPage.pdf();
  await myPage.evaluate('() => document.title');

  // Gracefully close the browser's process
  await browser.close();
}

Generate a PDF from a page #

import 'dart:io';
import 'package:puppeteer/puppeteer.dart';

main() async {
  // Start the browser and go to a web page
  var browser = await puppeteer.launch();
  var page = await browser.newPage();
  await page.goto('https://www.github.com', wait: Until.networkIdle);

  // Force the "screen" media or some CSS @media print can change the look
  await page.emulateMedia('screen');

  // Capture the PDF and save it to a file.
  await page.pdf(
      format: PaperFormat.a4,
      printBackground: true,
      pageRanges: '1',
      output: File('example/_github.pdf').openWrite());
  await browser.close();
}

Take a screenshot of a complete HTML page #

import 'dart:io';
import 'package:puppeteer/puppeteer.dart';

main() async {
  // Start the browser and go to a web page
  var browser = await puppeteer.launch();
  var page = await browser.newPage();

  // Setup the dimensions and user-agent of a particular phone
  await page.emulate(puppeteer.devices.pixel2XL);

  await page.goto('https://www.github.com', wait: Until.networkIdle);

  // Take a screenshot of the page
  var screenshot = await page.screenshot();

  // Save it to a file
  await File('example/_github.png').writeAsBytes(screenshot);

  await browser.close();
}

Take a screenshot of a specific node in the page #

import 'dart:io';
import 'package:puppeteer/puppeteer.dart';

main() async {
  // Start the browser and go to a web page
  var browser = await puppeteer.launch();
  var page = await browser.newPage();
  await page.goto('https://www.github.com', wait: Until.networkIdle);

  // Select an element on the page
  var form = await page.$('form[action="/join"]');

  // Take a screenshot of the element
  var screenshot = await form.screenshot();

  // Save it to a file
  await File('example/_github_form.png').writeAsBytes(screenshot);

  await browser.close();
}

Interact with the page and scrap content #

import 'package:puppeteer/puppeteer.dart';

main() async {
  var browser = await puppeteer.launch();
  var page = await browser.newPage();

  await page.goto('https://developers.google.com/web/');

  // Type into search box.
  await page.type('#searchbox input', 'Headless Chrome');

  // Wait for suggest overlay to appear and click "show all results".
  var allResultsSelector = '.devsite-suggest-all-results';
  await page.waitForSelector(allResultsSelector);
  await page.click(allResultsSelector);

  // Wait for the results page to load and display the results.
  const resultsSelector = '.gsc-results .gsc-thumbnail-inside a.gs-title';
  await page.waitForSelector(resultsSelector);

  // Extract the results from the page.
  var links = await page.evaluate(r'''resultsSelector => {
  const anchors = Array.from(document.querySelectorAll(resultsSelector));
  return anchors.map(anchor => {
    const title = anchor.textContent.split('|')[0].trim();
    return `${title} - ${anchor.href}`;
  });
}''', args: [resultsSelector]);
  print(links.join('\n'));

  await browser.close();
}

Create a static version of a Single Page Application #

import 'package:puppeteer/puppeteer.dart';

main() async {
  var browser = await puppeteer.launch();
  var page = await browser.newPage();
  await page.goto('https://www.w3.org');

  // Either use the helper to get the content
  var pageContent = await page.content;
  print(pageContent);

  // Or get the content directly by executing some Javascript
  var pageContent2 = await page.evaluate('document.documentElement.outerHTML');
  print(pageContent2);

  await browser.close();
}

Capture a screencast of the page #

The screencast feature is not part of the Puppeteer API. This example uses the low-level protocol API to send the commands to the browser.

import 'dart:convert';
import 'dart:io';
import 'package:image/image.dart' as image;
import 'package:puppeteer/puppeteer.dart';
import 'package:shelf/shelf_io.dart' as io;
import 'package:shelf_static/shelf_static.dart';

main() async {
  // Start a local web server and open the page
  var server =
      await io.serve(createStaticHandler('example/html'), 'localhost', 0);
  var browser = await puppeteer.launch();
  var page = await browser.newPage();
  await page.goto('http://localhost:${server.port}/rubiks_cube/index.html');

  // On each frame, decode the image and use the "image" package to create an
  // animated gif.
  var animation = image.Animation();
  page.devTools.page.onScreencastFrame.listen((event) {
    animation.addFrame(image.decodePng(base64.decode(event.data)));
  });

  // For this example, we change the CSS animations speed.
  await page.devTools.animation.setPlaybackRate(240);

  // Start the screencast
  await page.devTools.page.startScreencast(maxWidth: 150, maxHeight: 150);

  // Wait a few seconds and stop the screencast.
  await Future.delayed(Duration(seconds: 3));
  await page.devTools.page.stopScreencast();

  // Encode all the frames in an animated Gif file.
  File('example/_rubkis_cube.gif')
      .writeAsBytesSync(image.GifEncoder().encodeAnimation(animation));

  // Alternatively, we can save all the frames on disk and use ffmpeg to convert
  // it to a video file. (for example: ffmpeg -i frames/%3d.png -r 10 output.mp4)

  await browser.close();
  await server.close(force: true);
}

Low-level DevTools protocol #

This package contains a fully typed API of the Chrome DevTools protocol. The code is generated from the JSON Schema provided by Chrome.

With this API you have access to the entire capabilities of Chrome DevTools.

The code is in lib/protocol

import 'package:puppeteer/puppeteer.dart';

main() async {
  var browser = await puppeteer.launch();
  // Create a chrome's tab
  var page = await browser.newPage();

  // You can access the entire Chrome DevTools protocol.
  // This is useful to access information not exposed by the Puppeteer API
  // Be aware that this is a low-level, complex API.
  // Documentation of the protocol: https://chromedevtools.github.io/devtools-protocol/

  // Examples:

  // Start a screencast
  await page.devTools.page.startScreencast();

  // Change the animation speed for the document
  await page.devTools.animation.setPlaybackRate(10);

  // Access the memory information for the page
  await page.devTools.memory.getDOMCounters();

  // Go to https://chromedevtools.github.io/devtools-protocol/ to read more about
  // the protocol and use the code in `lib/protocol`.

  await browser.close();
}

Execute JavaScript code #

The Puppeteer API exposes several functions to run some Javascript code in the browser.

Like in this example from the official Node.JS library:

test(async () => {
  const result = await page.evaluate(x => {
    return Promise.resolve(8 * x);
  }, 7);
});

In the Dart port, you have to pass the JavaScript code as a string. The example above will be written as:

main() async {
  var result = await page.evaluate('''x => {
    return Promise.resolve(8 * x);
  }''', args: [7]);
}

The javascript code can be:

  • A function declaration (in the classical form with the function keyword or with the shorthand format (() =>))
  • An expression. In which case you cannot pass any arguments to the evaluate method.
import 'package:puppeteer/puppeteer.dart';

main() async {
  var browser = await puppeteer.launch();
  var page = await browser.newPage();

  // function declaration syntax
  await page.evaluate('function(x) { return x > 0; }', args: [7]);

  // shorthand syntax
  await page.evaluate('(x) => x > 0', args: [7]);

  // Multi line shorthand syntax
  await page.evaluate('''(x) => {  
    return x > 0;
  }''', args: [7]);

  // shorthand syntax with async
  await page.evaluate('''async (x) => {
    return await x;
  }''', args: [7]);

  // An expression.
  await page.evaluate('document.body');

  await browser.close();
}

If you are using IntellJ (or Webstorm), you can enable the syntax highlighter and the code-analyzer for the Javascript snippet with a comment like // language=js before the string.

main() {
  page.evaluate(
  //language=js
  '''function _(x) {
    return x > 0;
  }''', args: [7]);
}

Note: In a future version, we can image to compile the dart code to javascript on the fly before sending it to the browser (with ddc or dart2js).

Changelog #

1.9.0 (2019-06-30) #

  • Update Chromium version to 672088
  • Update dependencies
  • Fixes for Dart 2.4.0

1.8.0+1 (2019-06-22) #

  • Fix regression in page.tracing

1.8.0 (2019-06-22) #

  • Update Chromium version to 669486
  • Add an IOSink [output] parameter to [Page.pdf] as an alternative to returning the whole PDF bytes in memory.

1.7.3 (2019-06-06) #

  • Update Chromium version to 666595
  • Remove --disable-gpu flag passed to Chromium on Windows

1.7.2 (2019-06-05) #

  • Update Chromium version to 662092

1.7.1 (2019-05-27) #

  • Enable more Dart lints

1.7.0 (2019-05-27) #

  • Add page.ccessibility feature
  • Update default Chromium version to 662092

1.6.0 (2019-05-15) #

  • Add puppeteer.connect to connect to an existing Chromium instance
  • Add slowMo parameter for puppeteer.launch et puppeteer.connect to slow down communications with the browser.
  • Create Worker from service_worker and shared_worker.
  • Use a default viewport of 1280x1024 by default (allow to pass null to disable the default).
  • Add a small "plugin" system
  • Add the StealthPlugin to automatically applies various techniques to make detection of headless puppeteer harder.
  • Add Page.clickAndWaitForNavigation for convenience.
  • Add Page.coverage feature
  • Add Page.metrics feature
  • Add Page.tracing feature

1.5.0 (2019-05-07) #

  • Rename classes Frame, Request & Response to match the puppeteer API
  • Add Worker class
  • Add more test for the network API

1.4.0 (2019-05-04) #

  • Add more unit tests and more documentation
  • Fix bugs in request interception

1.3.0 (2019-05-02) #

  • Add more unit tests and more documentation
  • Update default chromium
  • Add example for screencast
  • Fix bugs

1.2.1 (2019-04-27) #

  • Add some documentation and examples in the source code (still work in progress: not all classes are documented yet).

1.2.0 (2019-04-24) #

  • Start a browser with puppeteer.launch instead of Browser.start to align with the puppeteer API
  • Add a list of all the devices from chrome: Accessible with: page.emulate(puppeteer.devices.iPhone6)
  • Add tests

1.1.0 (2019-04-22) #

  • Add some unit tests from the Puppeteer/Node.JS project and fix some bugs.

1.0.0 (2019-04-21) #

  • Initial port of Puppeteer in Dart

example/example.dart

import 'package:puppeteer/puppeteer.dart';

main() async {
  // Download the Chromium binaries, launch it and connect to the "DevTools"
  var browser = await puppeteer.launch();

  // Open a new tab
  var myPage = await browser.newPage();

  // Go to a page and wait to be fully loaded
  await myPage.goto('https://www.github.com', wait: Until.networkIdle);

  // Do something... See other examples
  await myPage.screenshot();
  await myPage.pdf();
  await myPage.evaluate('() => document.title');

  // Gracefully close the browser's process
  await browser.close();
}

Use this package as a library

1. Depend on it

Add this to your package's pubspec.yaml file:


dependencies:
  puppeteer: ^1.9.0

2. Install it

You can install packages from the command line:

with pub:


$ pub get

with Flutter:


$ flutter pub get

Alternatively, your editor might support pub get or flutter pub get. Check the docs for your editor to learn more.

3. Import it

Now in your Dart code, you can use:


import 'package:puppeteer/puppeteer.dart';
  
Popularity:
Describes how popular the package is relative to other packages. [more]
68
Health:
Code health derived from static analysis. [more]
100
Maintenance:
Reflects how tidy and up-to-date the package is. [more]
100
Overall:
Weighted score of the above. [more]
84
Learn more about scoring.

We analyzed this package on Jul 15, 2019, and provided a score, details, and suggestions below. Analysis was completed with status completed using:

  • Dart: 2.4.0
  • pana: 0.12.19

Platforms

Detected platforms: Flutter, other

Primary library: package:puppeteer/puppeteer.dart with components: io.

Dependencies

Package Constraint Resolved Available
Direct dependencies
Dart SDK >=2.3.0 <3.0.0
archive ^2.0.0 2.0.10
async ^2.2.0 2.2.0
collection ^1.14.0 1.14.11
http >=0.9.0 <0.13.0 0.12.0+2
logging ^0.11.3 0.11.3+2
meta ^1.1.1 1.1.7
path >=1.0.0 <2.0.0 1.6.2
petitparser >=2.2.0 <3.0.0 2.4.0
pool ^1.4.0 1.4.0
Transitive dependencies
args 1.5.2
charcode 1.1.2
convert 2.1.1
http_parser 3.1.3
source_span 1.5.5
stack_trace 1.9.3
string_scanner 1.0.4
term_glyph 1.1.0
typed_data 1.1.6
Dev dependencies
analyzer any
build_runner ^1.0.0
crypto any 2.0.6
dart_style ^1.2.7
image ^2.1.3
json_annotation ^2.0.0
json_serializable ^2.0.0
pedantic ^1.7.0 1.8.0+1
shelf ^0.7.2
shelf_static ^0.2.7
test any
yaml any

Admin