extract_structured_data method

Future<Online> extract_structured_data(String key, { List<String>? types, })

Extracts structured data (JSON-LD, microdata, RDFa) from the page and stores it in the context.

key is the key under which the extracted data will be stored in the context. types (optional) is a list of schema.org types to filter the results.

Returns the Online instance for method chaining.

Example usage:

await online
  .extract_structured_data('allStructuredData')
  .extract_structured_data('productData', types: ['Product']);
var allStructuredData = ctx.get<List<Map<String, dynamic>>>('allStructuredData');
var productData = ctx.get<List<Map<String, dynamic>>>('productData');

Implementation

Future<Online> extract_structured_data(String key,
    {List<String>? types}) async {
  Show.action('extracting', 'structured data');
  var structuredData = await (await page).evaluate('''(types) => {
  const jsonLd = Array.from(document.querySelectorAll('script[type="application/ld+json"]'))
    .map(script => JSON.parse(script.textContent));
  if (types && types.length > 0) {
    return jsonLd.filter(data => types.includes(data['@type']));
  }
  return jsonLd;
}''', args: [types]);
  ctx.set(key, List<Map<String, dynamic>>.from(structuredData));
  return this;
}