Pub

A Simple Web Scraper for Dart & Flutter

A very basic web scraper implementation to scrap html elements from a web page.

Pull requests certainly welcome.

Getting Started

In your pubspec.yaml root add:

dependencies:
  web_scraper:

then,

import 'package:web_scraper/web_scraper.dart';

Note that as of version 0.0.6, the project supports not only Flutter projects, but also Dart projects.

Implementation

    final webScraper = WebScraper('https://webscraper.io');
    if (await webScraper.loadWebPage('/test-sites/e-commerce/allinone')) {
        List<Map<String, dynamic>> elements = webScraper.getElement('h3.title > a.caption', ['href']);
        print(elements);
    }

Checkout web_scraper_test.dart file to have closer look on all functionalities.

Methods

MethodDescriptionArgumentsReturn Type
loadWebPageLoads the webpage into response objectString routeFuture <bool>
getPageContentReturns webpage's html in string formatVoidString body
getElementReturns List of elements found at specified addressString address, List <String> attributesList <Map<String, dynamic>>
getAllScriptsReturns the list of all data enclosed in script tags of the documentVoidList <String>
getScriptVariablesReturns Map between given variable names and list of their occurence in the script tagsList <String> variableNamesMap <String, dynamic>

Contributing

  • Please branch from develop to implement bug fix/new feature.
    • Ensure that code is formatted according to base dart rules.
  • Open a PR with develop as the PR target with a clear description.

Libraries

web_scraper