dghub_web_scrapper 1.0.0

Min Thant Htet.

DGHub Studio

Buy Me a Coffee

What is Web Scraping

• Content by Harkiran

Web scraping is an automatic method to obtain large amounts of data from websites. Most of this data is unstructured data in an HTML format which is then converted into structured data in a spreadsheet or a database so that it can be used in various applications. There are many different ways to perform web scraping to obtain data from websites. These include using online services, particular API’s or even creating your code for web scraping from scratch. Many large websites, like Google, Twitter, Facebook, StackOverflow, etc. have API’s that allow you to access their data in a structured format. This is the best option, but there are other sites that don’t allow users to access large amounts of data in a structured form or they are simply not that technologically advanced. In that situation, it’s best to use Web Scraping to scrape the website for data.

Web scraping requires two parts, namely the crawler and the scraper. The crawler is an artificial intelligence algorithm that browses the web to search for the particular data required by following the links across the internet. The scraper, on the other hand, is a specific tool created to extract data from the website. The design of the scraper can vary greatly according to the complexity and scope of the project so that it can quickly and accurately extract the data.

Installation

In the dependencies: section of your pubspec.yaml, add the following line:

dependencies:
  dghub_web_scrapper: <latest_version>

Import package

import 'package:dghub_web_scrapper/dghub_web_scrapper.dart';

Example - Normal

DGHubWebScrapper.get('https://cv.dghub.in/').then((html){

    }).onError((error, stackTrace) {

    });

Example - Support JavaScript

DGHubWebScrapper.getFullLoaded('https://cv.dghub.in/').then((html){

    }).onError((error, stackTrace) {

    });

Importants Methods and propriets

• Table by antonio-nicolau

Methods	Mean
html.title	Return the page title
html.getElementById	Return a single element searching for ID on the page
html.getElementsByClassName	Return a list of elements according class passed as parameter
html.getElementsByTagName	Return a list of elements according tag passed as parameter
html.querySelector	Return single element passing a list of selector
html.querySelectorAll	Return a list of elements passing a list of selector
text	Return text atribute from a tag returned
src	Return src atribute from a tag returned
href	Return href atribute from a tag returned

• Package created by Min Thant Htet