canScrape method
Checks if the given URL can be scraped with the provided configuration.
This method determines if there's a matching ScraperConfig for the given URL by checking if the URL's path matches any of the path patterns defined in the scraper configurations.
Parameters:
url
: The URL to check for scraping capabilityscraperConfigMap
: Map of domain names to lists of scraper configurations
Returns:
true
if a matching scraper configuration is foundfalse
if no matching configuration exists
Implementation
bool canScrape({
required Uri url,
required ScraperConfigMap scraperConfigMap,
}) {
ScraperConfig? scraperConfig = findScraperConfig(
url: url,
scraperConfigMap: scraperConfigMap,
);
return scraperConfig != null;
}