throwIfNotFound function
Throws a WebScraperError with statusCode 404 when a response indicates
the URL no longer exists. Other status codes are left to the caller.
Implementation
void throwIfNotFound(int statusCode, Uri url, bool debug) {
if (statusCode == 404) {
printLog("HTTP 404: $url is not available", debug, color: LogColor.red);
throw WebScraperError(
'URL not found (404): $url',
statusCode: 404,
);
}
}