SiteReputationTracker class
Tracks the reputation and success rate of websites for scraping
Constructors
- SiteReputationTracker.new({int maxSites = 100})
- Creates a new SiteReputationTracker
Properties
- hashCode → int
-
The hash code for this object.
no setterinherited
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
Methods
-
getOptimalHeaders(
String url, Map< String, String> defaultHeaders) → Map<String, String> - Gets the optimal headers for the given URL based on past success
-
getOptimalRetries(
String url, int defaultRetries) → int - Gets the optimal number of retries for the given URL based on past performance
-
getOptimalTimeout(
String url, int defaultTimeout) → int - Gets the optimal timeout for the given URL based on past performance
-
getReputation(
String url) → SiteReputation? - Gets the reputation for the given URL
-
hasProblematicErrorPattern(
String errorMessage) → bool - Checks if the given error message contains any known problematic patterns
-
isProblematicSite(
String url) → bool - Checks if a site is problematic based on its reputation
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
recordFailure(
String url, String errorMessage) → void - Records a failed scrape for the given URL with the error message
-
recordSuccess(
String url) → void - Records a successful scrape for the given URL
-
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited