EnhancedRateLimiter class
An enhanced rate limiter for web scraping with advanced features
Constructors
-
EnhancedRateLimiter.new({int defaultDelayMs = 1000, Map<
String, int> ? domainDelays, int maxRetries = 3, int initialBackoffMs = 1000, int maxBackoffMs = 60000, double backoffMultiplier = 2.0, RobotsTxtHandler? robotsTxtHandler, ScrapingLogger? logger}) - Creates a new EnhancedRateLimiter with the given parameters
Properties
- hashCode → int
-
The hash code for this object.
no setterinherited
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
Methods
-
clearAllRateLimits(
) → void - Clears all rate limit statuses
-
clearRateLimit(
String domain) → void - Clears rate limit status for a domain
-
execute<
T> ({required String url, required Future< T> fn(), String? userAgent, int priority = 0}) → Future<T> - Executes a function with rate limiting
-
getDomainDelay(
String domain) → int - Gets the current delay for a domain
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
setDomainDelay(
String domain, int delayMs) → void - Sets a custom delay for a domain
-
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited