features/web_scraping/robots_txt_handler library
Classes
- RobotsTxtHandler
- A class for handling robots.txt files
- RobotsTxtRules
- A class for representing robots.txt rules
- pivox package
- documentation
- features/web_scraping/robots_txt_handler.dart
pivox package
- Libraries
- core/builders/pivox_builder
- core/config/proxy_source_config
- core/constants/app_constants
- core/errors/exceptions
- core/factories/pivox_factory
- core/models/proxy_model
- core/utils/isolate_pool
- core/utils/logger
- core/utils/parallel_processor
- features/http_integration/dio/dio_proxy_interceptor
- features/http_integration/http/http_proxy_client
- features/proxy_management/data/cache/proxy_cache_manager
- features/proxy_management/data/datasources/proxy_local_datasource
- features/proxy_management/data/datasources/proxy_remote_datasource
- features/proxy_management/data/models/proxy_model
- features/proxy_management/data/repositories/proxy_repository_impl
- features/proxy_management/data/repositories/socks_proxy_validator
- features/proxy_management/debug/proxy_debugger
- features/proxy_management/domain/entities/proxy
- features/proxy_management/domain/entities/proxy_analytics
- features/proxy_management/domain/entities/proxy_auth
- features/proxy_management/domain/entities/proxy_filter_options
- features/proxy_management/domain/entities/proxy_protocol
- features/proxy_management/domain/entities/proxy_score
- features/proxy_management/domain/entities/proxy_validation_options
- features/proxy_management/domain/error/proxy_error
- features/proxy_management/domain/error/proxy_error_detector
- features/proxy_management/domain/error/proxy_error_type
- features/proxy_management/domain/error/proxy_retry_policy
- features/proxy_management/domain/monitoring/proxy_pool_health_monitor
- features/proxy_management/domain/repositories/proxy_repository
- features/proxy_management/domain/services/proxy_analytics_service
- features/proxy_management/domain/services/proxy_preloader_service
- features/proxy_management/domain/session/proxy_session
- features/proxy_management/domain/strategies/adaptive_rotation_strategy
- features/proxy_management/domain/strategies/advanced_rotation_strategy
- features/proxy_management/domain/strategies/geo_rotation_strategy
- features/proxy_management/domain/strategies/geographic_rotation_strategy
- features/proxy_management/domain/strategies/proxy_rotation_strategy
- features/proxy_management/domain/strategies/random_rotation_strategy
- features/proxy_management/domain/strategies/rotation_strategy
- features/proxy_management/domain/strategies/rotation_strategy_factory
- features/proxy_management/domain/strategies/round_robin_rotation_strategy
- features/proxy_management/domain/strategies/website_specific_strategy
- features/proxy_management/domain/strategies/weighted_rotation_strategy
- features/proxy_management/domain/usecases/get_proxies
- features/proxy_management/domain/usecases/get_validated_proxies
- features/proxy_management/domain/usecases/validate_proxy
- features/proxy_management/platform/android_proxy_setter
- features/proxy_management/presentation/managers/advanced_proxy_manager
- features/proxy_management/presentation/managers/advanced_proxy_manager_adapter
- features/proxy_management/presentation/managers/proxy_manager
- features/web_scraping/adaptive_scraping_strategy
- features/web_scraping/advanced_web_scraper
- features/web_scraping/caching/cache_manager
- features/web_scraping/caching/data_cache_manager
- features/web_scraping/concurrent_web_scraper
- features/web_scraping/content/content_detector
- features/web_scraping/content/content_validator
- features/web_scraping/content/structured_data_extractor
- features/web_scraping/content/structured_data_validator
- features/web_scraping/content/text_extractor
- features/web_scraping/cookie_manager
- features/web_scraping/data_cache_manager
- features/web_scraping/dynamic_user_agent_manager
- features/web_scraping/enhanced_rate_limiter
- features/web_scraping/headless_browser/headless_browser
- features/web_scraping/headless_browser/headless_browser_config
- features/web_scraping/headless_browser/headless_browser_factory
- features/web_scraping/headless_browser/headless_browser_result
- features/web_scraping/headless_browser/headless_browser_service
- features/web_scraping/headless_browser/specialized_headless_handlers
- features/web_scraping/lazy_loading/lazy_load_detector
- features/web_scraping/lazy_loading/lazy_load_handler
- features/web_scraping/memory/data_chunker
- features/web_scraping/memory_efficient_parser
- features/web_scraping/pagination/pagination_detector
- features/web_scraping/pagination/pagination_handler
- features/web_scraping/parallel/resource_monitor
- features/web_scraping/parallel/scraping_task
- features/web_scraping/parallel/task_scheduler
- features/web_scraping/pivox_scraper
- features/web_scraping/rate_limiter
- features/web_scraping/rate_limiting/rate_limiter
- features/web_scraping/rate_limiting/request_queue
- features/web_scraping/robots_txt_handler
- features/web_scraping/scraping_exception
- features/web_scraping/scraping_job_scheduler
- features/web_scraping/scraping_logger
- features/web_scraping/scraping_task_queue
- features/web_scraping/selector/adaptive_selector
- features/web_scraping/selector/selector_validator
- features/web_scraping/site_reputation_tracker
- features/web_scraping/specialized_site_handlers
- features/web_scraping/streaming_html_parser
- features/web_scraping/user_agent_rotator
- features/web_scraping/web_scraper
- features/web_scraping/web_scraper_extension
- features/web_scraping/web_scraper_extensions
- features/web_scraping/web_scraper_performance
- pivox