robots_txt 1.0.0 copy "robots_txt: ^1.0.0" to clipboard
robots_txt: ^1.0.0 copied to clipboard

outdated

Lightweight, sturdy and fully-documented parser of the `robots.txt` file.

1.0.0 #

  • Initial release
4
likes
0
pub points
31%
popularity

Publisher

verified publishervxern.dev

Lightweight, sturdy and fully-documented parser of the `robots.txt` file.

Repository (GitHub)
View/report issues

License

unknown (license)

Dependencies

sprint, web_scraper

More

Packages that depend on robots_txt