sloth_link_crawler 0.0.1

(WIP!) Web crawler to crawl all links of a website. Looking foward to respect robots.txt and use custom user agents.

example/sloth_link_crawler_example.dart

// TODO: Add example
void main() {}