robots_txt_processor 1.0.0 
robots_txt_processor: ^1.0.0 copied to clipboard
A robots.txt Dart package, useful to parse robots information and convert into more manageable object.
robots_txt_processor #
A robots.txt processor Dart package, converts from text to an easier to manipulate object.
Importing library #
import 'package:robots_txt_processor/robots_txt_processor.dart';
Process a robots.txt string with only one user-agent #
final RobotsTxt robotsTxt = processor.process('User-agent: googlebot');
print(robotsTxt.groups[0].userAgent);
Process a robots.txt string with only one sitemap #
final RobotsTxt robotsTxt = processor.process('Sitemap: http://example.com/sitemap.xml');
print(robotsTxt.sitemap[0]);
More info #
- Robots.txt specifications: https://developers.google.com/search/reference/robots_txt?hl=en
Metadata
0
likes
40
pub points
0%
popularity
Publisher
unverified uploader
Metadata
A robots.txt Dart package, useful to parse robots information and convert into more manageable object.
Repository (GitHub)
View/report issues
License
BSD-3-Clause (LICENSE)
