RobotsTxt class

Generates robots.txt content for your website.

final robots = RobotsTxt(
  baseUrl: 'https://myapp.com',
  rules: [
    RobotsRule(
      userAgent: '*',
      allow: ['/'],
      disallow: ['/admin', '/api', '/_internal'],
    ),
    RobotsRule(
      userAgent: 'Googlebot',
      allow: ['/'],
      crawlDelay: 1,
    ),
  ],
  sitemaps: ['/sitemap.xml'],
  additionalLines: ['# Updated: 2025-06-01'],
);

final content = robots.generate();

Constructors

RobotsTxt({required String baseUrl, List<RobotsRule> rules = const [], List<String>? sitemaps, List<String>? additionalLines})

const

RobotsTxt.allowAll({required String baseUrl, String sitemapPath = '/sitemap.xml'})

Create a simple robots.txt that allows everything

factory

RobotsTxt.blockAll({required String baseUrl})

Create a robots.txt that blocks everything

factory

Properties

additionalLines → List<String>?

final

baseUrl → String

final

hashCode → int

The hash code for this object.

no setter, inherited

rules → List<RobotsRule>

final

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

sitemaps → List<String>?

final

Methods

generate() → String

Generate the robots.txt content string

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

toString() → String

A string representation of this object.

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited