createCrawler method

Future<void> createCrawler({ required String name, required String role, required CrawlerTargets targets, List<String>? classifiers, String? configuration, String? crawlerSecurityConfiguration, String? databaseName, String? description, LineageConfiguration? lineageConfiguration, RecrawlPolicy? recrawlPolicy, String? schedule, SchemaChangePolicy? schemaChangePolicy, String? tablePrefix, Map<String, String>? tags, })

Creates a new crawler with specified targets, role, configuration, and optional schedule. At least one crawl target must be specified, in the s3Targets field, the jdbcTargets field, or the DynamoDBTargets field.

May throw InvalidInputException. May throw AlreadyExistsException. May throw OperationTimeoutException. May throw ResourceNumberLimitExceededException.

Parameter name : Name of the new crawler.

Parameter role : The IAM role or Amazon Resource Name (ARN) of an IAM role used by the new crawler to access customer resources.

Parameter targets : A list of collection of targets to crawl.

Parameter classifiers : A list of custom classifiers that the user has registered. By default, all built-in classifiers are included in a crawl, but these custom classifiers always override the default classifiers for a given classification.

Parameter configuration : Crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Configuring a Crawler.

Parameter crawlerSecurityConfiguration : The name of the SecurityConfiguration structure to be used by this crawler.

Parameter databaseName : The AWS Glue database where results are written, such as: arn:aws:daylight:us-east-1::database/sometable/*.

Parameter description : A description of the new crawler.

Parameter lineageConfiguration : Specifies data lineage configuration settings for the crawler.

Parameter recrawlPolicy : A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run.

Parameter schedule : A cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *).

Parameter schemaChangePolicy : The policy for the crawler's update and deletion behavior.

Parameter tablePrefix : The table prefix used for catalog tables that are created.

Parameter tags : The tags to use with this crawler request. You may use tags to limit access to the crawler. For more information about tags in AWS Glue, see AWS Tags in AWS Glue in the developer guide.

Implementation

Future<void> createCrawler({
  required String name,
  required String role,
  required CrawlerTargets targets,
  List<String>? classifiers,
  String? configuration,
  String? crawlerSecurityConfiguration,
  String? databaseName,
  String? description,
  LineageConfiguration? lineageConfiguration,
  RecrawlPolicy? recrawlPolicy,
  String? schedule,
  SchemaChangePolicy? schemaChangePolicy,
  String? tablePrefix,
  Map<String, String>? tags,
}) async {
  ArgumentError.checkNotNull(name, 'name');
  _s.validateStringLength(
    'name',
    name,
    1,
    255,
    isRequired: true,
  );
  ArgumentError.checkNotNull(role, 'role');
  ArgumentError.checkNotNull(targets, 'targets');
  _s.validateStringLength(
    'crawlerSecurityConfiguration',
    crawlerSecurityConfiguration,
    0,
    128,
  );
  _s.validateStringLength(
    'description',
    description,
    0,
    2048,
  );
  _s.validateStringLength(
    'tablePrefix',
    tablePrefix,
    0,
    128,
  );
  final headers = <String, String>{
    'Content-Type': 'application/x-amz-json-1.1',
    'X-Amz-Target': 'AWSGlue.CreateCrawler'
  };
  await _protocol.send(
    method: 'POST',
    requestUri: '/',
    exceptionFnMap: _exceptionFns,
    // TODO queryParams
    headers: headers,
    payload: {
      'Name': name,
      'Role': role,
      'Targets': targets,
      if (classifiers != null) 'Classifiers': classifiers,
      if (configuration != null) 'Configuration': configuration,
      if (crawlerSecurityConfiguration != null)
        'CrawlerSecurityConfiguration': crawlerSecurityConfiguration,
      if (databaseName != null) 'DatabaseName': databaseName,
      if (description != null) 'Description': description,
      if (lineageConfiguration != null)
        'LineageConfiguration': lineageConfiguration,
      if (recrawlPolicy != null) 'RecrawlPolicy': recrawlPolicy,
      if (schedule != null) 'Schedule': schedule,
      if (schemaChangePolicy != null)
        'SchemaChangePolicy': schemaChangePolicy,
      if (tablePrefix != null) 'TablePrefix': tablePrefix,
      if (tags != null) 'Tags': tags,
    },
  );
}