TableExternalDataConfiguration class
Constructors
-
TableExternalDataConfiguration({required Input<bool> autodetect, Input<TableExternalDataConfigurationAvroOptions?>? avroOptions, Input<TableExternalDataConfigurationBigtableOptions?>? bigtableOptions, Input<String?>? compression, Input<String?>? connectionId, Input<TableExternalDataConfigurationCsvOptions?>? csvOptions, Input<List<String>?>? decimalTargetTypes, Input<String?>? fileSetSpecType, Input<TableExternalDataConfigurationGoogleSheetsOptions?>? googleSheetsOptions, Input<TableExternalDataConfigurationHivePartitioningOptions?>? hivePartitioningOptions, Input<bool?>? ignoreUnknownValues, Input<String?>? jsonExtension, Input<TableExternalDataConfigurationJsonOptions?>? jsonOptions, Input<int?>? maxBadRecords, Input<String?>? metadataCacheMode, Input<String?>? objectMetadata, Input<TableExternalDataConfigurationParquetOptions?>? parquetOptions, Input<String?>? referenceFileSchemaUri, Input<String?>? schema, Input<String?>? sourceFormat, required Input<List<String>> sourceUris})
-
Creates a new TableExternalDataConfiguration.
autodetect Let BigQuery try to autodetect the schema
avroOptions Additional options if sourceFormat is set to
bigtableOptions Additional properties to set if
compression The compression type of the data source.
connectionId The connection specifying the credentials to be used to read
csvOptions Additional properties to set if
decimalTargetTypes Defines the list of possible SQL data types to which the source decimal values are converted. This list and the precision and the scale parameters of the decimal field determine the target type. In the order of NUMERIC, BIGNUMERIC, and STRING, a type is picked if it is in the specified list and if it supports the precision and the scale. STRING supports all precision and scale values. If none of the listed types supports the precision and the scale, the type supporting the widest range in the specified list is picked, and if a value exceeds the supported range when reading the data, an error will be thrown.
fileSetSpecType Specifies how source URIs are interpreted for constructing the file set to load.
googleSheetsOptions Additional options if
hivePartitioningOptions When set, configures hive partitioning
ignoreUnknownValues Indicates if BigQuery should
jsonExtension Used to indicate that a JSON variant, rather than normal JSON, is being used as the sourceFormat. This should only be used in combination with the JSON source format. Valid values are: GEOJSON.
jsonOptions Additional properties to set if
maxBadRecords The maximum number of bad records that
metadataCacheMode Metadata Cache Mode for the table. Set this to enable caching of metadata from external data source. Valid values are AUTOMATIC and MANUAL.
objectMetadata Object Metadata is used to create Object Tables. Object Tables contain a listing of objects (with their metadata) found at the sourceUris. If objectMetadata is set, sourceFormat should be omitted.
parquetOptions Additional properties to set if
referenceFileSchemaUri When creating an external table, the user can provide a reference file with the table schema. This is enabled for the following formats: AVRO, PARQUET, ORC.
schema A JSON schema for the external table. Schema is required
sourceFormat The data format. Please see sourceFormat under
sourceUris A list of the fully-qualified URIs that point to
const
-
TableExternalDataConfiguration.fromMap(Map<String, dynamic> map)
-
factory
Properties
-
autodetect
→ Input<bool>
-
Let BigQuery try to autodetect the schema
and format of the table.
final
-
avroOptions
→ Input<TableExternalDataConfigurationAvroOptions?>?
-
Additional options if
sourceFormat is set to
"AVRO". Structure is documented below.
final
-
bigtableOptions
→ Input<TableExternalDataConfigurationBigtableOptions?>?
-
Additional properties to set if
sourceFormat is set to "BIGTABLE". Structure is documented below.
final
-
compression
→ Input<String?>?
-
The compression type of the data source.
Valid values are "NONE" or "GZIP".
final
-
connectionId
→ Input<String?>?
-
The connection specifying the credentials to be used to read
external storage, such as Azure Blob, Cloud Storage, or S3. The
connectionId can have
the form {{project}}.{{location}}.{{connection_id}}
or projects/{{project}}/locations/{{location}}/connections/{{connection_id}}.
final
-
csvOptions
→ Input<TableExternalDataConfigurationCsvOptions?>?
-
Additional properties to set if
sourceFormat is set to "CSV". Structure is documented below.
final
-
decimalTargetTypes
→ Input<List<String>?>?
-
Defines the list of possible SQL data types to which the source decimal values are converted. This list and the precision and the scale parameters of the decimal field determine the target type. In the order of NUMERIC, BIGNUMERIC, and STRING, a type is picked if it is in the specified list and if it supports the precision and the scale. STRING supports all precision and scale values. If none of the listed types supports the precision and the scale, the type supporting the widest range in the specified list is picked, and if a value exceeds the supported range when reading the data, an error will be thrown.
final
-
fileSetSpecType
→ Input<String?>?
-
Specifies how source URIs are interpreted for constructing the file set to load.
By default source URIs are expanded against the underlying storage.
Other options include specifying manifest files. Only applicable to object storage systems. Docs
final
-
googleSheetsOptions
→ Input<TableExternalDataConfigurationGoogleSheetsOptions?>?
-
Additional options if
sourceFormat is set to "GOOGLE_SHEETS". Structure is
documented below.
final
-
hashCode
→ int
-
The hash code for this object.
no setterinherited
-
hivePartitioningOptions
→ Input<TableExternalDataConfigurationHivePartitioningOptions?>?
-
When set, configures hive partitioning
support. Not all storage formats support hive partitioning -- requesting hive
partitioning on an unsupported format will lead to an error, as will providing
an invalid specification. Structure is documented below.
final
-
ignoreUnknownValues
→ Input<bool?>?
-
Indicates if BigQuery should
allow extra values that are not represented in the table schema.
If true, the extra values are ignored. If false, records with
extra columns are treated as bad records, and if there are too
many bad records, an invalid error is returned in the job result.
The default value is false.
final
-
jsonExtension
→ Input<String?>?
-
Used to indicate that a JSON variant, rather than normal JSON, is being used as the sourceFormat. This should only be used in combination with the
JSON source format. Valid values are: GEOJSON.
final
-
jsonOptions
→ Input<TableExternalDataConfigurationJsonOptions?>?
-
Additional properties to set if
sourceFormat is set to "JSON". Structure is documented below.
final
-
maxBadRecords
→ Input<int?>?
-
The maximum number of bad records that
BigQuery can ignore when reading data.
final
-
metadataCacheMode
→ Input<String?>?
-
Metadata Cache Mode for the table. Set this to enable caching of metadata from external data source. Valid values are
AUTOMATIC and MANUAL.
final
-
objectMetadata
→ Input<String?>?
-
Object Metadata is used to create Object Tables. Object Tables contain a listing of objects (with their metadata) found at the sourceUris. If
objectMetadata is set, sourceFormat should be omitted.
final
-
parquetOptions
→ Input<TableExternalDataConfigurationParquetOptions?>?
-
Additional properties to set if
sourceFormat is set to "PARQUET". Structure is documented below.
final
-
referenceFileSchemaUri
→ Input<String?>?
-
When creating an external table, the user can provide a reference file with the table schema. This is enabled for the following formats: AVRO, PARQUET, ORC.
final
-
runtimeType
→ Type
-
A representation of the runtime type of the object.
no setterinherited
-
schema
→ Input<String?>?
-
A JSON schema for the external table. Schema is required
for CSV and JSON formats if autodetect is not on. Schema is disallowed
for Google Cloud Bigtable, Cloud Datastore backups, Avro, Iceberg, ORC and Parquet formats.
~>NOTE: Because this field expects a JSON string, any changes to the
string will create a diff, even if the JSON itself hasn't changed.
Furthermore drift for this field cannot not be detected because BigQuery
only uses this schema to compute the effective schema for the table, therefore
any changes on the configured value will force the table to be recreated.
This schema is effectively only applied when creating a table from an external
datasource, after creation the computed schema will be stored in
google_bigquery_table.schema
final
-
sourceFormat
→ Input<String?>?
-
The data format. Please see sourceFormat under
ExternalDataConfiguration
in Bigquery's public API documentation for supported formats. To use "GOOGLE_SHEETS"
the
scopes must include "https://www.googleapis.com/auth/drive.readonly".
final
-
sourceUris
→ Input<List<String>>
-
A list of the fully-qualified URIs that point to
your data in Google Cloud.
final
