createDataSourceFromS3 method
- required String dataSourceId,
- required S3DataSpec dataSpec,
- bool? computeStatistics,
- String? dataSourceName,
Creates a DataSource
object. A DataSource
references data that can be used to perform CreateMLModel
,
CreateEvaluation
, or CreateBatchPrediction
operations.
CreateDataSourceFromS3
is an asynchronous operation. In
response to CreateDataSourceFromS3
, Amazon Machine Learning
(Amazon ML) immediately returns and sets the DataSource
status to PENDING
. After the DataSource
has been
created and is ready for use, Amazon ML sets the Status
parameter to COMPLETED
. DataSource
in the
COMPLETED
or PENDING
state can be used to
perform only CreateMLModel
, CreateEvaluation
or
CreateBatchPrediction
operations.
If Amazon ML can't accept the input source, it sets the
Status
parameter to FAILED
and includes an error
message in the Message
attribute of the
GetDataSource
operation response.
The observation data used in a DataSource
should be ready to
use; that is, it should have a consistent structure, and missing data
values should be kept to a minimum. The observation data must reside in
one or more .csv files in an Amazon Simple Storage Service (Amazon S3)
location, along with a schema that describes the data items by name and
type. The same schema must be used for all of the data files referenced by
the DataSource
.
After the DataSource
has been created, it's ready to use in
evaluations and batch predictions. If you plan to use the
DataSource
to train an MLModel
, the
DataSource
also needs a recipe. A recipe describes how each
input variable will be used in training an MLModel
. Will the
variable be included or excluded from training? Will the variable be
manipulated; for example, will it be combined with another variable or
will it be split apart into word combinations? The recipe provides answers
to these questions.
May throw InvalidInputException. May throw InternalServerException. May throw IdempotentParameterMismatchException.
Parameter dataSourceId
:
A user-supplied identifier that uniquely identifies the
DataSource
.
Parameter dataSpec
:
The data specification of a DataSource
:
- DataLocationS3 - The Amazon S3 location of the observation data.
-
DataSchemaLocationS3 - The Amazon S3 location of the
DataSchema
. -
DataSchema - A JSON string representing the schema. This is not required
if
DataSchemaUri
is specified. -
DataRearrangement - A JSON string that represents the splitting and
rearrangement requirements for the
Datasource
.Sample -
"{"splitting":{"percentBegin":10,"percentEnd":60}}"
Parameter computeStatistics
:
The compute statistics for a DataSource
. The statistics are
generated from the observation data referenced by a
DataSource
. Amazon ML uses the statistics internally during
MLModel
training. This parameter must be set to
true
if the DataSource
needs to be
used for
MLModel
training.
Parameter dataSourceName
:
A user-supplied name or description of the DataSource
.
Implementation
Future<CreateDataSourceFromS3Output> createDataSourceFromS3({
required String dataSourceId,
required S3DataSpec dataSpec,
bool? computeStatistics,
String? dataSourceName,
}) async {
ArgumentError.checkNotNull(dataSourceId, 'dataSourceId');
_s.validateStringLength(
'dataSourceId',
dataSourceId,
1,
64,
isRequired: true,
);
ArgumentError.checkNotNull(dataSpec, 'dataSpec');
_s.validateStringLength(
'dataSourceName',
dataSourceName,
0,
1024,
);
final headers = <String, String>{
'Content-Type': 'application/x-amz-json-1.1',
'X-Amz-Target': 'AmazonML_20141212.CreateDataSourceFromS3'
};
final jsonResponse = await _protocol.send(
method: 'POST',
requestUri: '/',
exceptionFnMap: _exceptionFns,
// TODO queryParams
headers: headers,
payload: {
'DataSourceId': dataSourceId,
'DataSpec': dataSpec,
if (computeStatistics != null) 'ComputeStatistics': computeStatistics,
if (dataSourceName != null) 'DataSourceName': dataSourceName,
},
);
return CreateDataSourceFromS3Output.fromJson(jsonResponse.body);
}