JobPysparkConfig class

Constructors

JobPysparkConfig({Input<List<String>?>? archiveUris, Input<List<String>?>? args, Input<List<String>?>? fileUris, Input<List<String>?>? jarFileUris, Input<JobPysparkConfigLoggingConfig?>? loggingConfig, required Input<String> mainPythonFileUri, Input<Map<String, String>?>? properties, Input<List<String>?>? pythonFileUris})

Creates a new JobPysparkConfig. archiveUris HCFS URIs of archives to be extracted in the working directory of .jar, .tar, .tar.gz, .tgz, and .zip. args The arguments to pass to the driver. fileUris HCFS URIs of files to be copied to the working directory of Python drivers and distributed tasks. Useful for naively parallel tasks. jarFileUris HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks. loggingConfig The runtime logging config of the job mainPythonFileUri The HCFS URI of the main Python file to use as the driver. Must be a .py file. properties A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code. pythonFileUris HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.

const

JobPysparkConfig.fromMap(Map<String, dynamic> map)

factory

Properties

archiveUris → Input<List<String>?>?

HCFS URIs of archives to be extracted in the working directory of .jar, .tar, .tar.gz, .tgz, and .zip.

final

args → Input<List<String>?>?

The arguments to pass to the driver.

final

fileUris → Input<List<String>?>?

HCFS URIs of files to be copied to the working directory of Python drivers and distributed tasks. Useful for naively parallel tasks.

final

hashCode → int

The hash code for this object.

no setter, inherited

jarFileUris → Input<List<String>?>?

HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.

final

loggingConfig → Input<JobPysparkConfigLoggingConfig?>?

The runtime logging config of the job

final

mainPythonFileUri → Input<String>

The HCFS URI of the main Python file to use as the driver. Must be a .py file.

final

properties → Input<Map<String, String>?>?

A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.

final

pythonFileUris → Input<List<String>?>?

HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.

final

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

Methods

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

toMap() → Map<String, dynamic>

toString() → String

A string representation of this object.

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited