OrtCompileApi class final

\brief The OrtCompileApi struct provides functions to compile ONNX models.

Execution providers that support compilation fuse a subgraph into an EPContext node that wraps a provider-specific binary representation of the subgraph. For more details about the EPContext design, refer to: \htmlonly EPContext design document. \endhtmlonly

Example (error handling not shown): OrtStatus* status = NULL; OrtCompileApi* compile_api = ort_api->GetCompileApi(); OrtModelCompilationOptions* compile_options = NULL;

status = compile_api->CreateModelCompilationOptionsFromSessionOptions(env, session_options, &compile_options); status = compile_api->ModelCompilationOptions_SetInputModelPath(compile_options, ORT_TSTR("model.onnx")); status = compile_api->ModelCompilationOptions_SetOutputModelPath(compile_options, ORT_TSTR("model.compiled.onnx")); status = compile_api->CompileModel(env, compile_options); compile_api->ReleaseModelCompilationOptions(compile_options);

\since Version 1.22.

Inheritance
Implemented types
Available extensions

Constructors

OrtCompileApi()

Properties

address → Pointer<T>

Available on T, provided by the StructAddress extension

The memory address of the underlying data.
no setter
CompileModel ↔ Pointer<NativeFunction<OrtStatusPtr Function(Pointer<OrtEnv> env, Pointer<OrtModelCompilationOptions> model_options)>>
\brief Compiles an input ONNX model with the given compilation options.
getter/setter pair
CreateModelCompilationOptionsFromSessionOptions ↔ Pointer<NativeFunction<OrtStatusPtr Function(Pointer<OrtEnv> env, Pointer<OrtSessionOptions> session_options, Pointer<Pointer<OrtModelCompilationOptions>> out)>>
\brief Creates an OrtModelCompilationOptions object from an existing OrtSessionOptions object.
getter/setter pair
hashCode → int
The hash code for this object.
no setterinherited
ModelCompilationOptions_SetEpContextBinaryInformation ↔ Pointer<NativeFunction<OrtStatusPtr Function(Pointer<OrtModelCompilationOptions> model_compile_options, Pointer<Char> output_directory, Pointer<Char> model_name)>>
Sets information related to EP context binary file.
getter/setter pair
ModelCompilationOptions_SetEpContextEmbedMode ↔ Pointer<NativeFunction<OrtStatusPtr Function(Pointer<OrtModelCompilationOptions> model_compile_options, Bool embed_ep_context_in_model)>>
\brief Enables or disables the embedding of EPContext binary data into the ep_cache_context attribute of EPContext nodes. Defaults to false.
getter/setter pair
ModelCompilationOptions_SetFlags ↔ Pointer<NativeFunction<OrtStatusPtr Function(Pointer<OrtModelCompilationOptions> model_compile_options, Uint32 flags)>>
\brief Sets flags from OrtCompileApiFlags that represent one or more boolean options to enable.
getter/setter pair
ModelCompilationOptions_SetGraphOptimizationLevel ↔ Pointer<NativeFunction<OrtStatusPtr Function(Pointer<OrtModelCompilationOptions> model_compile_options, UnsignedInt graph_optimization_level)>>
Set the graph optimization level.
getter/setter pair
ModelCompilationOptions_SetInputModelFromBuffer ↔ Pointer<NativeFunction<OrtStatusPtr Function(Pointer<OrtModelCompilationOptions> model_compile_options, Pointer<Void> input_model_data, Size input_model_data_size)>>
\brief Sets the buffer that stores the bytes of the loaded ONNX model to compile.
getter/setter pair
ModelCompilationOptions_SetInputModelPath ↔ Pointer<NativeFunction<OrtStatusPtr Function(Pointer<OrtModelCompilationOptions> model_compile_options, Pointer<Char> input_model_path)>>
\brief Sets the file path to the input ONNX model to compile.
getter/setter pair
ModelCompilationOptions_SetOutputModelBuffer ↔ Pointer<NativeFunction<OrtStatusPtr Function(Pointer<OrtModelCompilationOptions> model_compile_options, Pointer<OrtAllocator> allocator, Pointer<Pointer<Void>> output_model_buffer_ptr, Pointer<Size> output_model_buffer_size_ptr)>>
\brief Configures model compilation to store the output compiled ONNX model in a buffer.
getter/setter pair
ModelCompilationOptions_SetOutputModelExternalInitializersFile ↔ Pointer<NativeFunction<OrtStatusPtr Function(Pointer<OrtModelCompilationOptions> model_compile_options, Pointer<Char> external_initializers_file_path, Size external_initializers_size_threshold)>>
\brief Optionally sets the file that should store external initializers for the compiled ONNX model. If not set, initializers are stored within the model.
getter/setter pair
ModelCompilationOptions_SetOutputModelGetInitializerLocationFunc ↔ Pointer<NativeFunction<OrtStatusPtr Function(Pointer<OrtModelCompilationOptions> model_compile_options, OrtGetInitializerLocationFunc get_initializer_location_func, Pointer<Void> state)>>
\brief Sets a OrtGetInitializerLocationFunc function that is called by ORT for every initializer in the generated model. Allows implementer to specify whether initializers should be stored within the model or externally.
getter/setter pair
ModelCompilationOptions_SetOutputModelPath ↔ Pointer<NativeFunction<OrtStatusPtr Function(Pointer<OrtModelCompilationOptions> model_compile_options, Pointer<Char> output_model_path)>>
\brief Sets the file path for the output ONNX model generated by CompileModel.
getter/setter pair
ModelCompilationOptions_SetOutputModelWriteFunc ↔ Pointer<NativeFunction<OrtStatusPtr Function(Pointer<OrtModelCompilationOptions> model_compile_options, OrtWriteBufferFunc write_func, Pointer<Void> state)>>
\brief Sets a OrtWriteBufferFunc function that is called by ORT to write out the output model's serialized ONNX bytes.
getter/setter pair
ReleaseModelCompilationOptions ↔ Pointer<NativeFunction<Void Function(Pointer<OrtModelCompilationOptions> input)>>
\name OrtModelCompilationOptions @{
getter/setter pair
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited

Methods

noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
toString() → String
A string representation of this object.
inherited

Operators

operator ==(Object other) → bool
The equality operator.
inherited