Compile property

Pointer<NativeFunction<OrtStatusPtr Function(Pointer<OrtEp> this_ptr, Pointer<Pointer<OrtGraph>> graphs, Pointer<Pointer<OrtNode>> fused_nodes, Size count, Pointer<Pointer<OrtNodeComputeInfo>> node_compute_infos, Pointer<Pointer<OrtNode>> ep_context_nodes)>> Compile

\brief Compile OrtGraph instances assigned to the OrtEp. Implementer must set a OrtNodeComputeInfo instance for each OrtGraph in order to define its computation function.

If the session is configured to generate a pre-compiled model, the execution provider must return EPContext nodes, as OrtNode instances, that ONNX Runtime uses to create a pre-compiled model, known as an "EPContext model". An EPContext model contains EPContext nodes. Each EPContext node encapsulates the pre-compiled binary data for a OrtGraph compiled for a specific execution provider. For more details about the EPContext design, refer to: \htmlonly EPContext design document. \endhtmlonly

\paramin this_ptr The OrtEp instance. \paramin graphs Array of count OrtGraph instances to compile. Each graph contains only the nodes for which the execution provider indicated support. Nested subgraphs contained by a node, such as an If or Loop, have separate OrtGraph instances. \paramin fused_nodes Array of count fused nodes that will replace the compiled graphs. Each fused node is an OrtNode initialized with the intended fused node name and input/output information. \paramin count The number of OrtGraph instances to compile. \paramout node_compute_infos Array of count OrtNodeComputeInfo instances that define each OrtGraph instance's computation function. The implementer allocates the OrtNodeComputeInfo instances. ORT calls ReleaseNodeComputeInfos() to release multiple instances in a batch. \paramout ep_context_nodes Output array of count OrtNode instances, each representing an EPContext node for a compiled OrtGraph. The execution provider must use OrtModelEditorApi::CreateNode to create the OrtNode instances. ONNX Runtime takes ownership of the OrtNode instances, so the execution provider must NOT call OrtApi::ReleaseNode. Should be ignored if the session is not configured to generate an EPContext model.

\snippet{doc} snippets.dox OrtStatus Return Value

\note Do NOT cache the provided OrtGraph instances in any of the OrtNodeComputeInfo functions because the graphs are only valid for the duration of the call to Compile. Any graph/node/input/output names that are needed by the OrtNodeComputeInfo functions must be copied and stored by the OrtEp.

\note As of version 1.24, implementation of this function is optional if the EP does not compile nodes and uses a kernel registry instead.

\since Version 1.23.

Implementation

external ffi.Pointer<
  ffi.NativeFunction<
    OrtStatusPtr Function(
      ffi.Pointer<OrtEp> this_ptr,
      ffi.Pointer<ffi.Pointer<OrtGraph>> graphs,
      ffi.Pointer<ffi.Pointer<OrtNode>> fused_nodes,
      ffi.Size count,
      ffi.Pointer<ffi.Pointer<OrtNodeComputeInfo>> node_compute_infos,
      ffi.Pointer<ffi.Pointer<OrtNode>> ep_context_nodes,
    )
  >
>
Compile;