SetSharedPrePackedWeight property

Pointer<NativeFunction<OrtStatusPtr Function(Pointer<OrtKernelImpl> this_ptr, Pointer<Pointer<Void>> buffer_data_ptrs, Pointer<Size> buffer_data_sizes, Size num_buffers, Int input_index)>> SetSharedPrePackedWeight
getter/setter pair

\brief Optional function that receives data for a shared pre-packed weight from ORT.

ORT calls this function after calling OrtKernelImpl::PrePackWeight for a specific input_index if:

  • OrtKernelImpl::PrePackWeight set the output parameter is_packed to true.
  • OrtKernelImpl::PrePackWeight stored weight data to share into the provided OrtSharedPrePackedWeightCache parameter (prepacked_weight_cache) via the API SharedPrePackedWeightCache_StoreWeightData.

Refer to the description of the "sharing-mode" in the documentation for OrtKernelImpl::PrePackWeight().

\note ORT will not call this function for an input_index that a previous call to OrtKernelImpl::PrePackWeight() did not elect to pre-pack and share.

\note This function is based on the internal OpKernel::UseSharedPrePackedBuffers() virtual function used within ORT.

\paramin this_ptr The OrtKernelImpl instance. \paramin buffer_data_ptrs An array of buffer data pointers that collectively hold the pre-packed data for a single shared weight. The buffers are provided in the same order and with the same contents (in a potentially different memory location) as the buffers passed into SharedPrePackedWeightCache_StoreWeightData() within the OrtKernelImpl::PrePackWeight() call for the same input_index. \paramin buffer_data_sizes An array of buffer byte sizes, one per element in buffer_data_ptrs. \paramin num_buffers The number of buffers used to store the data for the shared pre-packed weight. Specifies the number of elements in the buffer_data_ptrs and buffer_data_sizes arrays. \paramin input_index The input index of the tensor in this kernel. This index identifies the identity of the weight.

\snippet{doc} snippets.dox OrtStatus Return Value

\note Implementation of this function is generally optional. It is only required if OrtKernelImpl::PrePack() elects to share pre-packed weights.

\since Version 1.24.

Implementation

external ffi.Pointer<
  ffi.NativeFunction<
    OrtStatusPtr Function(
      ffi.Pointer<OrtKernelImpl> this_ptr,
      ffi.Pointer<ffi.Pointer<ffi.Void>> buffer_data_ptrs,
      ffi.Pointer<ffi.Size> buffer_data_sizes,
      ffi.Size num_buffers,
      ffi.Int input_index,
    )
  >
>
SetSharedPrePackedWeight;