liteRtLmOpenSessionBacking function

OpenSessionBacking liteRtLmOpenSessionBacking({
  1. required String abi,
  2. required BackendKind? backend,
  3. NpuStack? npuVendor,
  4. AndroidGpuFamily? androidGpu,
})

How the open sessions of a model loaded on backend on the host abi are backed: OpenSessionBacking.native only where that backend ran several live conversations correctly, OpenSessionBacking.virtual everywhere else. npuVendor is the stack an NPU load runs on; androidGpu the GPU family of an Android device, null when not read.

Implementation

OpenSessionBacking liteRtLmOpenSessionBacking({
  required String abi,
  required BackendKind? backend,
  NpuStack? npuVendor,
  AndroidGpuFamily? androidGpu,
}) {
  final verified = switch (backend) {
    BackendKind.cpu => liteRtLmConversationsCpuAbis.contains(abi),
    BackendKind.gpu =>
      abi == 'android_arm64'
          ? liteRtLmConversationsAndroidGpus.contains(androidGpu)
          : liteRtLmConversationsGpuAbis.contains(abi),
    BackendKind.npu =>
      npuVendor != null && liteRtLmConversationsNpus.contains((npuVendor, abi)),
    null => false,
  };
  return verified ? OpenSessionBacking.native : OpenSessionBacking.virtual;
}