genkit_hybrid 0.1.0
genkit_hybrid: ^0.1.0 copied to clipboard
Provider-agnostic hybrid routing for Genkit — combine on-device and cloud models behind one routing policy.
Changelog #
0.1.0 #
Initial release.
hybridModel(branches, strategy)— combine N named GenkitModelactions behind one routing policy; returns an ordinaryModel.hybridModelOnDeviceCloud(onDevice, cloud, strategy)— binary façade withkOnDevice/kCloudkeys.RoutingStrategyinterface returning an ordered list of branch keys (single = pre-routing, multi = fallback, empty = no decision).- Built-in strategies:
PreRoutingStrategy,FallbackStrategy,ConnectivityStrategy,InputSizeStrategy, and combinatorsFirstMatch,WithFallback. - Streaming fallback only before the first emitted token.
- Transient-vs-permanent error policy: fallback on availability errors, immediate propagation of permanent ones.