genkit_hybrid 0.2.0
genkit_hybrid: ^0.2.0 copied to clipboard
Provider-agnostic hybrid routing for Genkit — combine on-device and cloud models behind one routing policy.
Changelog #
0.2.0 #
- Add
CapabilityStrategy: route by request-required capabilities (vision/audio/tools/json). - Add
CostStrategy: budget-gate a premium branch against an app-supplied signal. - Add
cascadeModel: escalate to a premium branch when anacceptpredicate rejects the cheap response (non-streaming v1).
0.1.1 #
- Bump
genkitto^0.15.1(from^0.14.1). No API or behavioral changes; 32 tests green.
0.1.0 #
Initial release.
hybridModel(branches, strategy)— combine N named GenkitModelactions behind one routing policy; returns an ordinaryModel.hybridModelOnDeviceCloud(onDevice, cloud, strategy)— binary façade withkOnDevice/kCloudkeys.RoutingStrategyinterface returning an ordered list of branch keys (single = pre-routing, multi = fallback, empty = no decision).- Built-in strategies:
PreRoutingStrategy,FallbackStrategy,ConnectivityStrategy,InputSizeStrategy, and combinatorsFirstMatch,WithFallback. - Streaming fallback only before the first emitted token.
- Transient-vs-permanent error policy: fallback on availability errors, immediate propagation of permanent ones.