BenchmarkApi class

Constructors

BenchmarkApi([ApiClient? apiClient])

Properties

apiClient → ApiClient
final
hashCode → int
The hash code for this object.
no setterinherited
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited

Methods

getBenchmarkCatalog() → Future<BenchmarkCatalog?>
Is the canonical public benchmarks this arena runs — the id, title, axis, item count and upstream source of each, with native marking the ones the standardized harness runs today; the rest are registered and adapter-pending.
getBenchmarkCatalogWithHttpInfo() → Future<Response>
Is the canonical public benchmarks this arena runs — the id, title, axis, item count and upstream source of each, with native marking the ones the standardized harness runs today; the rest are registered and adapter-pending.
getBenchmarkClaims({String? benchmark, String? model, String? provider, String? source_, String? protocol}) → Future<ClaimsOut?>
Lists the effective published claims: what the leaderboard will use for each (benchmark, model) after the seed, the import and any stored correction are layered.
getBenchmarkClaimsWithHttpInfo({String? benchmark, String? model, String? provider, String? source_, String? protocol}) → Future<Response>
Lists the effective published claims: what the leaderboard will use for each (benchmark, model) after the seed, the import and any stored correction are layered.
getBenchmarkCompare(String a, String b, {String? benchmark}) → Future<Pairing?>
Is the ONLY valid arm-vs-arm test: it pairs the two models on the items BOTH completed, and answers rescue and damage counts with an exact-McNemar p.
getBenchmarkCompareWithHttpInfo(String a, String b, {String? benchmark}) → Future<Response>
Is the ONLY valid arm-vs-arm test: it pairs the two models on the items BOTH completed, and answers rescue and damage counts with an exact-McNemar p.
getBenchmarkHistory({String? benchmark, String? model}) → Future<HistoryOut?>
Returns each model's measured score per run over time, oldest first, with the change between runs.
getBenchmarkHistoryWithHttpInfo({String? benchmark, String? model}) → Future<Response>
Returns each model's measured score per run over time, oldest first, with the change between runs.
getBenchmarkLeaderboard({String? benchmark}) → Future<Leaderboard?>
Answers one row per model for the benchmark named — what our own harness measured, beside what the vendor claims, and the gap between them.
getBenchmarkLeaderboardWithHttpInfo({String? benchmark}) → Future<Response>
Answers one row per model for the benchmark named — what our own harness measured, beside what the vendor claims, and the gap between them.
getBenchmarkPresets() → Future<PresetList?>
Are the router blends available to compose from — a named set of model arms, the rank they escalate through and the panel width that bounds fan-out — each served by the model layer as enso-
getBenchmarkPresetsWithHttpInfo() → Future<Response>
Are the router blends available to compose from — a named set of model arms, the rank they escalate through and the panel width that bounds fan-out — each served by the model layer as enso-
noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
postBenchmarkClaims(PutClaimsIn putClaimsIn) → Future<PutClaimsOut?>
Records published claims: one to correct a number, many to import a leaderboard.
postBenchmarkClaimsWithHttpInfo(PutClaimsIn putClaimsIn) → Future<Response>
Records published claims: one to correct a number, many to import a leaderboard.
postBenchmarkPresets(Preset preset) → Future<PresetAccepted?>
Validates a router blend — its name, its arms, the rank they escalate through and the panel fan-out width — and answers 202 with the preset and the enso-
postBenchmarkPresetsWithHttpInfo(Preset preset) → Future<Response>
Validates a router blend — its name, its arms, the rank they escalate through and the panel fan-out width — and answers 202 with the preset and the enso-
postBenchmarkRuns(Suite suite) → Future<Admission?>
Admits and queues a benchmark run against a model or your own endpoint, and answers 202 with the receipt.
postBenchmarkRunsWithHttpInfo(Suite suite) → Future<Response>
Admits and queues a benchmark run against a model or your own endpoint, and answers 202 with the receipt.
toString() → String
A string representation of this object.
inherited

Operators

operator ==(Object other) → bool
The equality operator.
inherited