revali_redis 0.2.0

A Redis Streams message broker for Revali, built on the framework's MessageBroker contract.

CHANGELOG

0.2.0 | 08.15.26

Features

• Back off between retries, via retryAfter (default 5 seconds). Redelivery previously ran as fast as the read loop — fail, notice, claim, fail again — so a handler whose dependency was thirty seconds into a restart spent its entire maxDeliveries allowance inside that window, and a message that would have succeeded on the next attempt was dead-lettered instead. The wait doubles with each delivery already made and is capped at 32×, so a large maxDeliveries cannot push the last attempt days out; at the default the attempts land roughly 5s, 10s, 20s and 40s after the first failure. It is measured against Redis's own idle time for the entry rather than a timer in the process, so a consumer that restarts reads the same schedule the old one was working to instead of starting every entry's wait over. The claim then uses that backoff as its min-idle-time rather than 0, so an entry redelivered between the scan and the claim is refused by Redis instead of having a running handler restarted underneath it. Duration.zero restores the previous behaviour; dead-lettering is never delayed by it, since an entry with no allowance left has nothing to wait for.
• RedisBroker.connect() forwards retryAfter too, with the same default as the constructor — the field-by-field test against a constructor-built broker covers it, so the two cannot drift.
• RedisBroker scopes its consumer name through IsolateIdentity.scopeName rather than a private copy of the rule. Behaviour is unchanged; the point is that the rule now has one definition, and a broker written outside this repository can call the same thing instead of rediscovering the collision.

Fixes

• maxDeliveries allowed one more delivery than it named. The check was deliveries > maxDeliveries, so maxDeliveries: 3 ran the handler four times before dead-lettering. It is now the total, counted the way Redis counts it and including the first delivery: at 5, a handler that always throws runs five times and the sixth pass dead-letters. Proved against a real Redis, which owns the counter — the integration test asserts three attempts for maxDeliveries: 3, and reports four if the operator is put back.
• The repair paths starved under load. Retrying this consumer's own pending entries, and reclaiming another's, ran only on a pass whose read came back empty — and a queue with work always waiting never has one. For as long as the load lasted a failed message was neither retried nor dead-lettered: the same silent stall the retry path was added to end, reappearing under the one condition nobody had thought to test. Draining the queue is still the priority, so they stay off the hot path until due, but there is now a floor — they run at least once per retryAfter, or per blockFor if that is longer, and nothing can come due sooner than that anyway.

0.2.0 | 08.15.26

Features

• Back off between retries, via retryAfter (default 5 seconds). Redelivery previously ran as fast as the read loop — fail, notice, claim, fail again — so a handler whose dependency was thirty seconds into a restart spent its entire maxDeliveries allowance inside that window, and a message that would have succeeded on the next attempt was dead-lettered instead. The wait doubles with each delivery already made and is capped at 32×, so a large maxDeliveries cannot push the last attempt days out; at the default the attempts land roughly 5s, 10s, 20s and 40s after the first failure. It is measured against Redis's own idle time for the entry rather than a timer in the process, so a consumer that restarts reads the same schedule the old one was working to instead of starting every entry's wait over. The claim then uses that backoff as its min-idle-time rather than 0, so an entry redelivered between the scan and the claim is refused by Redis instead of having a running handler restarted underneath it. Duration.zero restores the previous behaviour; dead-lettering is never delayed by it, since an entry with no allowance left has nothing to wait for.
• RedisBroker.connect() forwards retryAfter too, with the same default as the constructor — the field-by-field test against a constructor-built broker covers it, so the two cannot drift.
• RedisBroker scopes its consumer name through IsolateIdentity.scopeName rather than a private copy of the rule. Behaviour is unchanged; the point is that the rule now has one definition, and a broker written outside this repository can call the same thing instead of rediscovering the collision.

Fixes

• maxDeliveries allowed one more delivery than it named. The check was deliveries > maxDeliveries, so maxDeliveries: 3 ran the handler four times before dead-lettering. It is now the total, counted the way Redis counts it and including the first delivery: at 5, a handler that always throws runs five times and the sixth pass dead-letters. Proved against a real Redis, which owns the counter — the integration test asserts three attempts for maxDeliveries: 3, and reports four if the operator is put back.
• The repair paths starved under load. Retrying this consumer's own pending entries, and reclaiming another's, ran only on a pass whose read came back empty — and a queue with work always waiting never has one. For as long as the load lasted a failed message was neither retried nor dead-lettered: the same silent stall the retry path was added to end, reappearing under the one condition nobody had thought to test. Draining the queue is still the priority, so they stay off the hot path until due, but there is now a floor — they run at least once per retryAfter, or per blockFor if that is longer, and nothing can come due sooner than that anyway.

0.1.0 | 08.15.26

Features

• A MessageBroker backed by Redis Streams, not pub/sub: Redis pub/sub is fire-and-forget, so a consumer that is restarting simply misses whatever was published — the opposite of what a work queue is for. Streams persist, and consumer groups give one delivery per group with redelivery until acknowledged.
• Speaks RESP over a socket with no third-party client, so the package has no dependency beyond revali_core. The decoder returns null on an incomplete reply and the connection buffers across packets, because a reply split across two packets is routine for a large XREADGROUP batch and decoding it as if whole desynchronises the connection rather than failing outright.
• Acknowledges only on success, and retries what a handler left unacknowledged. XREADGROUP ... > returns only messages never delivered to anyone, so an entry a handler threw on is pending and no ordinary read touches it again; each pass now scans this consumer's own pending entries and re-delivers them with XCLAIM, which is also what makes Redis count the delivery. Past maxDeliveries the entry is dead-lettered instead. None of this depends on claimAfter -- reclaiming is about entries another consumer abandoned, and is still opt-in, but a message failing in this consumer used to stop silently: never redelivered, never dead-lettered, never reported.
• Delivery is at least once; handlers must be idempotent. That is the broker contract, not a limitation of this implementation.
• Integration tests against a real Redis, skipped by default so a machine without one still gets a green suite. Run them with dart test --run-skipped --tags integration, pointing at a server with REDIS_TEST_HOST / REDIS_TEST_PORT.
• Reclaim entries a dead consumer left pending, via claimAfter. Redis tracks pending entries per consumer name, so a replica that dies mid-message strands them in a list nobody else reads — and a pod that is replaced rather than restarted never comes back to claim them. Off by default, since it changes when a message is redelivered. Set it well above the time a healthy handler takes, or a slow handler's work is taken from underneath it and processed twice.
• Dead-letter a message that keeps failing, after maxDeliveries (default 5), onto <topic>.dead with headers recording why and where it came from. Reclaiming without this turns a message that always fails into a retry storm: claimed, failed, left pending, claimed again, forever. The entry is acknowledged only after the copy succeeds, so a failed copy leaves it pending rather than losing it.
• Reclaiming runs only when a read came back with no fresh work, so it spends round trips on bookkeeping only when the queue is idle.
• RedisBroker.connect() takes every option the constructor does — blockFor, batchSize, claimAfter, maxDeliveries and deadLetterSuffix — each forwarded with an identical default, so no existing caller changes behaviour. It previously took only host, port and consumerName, which made the documented entry point the easy half of the API: claimAfter is what recovers work stranded by a consumer that died mid-message, and reaching it meant abandoning connect() and hand-wiring a ReconnectingRedisConnection plus a per-subscription connection factory — the exact wiring connect() exists to do. A test compares a connect()-built broker against a constructor-built one field by field rather than against literals, so a future edit to one default fails there instead of the two drifting apart quietly.
• Each worker isolate gets its own consumer name. Redis keys unacknowledged entries on the consumer name, so two consumers sharing one hide each other's pending work; an app with AppConfig.workers above 1 walked into that by construction, since every isolate runs the same createBroker() override and named itself identically — colliding with itself rather than with another replica. The name is now scoped by the isolate index the generated server publishes, which needs nothing from the app. The parent, index 0, is deliberately left alone rather than becoming -0: suffixing it too would be tidier and would also rename the consumer of every app that upgrades, stranding everything pending under the old name where nothing reads it again — recoverable only if claimAfter happens to be set, and it is off by default. Only the newly added worker isolates take names that never existed before. Asserted on the bytes of the XREADGROUP and XPENDING commands rather than on the getter, since a getter that agrees with itself proves nothing about what Redis is keying on.
• Survive a restart of the Redis server, which previously killed the process. Writing to the dead socket escaped as an unhandled SocketException because nothing observed the socket's done future — write errors do not arrive on the read stream. Behind that sat two further failures: the link was never reopened, and a consumer group lost with the server was never recreated, so every later XREADGROUP answered NOGROUP while the connection looked perfectly healthy. A connection now reopens and retries once; a RedisError is rethrown untouched, since that is the server answering rather than the link failing, and reconnecting would swallow the NOGROUP the read loop needs. The group is recreated from 0 rather than $: the stream is gone too, so recovery races the next publish, and a message that lands before the group is back would otherwise fall before $ and never be delivered to anyone — with the publish succeeding and the consumer looking fine. Recreation is backed off so a server still coming up does not get a tight XGROUP CREATE loop.

0.1.0

• Initial release.