Repository navigation
Add Requeuer hook preferred by the retry path - #6
Merged
Merged
Conversation
Add an optional Requeuer interface a Queue may implement to express the
retry move ("this failed, put it back") distinctly from Enqueue ("add a
new job"). The worker retry path now prefers Requeue with the computed
backoff when the queue implements it, and falls back to Enqueue
otherwise. RedisQueue does not implement Requeuer, so its behavior is
unchanged.
This lets a queue whose waiting and in-flight jobs live in separate
structures restore a failed in-flight job to its position instead of
enqueuing a duplicate.
EnqueuedAt is the Enqueue path's backoff channel (RedisQueue reads it as the score). The Requeuer path receives the backoff explicitly, so leave EnqueuedAt untouched there rather than bumping the persisted job's enqueue time on every retry for no reason a Requeuer should reason about.
A handler error reaching retry is a *wrappedHandlerError, which Worker.start deliberately suppresses. Returning it after a failed Requeue hid infrastructure errors that can strand an in-flight job until its lease lapses. On a Requeue failure, return a distinct error (not a wrappedHandlerError) that wraps the Requeue error, so the worker's ErrorFunc reports it. The Enqueue path is unchanged.
Set EnqueuedAt = now.Add(d) unconditionally, reverting the earlier Enqueue-only scoping. EnqueuedAt is documented as the job's next execution time and is persisted by a Requeuer and read by status reporting, so leaving it stale made the stored job disagree with its actual retry delay. The backoff d is still passed to Requeue explicitly as the scheduling instruction; the field and the instruction now agree.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add an optional Requeuer interface a Queue may implement to express the
retry move ("this failed, put it back") distinctly from Enqueue ("add a
new job"). The worker retry path now prefers Requeue with the computed
backoff when the queue implements it, and falls back to Enqueue
otherwise. RedisQueue does not implement Requeuer, so its behavior is
unchanged.
This lets a queue whose waiting and in-flight jobs live in separate
structures restore a failed in-flight job to its position instead of
enqueuing a duplicate.