Customize retry policy based on the error?

[stargeneration-winston]

Is there a way to differentiate the retry policy on the error encountered? e.g. external service connection error should trigger immediate retry, while a input args error should wait a few hours for user to respond

[brandur]

Check out the docs on custom retry policies. There's an example of exactly what you're looking for here:

https://riverqueue.com/docs/job-retries#client-retry-policy

// NextRetry returns the next retry time based on the non-generic JobRow
// which includes an up-to-date Errors list.
func (policy *LinearRetryPolicy) NextRetry(job *rivertype.JobRow) time.Time {
    // The latest error is not yet included in the job's Errors list, so we
    // add 1 to the length to account for that.
    return time.Now().Add((len(job.Errors) + 1) * 5 * time.Second)
}

[dragondgold]

Is there a way to get the latest job error at that moment? Otherwise the retry delay is always based on an older error instead of the current one

[brandur]

Hm, I'm having a hard time recommending an easy way to do that. @bgentry Can you take a look and make sure I didn't miss anything?

The next retry interface currently looks like this:

type ClientRetryPolicy interface {
	NextRetry(job *rivertype.JobRow) time.Time
}

I'm kind of wondering whether it should have looked like this instead:

type ClientRetryPolicy interface {
	NextRetry(job *rivertype.JobRow, lastErr error) time.Time
}

Or for completeness, it should maybe even have taken a context too:

type ClientRetryPolicy interface {
	NextRetry(ctx context.Context, job *rivertype.JobRow, lastErr error) time.Time
}

Unfortunately there's no way we can change it as this point, so we'd have to add a second retry interface if we wanted to go that direction.

@dragondgold The only workaround I can suggest for now would be to use a HookWorkEnd hook to put an error in the job row's Metadata field with an expected key, and then read it out in NextRetry. The work end hook returns before NextRetry, so this is messy, but it should work if you're really motivated.

[bgentry]

Yeah, this is not really possible as of today and is probably a shortcoming of the ClientRetryPolicy interface. I think you're right that the only good way to solve this without breaking existing users is a new type and option, like ClientRetryPolicyAdvanced or Expanded or something where the NextRetry has more args.

[dragondgold]

That would be very helpful for us, for internal errors we usually have shorter retry policies, but, when dealing with external services we sometimes need much longer policies, having the option to customize it would be great, do you want me to create an issue?