Skip to content

Instantly share code, notes, and snippets.

@viktorklang
Last active July 23, 2023 23:48
Show Gist options
  • Save viktorklang/9414163 to your computer and use it in GitHub Desktop.
Save viktorklang/9414163 to your computer and use it in GitHub Desktop.
Asynchronous retry for Future in Scala
import scala.concurrent.duration._
import scala.concurrent.ExecutionContext
import scala.concurrent.Future
import akka.pattern.after
import akka.actor.Scheduler
/**
* Given an operation that produces a T, returns a Future containing the result of T, unless an exception is thrown,
* in which case the operation will be retried after _delay_ time, if there are more possible retries, which is configured through
* the _retries_ parameter. If the operation does not succeed and there is no retries left, the resulting Future will contain the last failure.
**/
def retry[T](op: => T, delay: FiniteDuration, retries: Int)(implicit ec: ExecutionContext, s: Scheduler): Future[T] =
Future(op) recoverWith { case _ if retries > 0 => after(delay, s)(retry(op, delay, retries - 1)) }
Copy link

ghost commented Mar 14, 2014

Terse, terse, terse !

@afijog
Copy link

afijog commented Nov 4, 2015

Also with a Future instead of a block of code

def f = Future (... )
trait Retrying {
  def retry[T](f: => Future[T], delay: FiniteDuration, retries: Int)(implicit ec: ExecutionContext, s: Scheduler): Future[T] = {
    f recoverWith { case _ if retries > 0 => after(delay, s)(retry(f, delay, retries - 1)) }
  }
}

@chadselph
Copy link

chadselph commented Sep 6, 2016

Instead of having the uniform delay, it's nice to have a retries with backoff. So:

def retry[T](f: => Future[T], delays: Seq[FiniteDuration])(implicit ec: ExecutionContext, s: Scheduler): Future[T] = {
  f recoverWith { case _ if delays.nonEmpty => after(delays.head, s)(retry(f, delays.tail) }
}

and you can call with

retry(Future(1), Seq(1.seconds, 10.seconds, 30.seconds))

@123avi
Copy link

123avi commented Sep 27, 2016

@chadselph how about combine both by adding a default delay value

 def retry[T](f: => Future[T], delay: Seq[FiniteDuration], retries: Int, defaultDelay: FiniteDuration )(implicit ec: ExecutionContext, s: Scheduler): Future[T] = {
    f recoverWith { case _ if retries > 0 => after(delay.headOption.getOrElse(defaultDelay), s)(retry(f, delay.tail, retries - 1 , defaultDelay)) }
  }

now you can call

val retries: List[FiniteDuration] = List(200 millis, 200 millis , 500 millis, 1 seconds, 2 seconds)
retry(future, retries , 10, 300 millis)

thanks guys , this is nice !

@chadselph
Copy link

chadselph commented Oct 3, 2016

@123avi if you wanted this behavior I would probably suggest leaving retry the same but adding some helpers for generating lists of FiniteDurations.

object RetryDelays {
  def withDefault(delays: List[FiniteDuration], retries: Int, default: FiniteDuration) = {
    if (delays.length > retries) delays take retries
    else delays ++ List.fill(retries - delays.length)(default)
  }

  def withJitter(delays: Seq[FiniteDuration], maxJitter: Double, minJitter: Double) =
    delays.map(_ * (minJitter + (maxJitter - minJitter) * Random.nextDouble))

  val fibonacci: Stream[FiniteDuration] = 0.seconds #:: 1.seconds #:: (fibonacci zip fibonacci.tail).map{ t => t._1 + t._2 }
}

and use it like

retry(someFuture(), RetryDelays.withJitter(RetryDelays.fibonacci, 0.8, 1.2))

@123avi
Copy link

123avi commented Oct 6, 2016

@chadselph thanks !

@graingert
Copy link

@chadselph @viktorklang can you license these snippets explicitly? MIT would be nice.

@viktorklang
Copy link
Author

@graingert Apologies, sadly Github doesn't notify when there are comments to Gists.
I don't think my snippet is complex enough to give a license at all.

@nikolovivan
Copy link

nikolovivan commented Mar 30, 2018

I might be a bit late to this party, but I felt like it will be a useful contribution.

First of all, a very useful snippet and a nice bunch of follow-ups. However, if it is defined the following way:

def retry[T](f: => Future[T], delays: Seq[FiniteDuration])(implicit ec: ExecutionContext, s: Scheduler): Future[T] = {
  f recoverWith { case _ if delays.nonEmpty => after(delays.head, s)(retry(f, delays.tail) }
}

The f parameter, which is passed by-name, will get evaluated when you call f recoverWith. After this point, if the future indeed fails, you will just end up passing the same failed future as many times as you have delays. So it won't really retry - it will just waste some time.

The following is a potential work-around:

def retry[T](f: () => Future[T], delays: Seq[FiniteDuration])(implicit ec: ExecutionContext, s: Scheduler): Future[T] = {
  f() recoverWith { case _ if delays.nonEmpty => after(delays.head, s)(retry(f, delays.tail) 
}

Of course, you'll have to modify the way you call the method accordingly.

@tadej-mali
Copy link

Looking at example at https://docs.scala-lang.org/tour/by-name-parameters.html - isn't the by-name parameter evaluated each time when accessed? The condition: => Boolean would ne er evaluate to false when it was true initally. Or am I missing something?

@hrieke
Copy link

hrieke commented Jan 17, 2019

Allow me to agree with Graingert that a license would be great to have.
May I suggest the Beerware license?
"BeerWare: If you have the time and money, send me a bottle of your favourite beer. If not, just send me a mail or something. Copy and use as you wish; just leave the author's name where you find it."

@jeffrey-aguilera
Copy link

@nikolovivan - As @tadej-mali points out, the call-by-name parameter is evaluated each time it is referenced; it is not a thunk.

@xxyyz
Copy link

xxyyz commented Oct 27, 2020

Note that the operation parameter (originally op, then f in the comments) should not have any side effects (e.g. logging), since they might be done several times. In some special situations this might not be a problem though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment