How to wait for several Futures?

by

You could use a for-comprehension as follows instead:

val fut1 = Future{...}
val fut2 = Future{...}
val fut3 = Future{...}

val aggFut = for{
  f1Result <- fut1
  f2Result <- fut2
  f3Result <- fut3
} yield (f1Result, f2Result, f3Result)

In this example, futures 1, 2 and 3 are kicked off in parallel. Then, in the for comprehension, we wait until the results 1 and then 2 and then 3 are available. If either 1 or 2 fails, we will not wait for 3 anymore. If all 3 succeed, then the aggFut val will hold a tuple with 3 slots, corresponding to the results of the 3 futures.

Now if you need the behavior where you want to stop waiting if say fut2 fails first, things get a little trickier. In the above example, you would have to wait for fut1 to complete before realizing fut2 failed. To solve that, you could try something like this:

  val fut1 = Future{Thread.sleep(3000);1}
  val fut2 = Promise.failed(new RuntimeException("boo")).future
  val fut3 = Future{Thread.sleep(1000);3}

  def processFutures(futures:Map[Int,Future[Int]], values:List[Any], prom:Promise[List[Any]]):Future[List[Any]] = {
    val fut = if (futures.size == 1) futures.head._2
    else Future.firstCompletedOf(futures.values)

    fut onComplete{
      case Success(value) if (futures.size == 1)=> 
        prom.success(value :: values)

      case Success(value) =>
        processFutures(futures - value, value :: values, prom)

      case Failure(ex) => prom.failure(ex)
    }
    prom.future
  }

  val aggFut = processFutures(Map(1 -> fut1, 2 -> fut2, 3 -> fut3), List(), Promise[List[Any]]())
  aggFut onComplete{
    case value => println(value)
  }

Now this works correctly, but the issue comes from knowing which Future to remove from the Map when one has been successfully completed. As long as you have some way to properly correlate a result with the Future that spawned that result, then something like this works. It just recursively keeps removing completed Futures from the Map and then calling Future.firstCompletedOf on the remaining Futures until there are none left, collecting the results along the way. It’s not pretty, but if you really need the behavior you are talking about, then this, or something similar could work.

More Related Contents:

  1. Futures do not run before program termination
  2. Futures / Success race
  3. Scala Futures – built in timeout?
  4. Scala: List[Future] to Future[List] disregarding failed futures
  5. Scala’s “for comprehension” with futures
  6. Accessing value returned by scala futures
  7. How do I make a function involving futures tail recursive?
  8. Scala recover or recoverWith
  9. Can Scala actors process multiple messages simultaneously?
  10. Is the Scala 2.8 collections library a case of “the longest suicide note in history”? [closed]
  11. What is the purpose of type ascriptions in Scala?
  12. Scala: Why mapValues produces a view and is there any stable alternatives?
  13. Scala Map implementation keeping entries in insertion order?
  14. Modify collection inside a Spark RDD foreach
  15. Easy idiomatic way to define Ordering for a simple case class
  16. What is *so* wrong with case class inheritance?
  17. Scala: Why does Seq.contains take an Any argument, instead of an argument of the sequence type?
  18. Customize SparkContext using sparkConf.set(..) when using spark-shell
  19. How to apply a function to a tuple?
  20. What are the differences between asInstanceOf[T] and (o: T) in Scala?
  21. Spark: Transpose DataFrame Without Aggregating
  22. Advantages of F-bounded polymorphism over typeclass for return-current-type problem
  23. Scala Sets contain the same elements, but sameElements() returns false
  24. Can multi-projects from GIT be used as SBT dependencies?
  25. using variable length argument in scala
  26. Underscore in List.filter
  27. Is there an equivalent to SuppressWarnings in Scala?
  28. Why isn’t Validation a Monad?
  29. How is this case class match pattern working?
  30. Is it possible to have tuple assignment to variables in Scala? [duplicate]

Leave a Comment