A LITTLE GUIDE ON USING FUTURES FOR WEB DEVELOPERS
Why – Or Better Web Performance by Using Futures
Performance of web applications is important to users. A web site that is snappy will engage users much more. In frontend controllers you often need to access several backend services to display information. These backend services calls take some time to get the data from the backend servers. Often this is done one call after the other where the call times add up.
Suppose we have a web controller that accesses three backendservices for loading a user, getting new messages for the user and getting new job offers. The code would call the three backend services and then render the page:
case class User(email:String)
case class Message(m:String)
case class JobOffer(jo:String)
def userByEmail(email:String):User = User(email)
def newMessages:Seq[Message] = Seq(
Message("Hello World"),
Message("Bye!"))
def newOffers:Seq[JobOffer] = Seq(JobOffer("Janitor")
// On our frontend we would use these services to
// display a user home page.
val user = userByEmail("[email protected]") // takes 100ms
val messages = newMessages // takes 200ms
val offers = newOffers // takes 100ms
// render("Startpage", user, messages, newOffers)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
case class User(email:String)
case class Message(m:String)
case class JobOffer(jo:String)
def userByEmail(email:String):User = User(email)
def newMessages:Seq[Message] = Seq(
Message("Hello World"),
Message("Bye!"))
def newOffers:Seq[JobOffer] = Seq(JobOffer("Janitor")
// On our frontend we would use these services to
// display a user home page.
val user = userByEmail("[email protected]") // takes 100ms
val messages = newMessages // takes 200ms
val offers = newOffers // takes 100ms
// render("Startpage", user, messages, newOffers)
All three calls need to be executed to render the HTML for the page. With the timings of the three calls, the rendering will take at least 400ms. It would be nice to execute all three in parallel to speed up page rendering. To achieve this we can modify our service calls to return Futures.
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
def userByEmail(email:String):Future[User] = Future {
User(email)
}
def newMessages:Future[Seq[Message]] = Future {
Seq(Message("Hello World"), Message("Bye!"))
}
def newOffers:Future[Seq[JobOffer]] = Future {
Seq(JobOffer("Janitor"))
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
def userByEmail(email:String):Future[User] = Future {
User(email)
}
def newMessages:Future[Seq[Message]] = Future {
Seq(Message("Hello World"), Message("Bye!"))
}
def newOffers:Future[Seq[JobOffer]] = Future {
Seq(JobOffer("Janitor"))
}
Futures in Scala are executed in a different thread from a thread pool. Futures are boxes or wrappers around values that wrap the parallel execution of a calculation and the future value of that calculation. After the Future is finished the value is available. With Futures in place the code will take only 200ms to execute instead of 400ms as in the first version. This leads to faster response times from our website.
How to work with the value inside a Future? Suppose we want the email address of the User. We could wait until the execution of the service call is finished and get the value. This would diminish the value of our parallel execution. Better we can work with the value in the future!
val user = userByEmail("[email protected]")
// Future[String]
val email = user.map { _.email }
// non concurrent: user.email
1
2
3
4
5
6
7
val user = userByEmail("[email protected]")
// Future[String]
val email = user.map { _.email }
// non concurrent: user.email
This way the mapping function is called when the future is completed but our code still runs in parallel.
Getting the Value From a Future
When is a Future executed and the value rendered in a web framework? When we think of the Future as a box, at one point someone needs to open the box to get the value. We can open the box and get the value of all combined futures with Await. Await also takes a timeout value:
import scala.concurrent.duration._
// Future[String]
val emailF = userByEmail("[email protected]")
.map { _.email }
// String
val email = Await.result(emailF, 1.seconds)
// render(email)
1
2
3
4
5
6
7
8
9
10
import scala.concurrent.duration._
// Future[String]
val emailF = userByEmail("[email protected]")
.map { _.email }
// String
val email = Await.result(emailF, 1.seconds)
// render(email)
Await waits for the Future to return and in this case with a maximum execution time of 1 second. After this we can hand the email to our web templating for rendering the template. We want to open the box as late as possible as opening each box is blocking a thread which we want to prevent.
WebFramework that Supports Futures
So why open the box at all? Better yet to use a web framework that can handle asynchronicity natively like Play Framework.
// returns Future[String]
val emailF = userByEmail("[email protected]")
.map { _.email }
// returns Future[Result] to the request handler
emailF.map { email => Ok(dashboard.render(email) }
1
2
3
4
5
6
7
// returns Future[String]
val emailF = userByEmail("[email protected]")
.map { _.email }
// returns Future[Result] to the request handler
emailF.map { email => Ok(dashboard.render(email) }
Play can directly work with the Future and return the result to the browser as soon as the Future is completed, while our own code finishes and frees the request thread.
Combining Futures into new Futures
We do not want to use Await for every Future or hand every Future to the async web framework. We want to combine Futures into one Future and work with this combined Future. How do you combine the different futures from the backend calls into one result? We can use map as above and then nest service calls.
// Returns
// Future[Future[Future[(User, Seq[Message], Seq[JobOffer])]]]
userByEmail("[email protected]").map { user =>
newMessages.map { messages => {
newOffers.map { offers => (user,messages,offers) }
}
}}
1
2
3
4
5
6
7
8
// Returns
// Future[Future[Future[(User, Seq[Message], Seq[JobOffer])]]]
userByEmail("[email protected]").map { user =>
newMessages.map { messages => {
newOffers.map { offers => (user,messages,offers) }
}
}}
Ordinary map calls do not work, as we get a deeply nested Future. So for the outer map method calls we use flatMap to flatten a Future[Future[A]] result into a Future[A]
// Returns Future[(User, Seq[Message], Seq[JobOffer])]
userByEmail("[email protected]").flatMap { user =>
newMessages.flatMap { messages => {
newOffers.map { offers => (user,messages,offers) }
}
}}
1
2
3
4
5
6
7
// Returns Future[(User, Seq[Message], Seq[JobOffer])]
userByEmail("[email protected]").flatMap { user =>
newMessages.flatMap { messages => {
newOffers.map { offers => (user,messages,offers) }
}
}}
With many service calls this becomes unreadable though.
Serial Execution
Scala has a shortcut for nested flatMap calls in the form of for comprehensions. Scala for comprehensions – which are syntactic sugar for flatMap and filter – make the code more readable. The yield block is executed when all Futures have returned.
Example: Serial execution
// Returns Future[(User, Seq[Message], Seq[JobOffer])]
// ** NOT EXECUTED IN PARALLEL **
val result = for (
user <- userByEmail("[email protected]"); // User
messages <- newMessages; // Seq[Messages]
offers <- newOffers // Seq[JobOffer]
) yield {
// do something here
(user, messages, offers)
}
1
2
3
4
5
6
7
8
9
10
11
// Returns Future[(User, Seq[Message], Seq[JobOffer])]
// ** NOT EXECUTED IN PARALLEL **
val result = for (
user <- userByEmail("[email protected]"); // User
messages <- newMessages; // Seq[Messages]
offers <- newOffers // Seq[JobOffer]
) yield {
// do something here
(user, messages, offers)
}
The for comprehension is desugared into a chain of flatMaps (see above) and the methods are called after each other and therefor the futures are created after each other. This also means the futures are not executed in parallel.
Sometimes we want to execute Futures in serial one after the other but stop after the first failure. Michael Pollmeier has an example:
serialiseFutures(List(10, 20)) { i ?
Future {
Thread.sleep(i)
i * 2
}
}
1
2
3
4
5
6
7
serialiseFutures(List(10, 20)) { i ?
Future {
Thread.sleep(i)
i * 2
}
}
which can be implemented by:
def serialiseFutures[A, B](l: Iterable[A])(fn: A => Future[B])
(implicit ec: ExecutionContext): Future[List[B]] =
l.foldLeft(Future(List.empty[B])) {
(previousFuture, next) =>
for {
previousResults <- previousFuture
next <- fn(next)
} yield previousResults :+ next
}
1
2
3
4
5
6
7
8
9
10
def serialiseFutures[A, B](l: Iterable[A])(fn: A => Future[B])
(implicit ec: ExecutionContext): Future[List[B]] =
l.foldLeft(Future(List.empty[B])) {
(previousFuture, next) =>
for {
previousResults <- previousFuture
next <- fn(next)
} yield previousResults :+ next
}
Parallel Execution
For parallel execution of the futures we need to create them before the for comprehension.
Example: Parallel execution
// Futures are created here
val userF = userByEmail("[email protected]");
val messagesF = newMessages
val offersF = newOffers
// Returns Future[(User, Seq[Message], Seq[JobOffer])]
// ** Executed in parallel **
for (
user <- userF;
messages <- messagesF;
offers <- offersF
) yield {
(user, messages, offers)
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
// Futures are created here
val userF = userByEmail("[email protected]");
val messagesF = newMessages
val offersF = newOffers
// Returns Future[(User, Seq[Message], Seq[JobOffer])]
// ** Executed in parallel **
for (
user <- userF;
messages <- messagesF;
offers <- offersF
) yield {
(user, messages, offers)
}
Sitenote: If you’re more into FP you can use Applicatives to combine Future results into a tuple.
Working with dependent service calls
What happens if two service calls depend on each other? For example the messages call depends on the user call. With for comprehensions this is easy to solve as each line can depend on former lines.
// Working with dependend calls
def newMessages(user:User):Future[Seq[Message]] = Future {
Seq(Message("Hello " + user.email), Message("Bye!"))
}
def userByEmail(email:String):Future[User] = Future {
User(email)
}
def newOffers:Future[Seq[JobOffer]] = Future {
Seq(JobOffer("Janitor"))
}
// Future is created here, runs in parallel
val offersF = newOffers
// Returns Future[(User, Seq[Message], Seq[JobOffer])]
for (
user <- userByEmail("[email protected]");
messages <- newMessages(user); // userByEmail and messages
// run in serial
offers <- offersF // This runs in parallel
) yield {
(user, messages, offers)
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
// Working with dependend calls
def newMessages(user:User):Future[Seq[Message]] = Future {
Seq(Message("Hello " + user.email), Message("Bye!"))
}
def userByEmail(email:String):Future[User] = Future {
User(email)
}
def newOffers:Future[Seq[JobOffer]] = Future {
Seq(JobOffer("Janitor"))
}
// Future is created here, runs in parallel
val offersF = newOffers
// Returns Future[(User, Seq[Message], Seq[JobOffer])]
for (
user <- userByEmail("[email protected]");
messages <- newMessages(user); // userByEmail and messages
// run in serial
offers <- offersF // This runs in parallel
) yield {
(user, messages, offers)
}
Now userByEmail and newMessages run in serial, as the second depends on the first, while the call to newOffers runs in parallel, as we create the Future before the for comprehension and therefor it starts to run before.
The nice thing about Futures is how composable they are. For cleaner code we can create a method userWithMessages and reuse this in our final for comprehension. This way the code is easier to understand, parts are reusable and serial and parallel execution can easier be seen:
// Runs here
val offersF = newOffers
// Needs to run in serial as newMessages depends on userByEmail
def userWithMessages(email:String):Future[(User,Seq[Message])] = for (
user <- userByEmail(email);
messages <- newMessages(user)
) yield (user,messages)
// Returns Future[(User, Seq[Message], Seq[JobOffer])]
for (
(user,messages) <- userWithMessages("[email protected]");
offers <- offersF
) yield {
(user, messages, offers)
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
// Runs here
val offersF = newOffers
// Needs to run in serial as newMessages depends on userByEmail
def userWithMessages(email:String):Future[(User,Seq[Message])] = for (
user <- userByEmail(email);
messages <- newMessages(user)
) yield (user,messages)
// Returns Future[(User, Seq[Message], Seq[JobOffer])]
for (
(user,messages) <- userWithMessages("[email protected]");
offers <- offersF
) yield {
(user, messages, offers)
}
Using the method call userWithMessages in the for comprehension block works here, because it is the first and only method call. With several calls like above use
val userWithMessages = userWithMessages(...)
1
2
val userWithMessages = userWithMessages(...)
to execute in parallel before the for comprehension block.
Turning a Sequence of Futures into a Future of Sequence
If we want to get several users and our API does not support usersByEmail(Seq[String]) to get many users with one call we need to call the service for all users.
val users = Seq(
// Seq[Future]
users.map(userByEmail)
1
2
3
4
5
6
7
val users = Seq(
// Seq[Future]
users.map(userByEmail)
Here the usage of futures has major benefits as it dramatically speeds up execution. Suppose we have 10 calls with 100ms each then the serial version will take 10x100ms (1 second) whereas a parallel version from above will take 100ms.
How do we compose the result of Seq[Future[A]] with other futures? We need to turn this Sequence of Futures into a Future of Sequence. There is a method for this:
// Turns Seq[Future[User]] into Future[Seq[User]]
Future.sequence(users)
1
2
3
// Turns Seq[Future[User]] into Future[Seq[User]]
Future.sequence(users)
The future returns when all futures have returned.
Using Future.traverse
Another way is to directly use traverse
// returns Seq[Future[User]
users.map(userByEmail)
// returns Future[Seq[User]]
Future.traverse(users)(userByEmail) /
1
2
3
4
5
6
// returns Seq[Future[User]
users.map(userByEmail)
// returns Future[Seq[User]]
Future.traverse(users)(userByEmail) /
Composing Futures that contain Options
Often service calls or database calls return Option[A] for the possibility that no return value exists e.g. in the database. Suppose or User Server returns Future[Option[User]] and we want to work with the user. Sadly this does not work:
def userByEmail(email:String):Future[Option[User]] = Future {
Some(User(email))
}
// Does not work
for (
// Returns Option[User] not User
userO <- userByEmail("[email protected]");
user <- userO; // We want the user out of the
// option but this does not work
) yield {
(user)
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
def userByEmail(email:String):Future[Option[User]] = Future {
Some(User(email))
}
// Does not work
for (
// Returns Option[User] not User
userO <- userByEmail("[email protected]");
user <- userO; // We want the user out of the
// option but this does not work
) yield {
(user)
}
This does not work as ‘for’ is syntactic sugar for flatMap, meaning the results are flattened, e.g. List[List[A]] is flattened into List[A]. But how to flat Future[Option[A]]? So composing Futures that return Options is a little more difficult. It also doesn’t work because we wrap a container (Option) into another container (Future) but want to work on the inner value with map and flatmap.
Using a FutureOption Class
One solution for keeping the Option is to write a FutureOption that combines a Future and an Option into one class. On example can be found on the edofic Blog:
case class FutureO[+A](future: Future[Option[A]]) extends AnyVal {
def flatMap[B](f: A => FutureO[B])
(implicit ec: ExecutionContext): FutureO[B] = {
FutureO {
future.flatMap { optA =>
optA.map { a =>
f(a).future
} getOrElse Future.successful(None)
}
}
}
def map[B](f: A => B)
(implicit ec: ExecutionContext): FutureO[B] = {
FutureO(future.map(_ map f))
}
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
case class FutureO[+A](future: Future[Option[A]]) extends AnyVal {
def flatMap[B](f: A => FutureO[B])
(implicit ec: ExecutionContext): FutureO[B] = {
FutureO {
future.flatMap { optA =>
optA.map { a =>
f(a).future
} getOrElse Future.successful(None)
}
}
}
def map[B](f: A => B)
(implicit ec: ExecutionContext): FutureO[B] = {
FutureO(future.map(_ map f))
}
}
In our example from above it could be used like this:
val userFO = for (
user <- FutureO(userByEmail("[email protected]"));
// depend on user here
) yield {
// do something with user here
(user)
})
// Future[Option[User]]
val user = userFO.future
1
2
3
4
5
6
7
8
9
10
11
val userFO = for (
user <- FutureO(userByEmail("[email protected]"));
// depend on user here
) yield {
// do something with user here
(user)
})
// Future[Option[User]]
val user = userFO.future
Using OptionT
As this problem often arises in functional code working with containers it has been solved before with transformers. These allow stacking of containers and transform the effects of two containers (Option having the optional effect and Future the future effect) into a new container that combines these effects. Just the thing we need.
Luckily Scalaz has a generic transformer for Option called OptionT that can combine the effect of Option with another container. Combining also means that our flatMap and map methods work with the innermost value.
import scalaz.OptionT._
// With Scalaz
// Returns OptionT[Future,User]
val userOt = for (
user <- optionT(userByEmail("[email protected]"))
// depend on user here
) yield {
// do something with user here
(user)
}
// run returns the value wrapped in the containers
// before the transformation
// Future[Option[User]]
val user = userOt.run
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import scalaz.OptionT._
// With Scalaz
// Returns OptionT[Future,User]
val userOt = for (
user <- optionT(userByEmail("[email protected]"))
// depend on user here
) yield {
// do something with user here
(user)
}
// run returns the value wrapped in the containers
// before the transformation
// Future[Option[User]]
val user = userOt.run
This way we can compose calls that return Future[Option[A]].
Combining Option[A] with Seq[A]
Combining this with our messages service call that returns Seq[Message] gets a little more complex, but can be done.
import scalaz.OptionT._
// Returns OptionT[Future,(User, Seq[Message])]
val resultOt = for (
user <- optionT(userByEmail("[email protected]"));
messages <- newMessages.liftM
// Can also be written as
// messages <- optionT(newMessages.map(_.some))
) yield {
(user,messages)
}
// Future[Option[(User, Seq[Message])]
val result = resultOt.run
1
2
3
4
5
6
7
8
9
10
11
12
13
import scalaz.OptionT._
// Returns OptionT[Future,(User, Seq[Message])]
val resultOt = for (
user <- optionT(userByEmail("[email protected]"));
messages <- newMessages.liftM
// Can also be written as
// messages <- optionT(newMessages.map(_.some))
) yield {
(user,messages)
}
// Future[Option[(User, Seq[Message])]
val result = resultOt.run
The interesting line is messages <- newMessages.liftM where newMessages returns Future[Seq[Message]]. To get this working with Future[Option[A]] we can either transform this by hand or use liftM. LiftM automatically “lifts” the value into the right containers, OptionT[Future, Seq[Message]] in this case.
Combining Iterating over Lists with Futures
Sometimes you want to work with futures and iterate over Future[List[A]]. This can be achieved with nested for comprehensions:
def newMessages:Future[Seq[Message]] = Future {
Seq(Message("Hello World"), Message("Bye!"))
}
for (
user <- userByEmail("[email protected]");
messages <- newMessages
) yield for (
message <- messages
) yield {
// Do somethin here
// Returns Future[Seq[(String, Message)]]
(user.email, message)
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
def newMessages:Future[Seq[Message]] = Future {
Seq(Message("Hello World"), Message("Bye!"))
}
for (
user <- userByEmail("[email protected]");
messages <- newMessages
) yield for (
message <- messages
) yield {
// Do somethin here
// Returns Future[Seq[(String, Message)]]
(user.email, message)
}
Error Handling
For Futures and Futures with Options all the error handling strategies exist that I wrote about in “Some thoughts on Error Handling”.
But there are more. For combining Try with Future one can convert the Try to a Future. It makes more sense to flatten Future[Try[]] than flatten Future[Option[]]. Try and Future have similar semantics – Success and Failure – whereas Future and Option differ in their semantics.
def registerUser(u:User):Try[User] = ...
val result = for{
u <- userByEmail("[email protected]");
registered <- tryToFuture(registerUser(u))
} yield registered
result.map { /* Success Block */ } recover { /* Failure Block */ }
def tryToFuture[T](t:Try[T]):Future[T] = {
t match{
case Success(s) => Future.successful(s)
case Failure(ex) => Future.failed(ex)
}
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
def registerUser(u:User):Try[User] = ...
val result = for{
u <- userByEmail("[email protected]");
registered <- tryToFuture(registerUser(u))
} yield registered
result.map { /* Success Block */ } recover { /* Failure Block */ }
def tryToFuture[T](t:Try[T]):Future[T] = {
t match{
case Success(s) => Future.successful(s)
case Failure(ex) => Future.failed(ex)
}
}
Beside this Future in Scala has many ways to handle error conditions with fallbackTo, recover and recoverWith which to me are the preferred ways instead of failure callbacks which also exist.
Futures can make your frontend code faster and your customers happier. I hope this small guide helped you understanding Futures and how to handle them.
If you have ideas about handling Futures or general feedback, reply to @codemonkeyism
Notes
The article uses container or box for describing Future and Option. The term that is usually used is Monad. Often this term confuses people so I have used the simpler box and container instead.
The article assumes creating Futures creates parallel running code. This is not always the case and depends on the number of available cores and the thread pool or concurrency implementation underlying the Futures. For production deployment and performance you need to read about thread pools and how to tune them to your application and hardware. Usually an application has different thread pools e.g. for short running code, long running code or blocking (e.g. IO) code.
The article assumes that code in Future is not blocking. If you use blocking code in Futures, e.g. IO with blocking database drivers, this has impact on your performance.
Stephan the codemonkey writes some code since 35 years, was CTO in some companies and currently helps startups with their technical challenges