scala - For Expressions Revisited

First we will introduce the For expressions, and we will show that for expression is actually is translated to the higher-order functions, they are : map, flatMap, and filter provider a powerful construction to deal with lists. 

for expression and high-order functions overview

high-order functions, such as map, flatMap and filter provides for translating and filtering on the list. and you can actually rewrite the high-order functions with for expressions. an exampe is as follow

case class Person(name: String, isMale : Boolean, children: Person *)

val lara = Person("Lava", false)
val bob = Person("Bob", false)
val julie = Person("Julie", false, lara, bob)
val persons = List(lara, bob, julie)

persons filter (p => !p.isMale)  flatMap (p => (p.children map (c => (p.name, c.name))))

actually withFilter call are more effecient. 

// the withFilter is more effcient
persons withFilter  (p => !p.isMale)  flatMap (p => (p.children map (c => (p.name, c.name))))

the above code is hard to understand, but  you can revisit the code with the for expression the for experssion will be translated to code similar to the above.  

for (p <- persons; if !p.isMale; c <- p.children)
  yield (p.name, c.name)

a for expressions that yield a result are translated by the compiler into the combination of invocation of the higher-order method map, flatMap, and withFilter. all for loops without yield-order are translated into a smaller set of higher-order functions: just withFilter and foreach.

For expression syntax and etc..

the general for expression's syntax is like this; 

for (generator; definitions; filters)

examples of that is as such . 

case class Person(name: String, isMale : Boolean, children: Person *)

val lara = Person("Lava", false)
val bob = Person("Bob", false)
val julie = Person("Julie", false, lara, bob)
val persons = List(lara, bob, julie)

with the aboce code definition, we can do with the for expresions

// SCala for expressoin in depth
// for (seq) yield expr
// for (generator; definitions; filters)

// e.g.
for (p <- persons; n = p.name; if (n startsWith "To")) yield n

to see it more clearer, 

// better formulate as follow
for {// you can use braces instead of parenthesis
  p <- persons    //; is optional
  n = p.name
  if (n startsWith "To")
} yield n

you can hve more than one generator, the code is as follow.

// a NOTE: the definitions, such as pat = expr, actually it introduce new variables, it binds pattern to the value of expr, so it has the same effect as a val definitions
// val x = expr

// multiple for generator
for (x <- List(1, 2); y <- List("one", "two")) yield (x, y)

N-queens problem. 

// file
//  N_queens_problems.scala
// descriptoin
//  an example of using for expression 

def queens(n :Int) : List[List[(Int,  Int)]] = {
  def placeQueens(k : Int) : List[List[(Int, Int)]] = 
    if (k == 0)
      List(List())
    else
      for { 
        queens <- placeQueens(k - 1)
        column <- 1 to n
        queen = (k, column)
        if isSafe(queen, queens)
      } yield queen :: queens
  
  placeQueens(n)
}

def isSafe(queen: (Int, Int), queens : List[(Int, Int)]) = 
  queens forall (q => !inCheck(queen, q))

def inCheck (q1 : (Int, Int), q2 : (Int, Int)) = 
  q1._1 == q2._1 || // same row
  q1._2 == q2._2 || // same column
(q1._1 - q2._2).abs == (q1._2 - q2._2).abs //  on diagonal

it is a problem solved with the for expression, basically the algorithm is that you can get the result of N by check each position on (N, x)  against every element that is valid on results N - 1;

Querying with for Expressions

suppose we have a case class as such 

case class Book(title : String, authors : String*)

and with the data type, we will use the following data to place our query. 

val books : List[Book] = 
  List(
      Book(
         "Structured and Interpretation of Computer Programs", 
         "Abelson, Harold", "Sussman, Geralds J."
      ),
      Book(
         "Principles of Compiler Design", 
         "Aho, Alfred", "Ullman, Jeffrey"
      ),
       Book(
         "Programming in Modula-2", 
         "Wirth, Niklaus"
      ),
       Book(
         "Elements of ML Programming", 
         "Ullman, Jeffrey"
      ),
       Book(
         "The Java Language Specification", 
         "Gosling, James", "Joy, Bill", "Steele, Guy", "Bracha, Gilad"
      )
  )
  

all books with author's last name is "Gosling"

// all books with author's last name is "Gosling"
  
for (b <- books ; a <- b.authors if a startsWith "Gosling") yield b.title

 books with string "Program" in their to their title

  // books with string "Program" in their to their title
for (b <- books if (b.title indexOf "Program") >= 0)
  yield b.title

names of the authors who has written at least two books.NOTE: authros may apear more than once.

for (b1 <- books; b2 <- books; if b1 != b2;
  a1 <- b1.authors ; a2 <- b2.authors if a1 == a2
) yield a1

to remove the duplicate. 

// remove updlicates
def removeDuplicates[A](xs: List[A]) : List[A] = {
  if (xs.isEmpty) xs
  else 
    xs.head :: removeDuplicates(
        xs.tail filter (x => x != xs.head)
    )
}

to use the removeDuplicate.

// remove the duplciates from the result 
//removeDuplicates(res7)
//
//res8: List[String] = List(Ullman, Jeffrey)

Translation of for Expressions

there are several way of translation. 

1. translating for expressions with one generator

for (x <- expr1) yield expr2

// where x is a variable , such an expression is translated to 

expr1.map (x => expr2)

2. translating for expression starting with a generator and a filter

for (x <- expr1 if expr2) yield expr3

it is translated first ot 

for (x <- expr1 withFilter(x => expr2)) yield expr3

and then it is further translated to

expr1 withFilter (x => expr2) map (x => expr3)

similarly, if there are multiple generators, like

for (x <- expr1 if expr2; seq) yield expr3

is translated to the following, then the translation continues

for (x <- expr1 withFilter expr2; seq) yield expr3

3. Translating for expressions starting with two generators

for (x <- expr1; y <- expr2; seq) yield expr3

this kind of definitions is translated to flatMap expressions

expr1.flatMap(x => for (y <- expr2; seq)) yield expr3

conclusions: with the three kinds of translation, it is sufficient  to translatoin all for expressions that contains just generators and filters, and where generators bind only simple variables. 

for instance.

// for instance
for (b1 <- books; b2 <- books; if b1 != b2;
  a1 <- b1.authors ; a2 <- b2.authors if a1 == a2
) yield a1

// is translated to 
books flatMap (b1 => 
  books withFilter (b2 => b1 != b2) flatMap (b2 => 
    b1.authros flatMap (a1 -> 
      b2.authors flatMap (a1 =>
        b2.authors withFilter (a2 => a1 == a2) map (a2 => a1)  
      )
    )
  )  
)

 A note on the before, the three translation does not translate whole "patterns" instead of a simple variable and it does not cover definitions

4. translating pattersn in generators

// e.g. if the pattern is a tuple
for ((x1, ..., xn) <- expr1) yield expr2
// is translated to 
expr1.map{
  case (x1, .... xn) => expr2
}

things becomes a bit more involved if the left hand side of the generator is an arbitrary pattern pat instead of a single variable or a tuple, in this case

for (pat <- expr1) yield expr2

is translated to 

exp1 withFilter {
  case pat => true
  case _ => false
} map {
  case pat => expr2
}

5. Translating definitions
the typical definition is like this:

for (x <- expr1; y = expr2; seq) yield expr3

that will be translated to the following 

for ((x, y) <- for (x <- expr1) yield (x, expr2); seq) yield expr3

so that you can see that the expr2 is evaluated each time there is a new x value being generated.  as a programmer you won't think this is a good idea, because for expression that does not bound to variable in the preceding generator.

so, instead of writting hte following

for ( x <- 1 to 1000; y = expensiveComputationNotInvolvingX) 
  yield x * y

it is usually write as such 

val y = expensiveComputationNotInvolvingX
for (x <- 1 to 1000) yield x * y 

6. Translating for loops
1-5 shows you how the for expression is translated with a yield expression contained, what if there is no "yield" statement.

actually the foreach will be used

for (x <- expr1) body 

translated to 

expr1 foreach (x => body)

a larger example 

for (x <- expr1; if expr2; y <- expr2) body 

is translated to 

expr1 withFiler (x => expr2) foreach (x => expr3) foreach (y => body)

another examples 

var sum = 0
for (xs <- xss; x <- xs) sum += x

var sum = 0
xss foreach (xs => 
  xs foreach (x =>
    sum += x
  )
)

Going the other way

the for expresion can be translated into applications of the high-order functions map, flatMap, and withFilter. in fact, you can equally well go the other way.

// file
//  for_expression_translation_the_other_way.scala
// description
//  you can translate map, flatMap and withFilter  into for expressions

object Demo {
  def map[A, B](xs : List[A], f : A => B) : List[B] = for (x <- xs) yield f(x)
  def flatMap[A, B](xs : List[A], f : A => List[B]) : List[B] = for (x <- xs; y <- f(x)) yield y
  def filter[A](xs : List[A], f : A => Boolean) : List[A] = for (x <- xs if p(x)) yield x
}

Generalizing For

you have seen the for expression on the list and array, actually for expresion can work on many more types, such as ranges, iterators, streams and all implementation of set. 

for the custom type to support full range of for expressions. you need to define map, flatMap, withFilter and foreach as methods of your data type. 

generally speaking . 

1. if you type just define map, it allows for expressions consisting of a single generator
2. if it define flatMap as well, it allows for expression consisting of several generators
3. if it defines foreach, it allows for loop (both with single and multiple generators)
4. if it defines withFilter, it allows for filter expression starting with an if in teh for expresion.

abstract class C[A] {
  def map[B](f : A => B) : C[B]
  def flatMap[B](f : A => B) : C[B]
  def withFilter(p : A => Boolean) : C[A]
  def foreach(b : A => Unit) : Unit
}