What is Scala's yield?

I think the accepted answer is great, but it seems many people have failed to grasp some fundamental points.

First, Scala's for comprehensions are equivalent to Haskell's do notation, and it is nothing more than a syntactic sugar for composition of multiple monadic operations. As this statement will most likely not help anyone who needs help, let's try again… :-)

Scala's for comprehensions is syntactic sugar for composition of multiple operations with map, flatMap and filter. Or foreach. Scala actually translates a for-expression into calls to those methods, so any class providing them, or a subset of them, can be used with for comprehensions.

First, let's talk about the translations. There are very simple rules:

1. This

   for(x <- c1; y <- c2; z <-c3) {...}
   

   is translated into

   c1.foreach(x => c2.foreach(y => c3.foreach(z => {...})))
   

2. This

   for(x <- c1; y <- c2; z <- c3) yield {...}
   

   is translated into

   c1.flatMap(x => c2.flatMap(y => c3.map(z => {...})))
   

3. This

   for(x <- c; if cond) yield {...}
   

   is translated on Scala 2.7 into

   c.filter(x => cond).map(x => {...})
   

   or, on Scala 2.8, into

   c.withFilter(x => cond).map(x => {...})
   

   with a fallback into the former if method withFilter is not available but filter is. Please see the section below for more information on this.

4. This

   for(x <- c; y = ...) yield {...}
   

   is translated into

   c.map(x => (x, ...)).map((x,y) => {...})
   

When you look at very simple for comprehensions, the map/foreach alternatives look, indeed, better. Once you start composing them, though, you can easily get lost in parenthesis and nesting levels. When that happens, for comprehensions are usually much clearer.

I'll show one simple example, and intentionally omit any explanation. You can decide which syntax was easier to understand.

l.flatMap(sl => sl.filter(el => el > 0).map(el => el.toString.length))

or

for {
  sl <- l
  el <- sl
  if el > 0
} yield el.toString.length

withFilter

Scala 2.8 introduced a method called withFilter, whose main difference is that, instead of returning a new, filtered, collection, it filters on-demand. The filter method has its behavior defined based on the strictness of the collection. To understand this better, let's take a look at some Scala 2.7 with List (strict) and Stream (non-strict):

scala> var found = false
found: Boolean = false

scala> List.range(1,10).filter(_ % 2 == 1 && !found).foreach(x => if (x == 5) found = true else println(x))
1
3
7
9

scala> found = false
found: Boolean = false

scala> Stream.range(1,10).filter(_ % 2 == 1 && !found).foreach(x => if (x == 5) found = true else println(x))
1
3

The difference happens because filter is immediately applied with List, returning a list of odds -- since found is false. Only then foreach is executed, but, by this time, changing found is meaningless, as filter has already executed.

In the case of Stream, the condition is not immediatelly applied. Instead, as each element is requested by foreach, filter tests the condition, which enables foreach to influence it through found. Just to make it clear, here is the equivalent for-comprehension code:

for (x <- List.range(1, 10); if x % 2 == 1 && !found) 
  if (x == 5) found = true else println(x)

for (x <- Stream.range(1, 10); if x % 2 == 1 && !found) 
  if (x == 5) found = true else println(x)

This caused many problems, because people expected the if to be considered on-demand, instead of being applied to the whole collection beforehand.

Scala 2.8 introduced withFilter, which is always non-strict, no matter the strictness of the collection. The following example shows List with both methods on Scala 2.8:

scala> var found = false
found: Boolean = false

scala> List.range(1,10).filter(_ % 2 == 1 && !found).foreach(x => if (x == 5) found = true else println(x))
1
3
7
9

scala> found = false
found: Boolean = false

scala> List.range(1,10).withFilter(_ % 2 == 1 && !found).foreach(x => if (x == 5) found = true else println(x))
1
3

This produces the result most people expect, without changing how filter behaves. As a side note, Range was changed from non-strict to strict between Scala 2.7 and Scala 2.8.

It is used in sequence comprehensions (like Python's list-comprehensions and generators, where you may use yield too).

It is applied in combination with for and writes a new element into the resulting sequence.

Simple example (from scala-lang)

/** Turn command line arguments to uppercase */
object Main {
  def main(args: Array[String]) {
    val res = for (a <- args) yield a.toUpperCase
    println("Arguments: " + res.toString)
  }
}

The corresponding expression in F# would be

[ for a in args -> a.toUpperCase ]

or

from a in args select a.toUpperCase 

in Linq.

Ruby's yield has a different effect.

Yes, as Earwicker said, it's pretty much the equivalent to LINQ's select and has very little to do with Ruby's and Python's yield. Basically, where in C# you would write

from ... select ??? 

in Scala you have instead

for ... yield ???

It's also important to understand that for-comprehensions don't just work with sequences, but with any type which defines certain methods, just like LINQ:

• If your type defines just map, it allows for-expressions consisting of a single generator.
• If it defines flatMap as well as map, it allows for-expressions consisting of several generators.
• If it defines foreach, it allows for-loops without yield (both with single and multiple generators).
• If it defines filter, it allows for-filter expressions starting with an if in the for expression.

Unless you get a better answer from a Scala user (which I'm not), here's my understanding.

It only appears as part of an expression beginning with for, which states how to generate a new list from an existing list.

Something like:

var doubled = for (n <- original) yield n * 2

So there's one output item for each input (although I believe there's a way of dropping duplicates).

This is quite different from the "imperative continuations" enabled by yield in other languages, where it provides a way to generate a list of any length, from some imperative code with almost any structure.

(If you're familiar with C#, it's closer to LINQ's select operator than it is to yield return).