What are the benefits of a Deferred Execution in LINQ?

LINQ uses a Deferred Execution model which means that resulting sequence is not returned at the time the Linq operators are called, but instead these operators return an object which then yields elements of a sequence only when we enumerate this object.

While I understand how deferred queries work, I'm having some trouble understanding the benefits of deferred execution:

1) I've read that deferred query executing only when you actually need the results can be of great benefit. So what is this benefit?

2) Other advantage of deferred queries is that if you define a query once, then each time you enumerate the results, you will get different results if the data changes.

a) But as seen from the code below, we're able to achieve the same effect ( thus each time we enumerate the resource, we get different result if data changed ) even without using deferred queries:

List<string> sList = new List<string>( new[]{ "A","B" });

foreach (string item in sList)
    Console.WriteLine(item); // Q1 outputs AB

sList.Add("C");

foreach (string item in sList)
    Console.WriteLine(item); // Q2 outputs ABC

3) Are there any other benefits of deferred execution?

Solution 1:

The main benefit is that this allows filtering operations, the core of LINQ, to be much more efficient. (This is effectively your item #1).

For example, take a LINQ query like this:

 var results = collection.Select(item => item.Foo).Where(foo => foo < 3).ToList();

With deferred execution, the above iterates your collection one time, and each time an item is requested during the iteration, performs the map operation, filters, then uses the results to build the list.

If you were to make LINQ fully execute each time, each operation (Select / Where) would have to iterate through the entire sequence. This would make chained operations very inefficient.

Personally, I'd say your item #2 above is more of a side effect rather than a benefit - while it's, at times, beneficial, it also causes some confusion at times, so I would just consider this "something to understand" and not tout it as a benefit of LINQ.

In response to your edit:

In your particular example, in both cases Select would iterate collection and return an IEnumerable I1 of type item.Foo. Where() would then enumerate I1 and return IEnumerable<> I2 of type item.Foo. I2 would then be converted to List.

This is not true - deferred execution prevents this from occurring.

In my example, the return type is IEnumerable<T>, which means that it's a collection that can be enumerated, but, due to deferred execution, it isn't actually enumerated.

When you call ToList(), the entire collection is enumerated. The result ends up looking conceptually something more like (though, of course, different):

List<Foo> results = new List<Foo>();
foreach(var item in collection)
{
    // "Select" does a mapping
    var foo = item.Foo; 

    // "Where" filters
    if (!(foo < 3))
         continue;

    // "ToList" builds results
    results.Add(foo);
}

Deferred execution causes the sequence itself to only be enumerated (foreach) one time, when it's used (by ToList()). Without deferred execution, it would look more like (conceptually):

// Select
List<Foo> foos = new List<Foo>();
foreach(var item in collection)
{
    foos.Add(item.Foo);
}

// Where
List<Foo> foosFiltered = new List<Foo>();
foreach(var foo in foos)
{
    if (foo < 3)
        foosFiltered.Add(foo);
}    

List<Foo> results = new List<Foo>();
foreach(var item in foosFiltered)
{
    results.Add(item);
}

Solution 2:

Another benefit of deferred execution is that it allows you to work with infinite series. For instance:

public static IEnumerable<ulong> FibonacciNumbers()
{
    yield return 0;
    yield return 1;

    ulong previous = 0, current = 1;
    while (true)
    {
        ulong next = checked(previous + current);
        yield return next;
        previous = current;
        current = next;

    }
}

(Source: http://chrisfulstow.com/fibonacci-numbers-iterator-with-csharp-yield-statements/)

You can then do the following:

var firstTenOddFibNumbers = FibonacciNumbers().Where(n=>n%2 == 1).Take(10);
foreach (var num in firstTenOddFibNumbers)
{
    Console.WriteLine(num);
}

Prints:

1
1
3
5
13
21
55
89
233
377

Without deferred execution, you would get an OverflowException or if the operation wasn't checked it would run infinitely because it wraps around (and if you called ToList on it would cause an OutOfMemoryException eventually)

Solution 3:

An important benefit of deferred execution is that you receive up-to-date data. This may be a hit on performance (especially if you are dealing with absurdly large data sets) but equally the data might have changed by the time your original query returns a result. Deferred execution makes sure you will get the latest information from the database in scenarios where the database is updated rapidly.