Why cannot IEnumerable<struct> be cast as IEnumerable<object>?

Solution 1:

Why is the last line not allowed?

Because double is a value type and object is a reference type; covariance only works when both types are reference types.

Is this because double is a value type that doesn't derive from object, hence the covariance doesn't work?

No. Double does derive from object. All value types derive from object.

Now the question you should have asked:

Why does covariance not work to convert IEnumerable<double> to IEnumerable<object>?

Because who does the boxing? A conversion from double to object must box the double. Suppose you have a call to IEnumerator<object>.Current that is "really" a call to an implementation of IEnumerator<double>.Current. The caller expects an object to be returned. The callee returns a double. Where is the code that does the boxing instruction that turns the double returned by IEnumerator<double>.Current into a boxed double?

It is nowhere, that's where, and that's why this conversion is illegal. The call to Current is going to put an eight-byte double on the evaluation stack, and the consumer is going to expect a four-byte reference to a boxed double on the evaluation stack, and so the consumer is going to crash and die horribly with an misaligned stack and a reference to invalid memory.

If you want the code that boxes to execute then it has to be written at some point, and you're the person who gets to write it. The easiest way is to use the Cast<T> extension method:

IEnumerable<object> objects2 = doubleenumerable.Cast<object>();

Now you call a helper method that contains the boxing instruction that converts the double from an eight-byte double to a reference.

UPDATE: A commenter notes that I have begged the question -- that is, I have answered a question by presupposing the existence of a mechanism which solves a problem every bit as hard as a solution to the original question requires. How does the implementation of Cast<T> manage to solve the problem of knowing whether to box or not?

It works like this sketch. Note that the parameter types are not generic:

public static IEnumerable<T> Cast<T>(this IEnumerable sequence) 
{
    if (sequence == null) throw ...
    if (sequence is IEnumerable<T>) 
        return sequence as IEnumerable<T>;
    return ReallyCast<T>(sequence);
}

private static IEnumerable<T> ReallyCast<T>(IEnumerable sequence)
{
    foreach(object item in sequence)
        yield return (T)item;
}

The responsibility for determining whether the cast from object to T is an unboxing conversion or a reference conversion is deferred to the runtime. The jitter knows whether T is a reference type or a value type. 99% of the time it will of course be a reference type.

Solution 2:

To understand what is allowed and not allowed, and why things behave as they do, it is helpful to understand what's going on under the hood. For every value type, there exists a corresponding type of class object, which--like all objects--will inherit from System.Object. Each class object includes with its data a 32-bit word (x86) or 64-bit longword (x64) which identifies its type. Value-type storage locations, however, do not hold such class objects or references to them, nor do they have a word of type data stored with them. Instead, each primitive-value-type location simply holds the bits necessary to represent a value, and each struct-value-type storage location simply holds the contents of all the public and private fields of that type.

When one copies a variable of type Double to one of type Object, one creates a new instance of the class-object type associated with Double and copies all the bytes from the original to that new class object. Although the boxed-Double class type has the same name as the Double value type, this does not lead to ambiguity because they can generally not be used in the same contexts. Storage locations of value types hold raw bits or combinations of fields, without stored type information; copying one such storage location to another copies all bytes, and consequently copies all public and private fields. By contrast, heap objects of types derived from value types are heap objects, and behave like heap objects. Although C# regards the contents of value-type storage locations as though they are derivatives of Object, under the hood the contents of such storage locations are simply collections of bytes, effectively outside the type system. Since they can only be accessed by code which knows what the bytes represent, there is no need to store such information with the storage location itself. Although the necessity for boxing when calling GetType on a struct is often described in terms of GetType being a non-shadowed, non-virtual function, the real necessity stems from the fact that the contents of a value-type storage location (as distinct from the location itself) don't have type information.