Why does TaskCanceledException occur?

I have the following test code:

void Button_Click(object sender, RoutedEventArgs e)
{
    var source = new CancellationTokenSource();

    var tsk1 = new Task(() => Thread1(source.Token), source.Token);
    var tsk2 = new Task(() => Thread2(source.Token), source.Token);

    tsk1.Start();
    tsk2.Start();

    source.Cancel();

    try
    {
        Task.WaitAll(new[] {tsk1, tsk2});
    }
    catch (Exception ex)
    {
        // here exception is caught
    }
}

void Thread1(CancellationToken token)
{
    Thread.Sleep(2000);

    // If the following line is enabled, the result is the same.
    // token.ThrowIfCancellationRequested();
}

void Thread2(CancellationToken token)
{
    Thread.Sleep(3000);
}

In the thread methods I don't throw any exceptions, but I get TaskCanceledException in try-catch block of the outer code which starts the tasks. Why this happens and what is the purpose of token.ThrowIfCancellationRequested(); in this case. I believe the exception should only be thrown if I call token.ThrowIfCancellationRequested(); in the thread method.

I believe this is expected behavior because you're running in to a variation of a race condition.

From How to: Cancel a task and its children:

The calling thread does not forcibly end the task; it only signals that cancellation is requested. If the task is already running, it is up to the user delegate to notice the request and respond appropriately. If cancellation is requested before the task runs, then the user delegate is never executed and the task object transitions into the Canceled state.

and from Task Cancellation:

You can terminate the operation by [...] simply returning from the delegate. In many scenarios this is sufficient; however, a task instance that is "canceled" in this way transitions to the RanToCompletion state, not to the Canceled state.

My educated guess here is that while you are calling .Start() on your two tasks, chances are that one (or both of them) didn't actually start before you called .Cancel() on your CancellationTokenSource. I bet if you put in at least a three second wait between the start of the tasks and the cancellation, it won't throw the exception. Also, you can check the .Status property of both tasks. If I'm right, the .Status property should read TaskStatus.Canceled on at least one of them when the exception is thrown.

Remember, starting a new Task does not guarantee a new thread being created. It falls to the TPL to decide what gets a new thread and what is simply queued for execution.