Interchange finite and infinite sum

Solution 1:

Well, your second question is just answered by a particular case of the Fubini's Theorem. Let me explain it:

1 - You may always interchange a finite sum with an infinite one. This is just a consequence of linearity of the integral. When you have a countably infinite sum, you may take it as an integral, and one of the first properties of an integral seen on a measure theory course will be that, for $f_1,...,f_n$ 'fine' functions, then

$$ \sum _k \int f_k = \int \sum_k f_k $$.

2 - Your second question is a more interesting, and it is completely answered by Fubini's theorem. There is a much simpler version for sequences, though, and it states that:

If a doubly-indexed infinite sequence $\{x_{n,m}\}$ is absolutely summable in some way, i.e., if, for exemple,

$$ \sum _m \sum_n |x_{m,n}| < + \infty $$

Then you may change the order of summation and obtain the same result. This also holds for only positive sequences, and the result is the same.

As I already mentioned, this is part of a general Theorem called Fubini's Theorem, and I leave it up to you to check it, if interested.

Even more can be said in this case: in this case, you may even sum the sequence in a 'random' way, and you are still going to get the same result, where by "random way" we mean that, for every permutation $ \sigma : \mathbb{N} \times \mathbb{N} \to \mathbb{N} \times \mathbb{N} $, then the first sum is also equal

$$ \sum_{i,j} x_{\sigma(i,j)} $$

Solution 2:

João's answer only addresses the case where each of the sequences $\{f_{x,y}\}_{y=0}^\infty$ is integrable, that is, absolutely convergent. If these series are not absolutely convergent, you cannot (and should not) think of them as Lebesgue integrals: the limit depends on the order of summation! However, a finite sum may also be interchanged with an infinite one if the series are only conditionally convergent.

Proposition 1. Let $I$ be non-empty and finite, and suppose that for each $x\in I$ the series $\sum_{y=0}^\infty f_{x,y}$ converges. Then the series $\sum_{y=0}^\infty \sum_{x\in I} f_{x,y}$ also converges, and we have $$ \sum_{x\in I} \sum_{y=0}^\infty f_{x,y} = \sum_{y=0}^\infty \sum_{x\in I} f_{x,y}. $$ Proof. We have \begin{align} \sum_{x\in I} \sum_{y=0}^\infty f_{x,y} &= \sum_{x\in I} \left(\lim_{n\to\infty} \sum_{y=0}^n f_{x,y}\right)\tag*{(by definition)}\\[1ex] &= \lim_{n\to\infty} \left(\sum_{x\in I} \sum_{y=0}^n f_{x,y}\right)\tag*{(since addition is continuous)}\\[1ex] &= \lim_{n\to\infty} \left(\sum_{y=0}^n \sum_{x\in I} f_{x,y}\right)\tag*{(interchanging finite sums)}\\[1ex] &= \sum_{n=0}^\infty \sum_{x\in I} f_{x,y}.\tag*{$\Box$} \end{align}

The proof shows that the result holds in any abelian topological group (or even semigroup), the minimum structure needed to talk about infinite series. (For instance, this means that you can use it in topological vector spaces without having to resort to vector-valued integration.)

We cannot omit the assumption that for each $x\in I$ the series $\sum_{y=0}^\infty f_{x,y}$ converges, as illustrated by the following example.

Example 2. Let $I := \{-1,1\}$. For all $x\in I$ and $y\in \mathbb{N}$ we define $f_{x,y} := x\cdot y$. Now we have $$\sum_{y=0}^\infty \sum_{x\in I} f_{x,y} = \sum_{y=0}^\infty (-y + y) = \sum_{y=0}^\infty 0 = 0, $$ whereas $$\sum_{x\in I} \sum_{y=0}^\infty f_{x,y} = \left(\sum_{y=0}^\infty -y\right) + \left(\sum_{y=0}^\infty y\right) = -\infty + \infty, $$ which is undefined.

The point is that you still have to be a bit careful, and not blindly rearrange combinations of finite and infinite sums as they come up.

When $I$ is countably infinite, the best tool available is Fubini's theorem from measure theory, as already pointed out by others (João Ramos and A.Γ.). If the double series is not absolutely convergent, you will probably have to examine the outcome on a case-by-case basis.

Like in the case where $I$ was finite, it may happen that only one of the two double sums exists (c.f. example 2 above). However, even if both double series converge, their values might be unequal. Good examples of this have already been given on this website, for instance here and here.