Switching order of supremum for doubly indexed sequence?

Suppose you have a doubly indexed sequence of reals, $(\alpha_{ij})$. Why is $$ \sup_i \;\sup_j\ \alpha_{ij}=\sup_j\;\sup_i\ \alpha_{ij}? $$ I know one approach is to note $\alpha_{mn}\leq\sup_j\;\sup_i\alpha_{ij}$ for any $m$ and $n$. Why is this exactly? I don't really know what $\sup_j\sup_i\alpha_{ij}$ means. Does it mean first fix some $j$ and find the supremum of $\alpha_{ij}$ as $j$ remains fixed as $i$ runs over $\mathbb{N}$? And then after that, find the supremum of $\sup_i\alpha_{ij}$ as $j$ runs over $\mathbb{N}$? It's not clear how I would find $\sup_i\alpha_{ij}$ for fixed arbitrary $j$ first. Thanks.

Solution 1:

Yes, it means exactly what you wrote.

The reason that

$$\sup_i\,\sup_j\ \alpha_{ij}=\sup_j\,\sup_i\ \alpha_{ij}$$

is that they're both equal to

$$\sup_{i,j}\,\alpha_{ij}\;.$$

For assume that

$$\sup_i\,\sup_j\ \alpha_{ij}\lt\sup_{i,j}\,\alpha_{ij}\;.$$

Then $\sup_i\,\sup_j\ \alpha_{ij}$ is not an upper bound for the $\alpha_{ij}$ (since there is no upper bound less than the supremum). Thus there is some $\alpha_{kl}$ greater than $\sup_i\,\sup_j\ \alpha_{ij}$. But this $\alpha_{kl}$ would make $\sup_j\alpha_{kj}$ be at least $\alpha_{kl}$, and thus $\sup_i\,\sup_j\alpha_{ij}$ would also be at least $\alpha_{kl}$, a contradiction.

Similarly, if

$$\sup_i\,\sup_j\ \alpha_{ij}\gt\sup_{i,j}\,\alpha_{ij}\;,$$

then some $\alpha_{kl}$ would have to be greater than $\sup_{i,j}a_{ij}$, which is impossible.

Note that this only works because both operations are suprema. If you take, say, the infimum with respect to $i$ and the supremum with respect to $j$, then it does matter in which order you perform those operations.

Solution 2:

To directly argue: $$ a_{ij} \leq \sup_j a_{ij}\implies \sup_{i}a_{ij} \leq \sup_i\sup_j a_{ij} \implies \sup_j\sup_{i}a_{ij} \leq \sup_i\sup_j a_{ij} $$