What is the difference between max and sup?

I am studying KS (Kolmogorov-Sinai) entropy of order q and it can be defined as

$$ h_q = \sup_P \left(\lim_{m\to\infty}\left(\frac 1 m H_q(m,ε)\right)\right) $$

Why is it defined as supremum over all possible partitions P and not maximum?

When do people use supremum and when maximum?

Solution 1:

If the maximum exists, then the supremum and maximum are the same. However sometimes the maximum does not exist, and there is no maximal element. In this case it still makes sense to talk about a least upper bound.

The classic example is the set of all rationals whose square is less than or equal to $2$. That is the set $$A=\left\{ r\in\mathbb{Q}:\ r^{2}\leq2\right\}.$$

$A$ has no maximal element, however it does have a supremum and $\sup A=\sqrt{2}$.

An even simpler example is the set of all reals that are strictly less than $2$: $$B=\left\{ r\in\mathbb{R}:\ r<2\right\}.$$ This set has no maximum since for any $x\in B$ the element $\frac{x+2}{2}$ satisfies $x<\frac{x+2}{2}<2$. However it is not hard to see that $\sup B=2$.

Solution 2:

A maximum is the largest number WITHIN a set. A sup is a number that BOUNDS a set. A sup may or may not be part of the set itself (0 is not part of the set of negative numbers, but it is a sup because it is the least upper bound). If the sup IS part of the set, it is also the max.

Solution 3:

Consider for example the set $X = (0,1)$. Then $\sup X=1$ but $\max X$ does not exist.

Generally, for a set $X\subset {\mathbb R}$, we say $x=\max X$ if $\color{blue}{x\in X}$ and $$\forall y\in X, y\leq x.$$

We have $x=\sup X$ if $$\forall y\in X, y\leq x$$ and $$\forall\epsilon>0,\quad\exists y\in X, \quad\text{s.t.}\quad y>x-\epsilon.$$