Show that the maximum of a set of convex functions is again convex

Well, the domain of $f$ must be the intersection of the domains of the constituent functions, otherwise the $\max$ cannot be defined.

More generally, suppose $f_\alpha :C \to \mathbb{R}$ is a family of convex functions, with $\alpha \in A$, some index set, and let $f(x) = \sup_{\alpha \in A} f_{\alpha}(x)$.

Then, for any $\alpha \in A$, $\lambda \in [0,1]$, \begin{eqnarray} f_\alpha(\lambda x + (1-\lambda) y) &\le& \lambda f_\alpha(x) + (1-\lambda) f_\alpha(y) \\ & \le & \sup_{\alpha' \in A} (\lambda f_{\alpha'}(x) + (1-\lambda) f_{\alpha'}(y)) \\ & \le & \sup_{\alpha' \in A} \lambda f_{\alpha'}(x) + (1-\lambda) \sup_{\alpha' \in A}f_{\alpha'}(y) \\ &=& \lambda f(x) + (1-\lambda) f(y) \end{eqnarray}

How taking the supremum of the left hand side gives the desired result: $$f(\lambda x + (1-\lambda) y) \le \lambda f(x) + (1-\lambda) f(y)$$

Alternatively, you could note that $\operatorname{epi} f = \cap_{\alpha \in A} \operatorname{epi} f_\alpha$, and that the intersection of convex sets is convex.


Since it is given that all $f_i(x)$ are convex, we can write the following inequality using Jensen's Inequality.

$$ f_i(\theta x_1 + (1-\theta)x_2) \le \theta f_i(x_1) + (1-\theta)f_i(x_2); \qquad \theta \in [0,1] $$

We take maximum of the both sides.

$$ \underset{i}{\text{max}} \left\{ f_i(\theta x_1 + (1-\theta)x_2) \right\} \le \underset{i}{\text{max}} \left\{ \theta f_i(x_1) + (1-\theta)f_i(x_2) \right\} \qquad \ldots \text{(I)} $$

It is clear that

$$ \underset{i}{\text{max}} \left\{\theta f_i(x_1) + (1-\theta)f_i(x_2) \right\} \le \underset{i}{\text{max}} \left\{\theta f_i(x_1) \right\} + \underset{i}{\text{max}} \left\{(1-\theta)f_i(x_2) \right\}. \qquad \ldots \text{(II)} $$

Substitiute $\text{(II)}$ into $\text{(I)}$:

$$ \underset{i}{\text{max}} \left\{ f_i(\theta x_1 + (1-\theta)x_2) \right\} \le \underset{i}{\text{max}} \left\{\theta f_i(x_1) \right\} + \underset{i}{\text{max}} \left\{(1-\theta)f_i(x_2) \right\} \qquad \ldots \text{(III)} $$

From the definition of $f(x)$ in the OP we can write $\text{(III)}$ as

$$ f(\theta x_1 + (1-\theta)x_2) \le \theta f(x_1) + (1-\theta)f(x_2). \qquad \ldots \text{(IV)} $$

Being able to write $f(x)$ in the form $\text{(IV)}$ implies that it is convex.