What is the support function $h_K(x)\equiv \sup_{z \in K} \langle z, x \rangle$ of a set $K$?

If you take $K$ to be convex, the support function is, in some sense, a tool for a dual representation of the set as the intersection of half-spaces.

Let's assume that we're in $\mathbb R^n$ for simplicity. A hyperplane can be characterized by a direction $\boldsymbol x\in\mathbb R^n$ and a scalar $b\in\mathbb R$, let's write $H=(\boldsymbol x;b)$ one such hyperplane, the set of points $\boldsymbol z\in\mathbb R^n$ on the hyperplane $H$ are then given by $$ \langle \boldsymbol z,\boldsymbol x\rangle \quad = \quad b. $$ The set of points $\boldsymbol z$ lying on one side of the hyperplane $H$ can thus always be written as $\langle \boldsymbol z, \boldsymbol x\rangle\le b$ (modulo a change of sign). So considering $$\sup_{\boldsymbol z\in K} \langle \boldsymbol z,\boldsymbol x\rangle $$ amounts to finding the $b(\boldsymbol x)$ for the direction $\boldsymbol x$ such that set $K$ lies on one side of the hyperplane $(\boldsymbol x,b(\boldsymbol x))$ or equivalently, such that all $z\in K$ verify $\langle \boldsymbol z, \boldsymbol x\rangle \le b(\boldsymbol x)$.

Then $K$ can be understood as the intersection of all the half-spaces thus defined.

It can maybe be useful to look at a basic example: consider the region $K=[0,1]\times [0,1]$. Then let's consider the $x$-direction with the vector $\boldsymbol v=(1,0)^t$, we get $$ h_K(\boldsymbol v) = \max_{\boldsymbol w\in K} \langle\boldsymbol w,\boldsymbol v \rangle = \max_{w_1\in[0,1]} w_1 = 1 $$ and the hyperplane $(\boldsymbol v,1)$ (i.e, the vertical line $x=1$) is indeed such that $K$ lies on strictly one side of it. Doing the same thing for the direction $-\boldsymbol v$, and the perpendicular directions will bring us the for sides of the region. This is a bit of a trivial example but hopefully it can help somewhat for the intuition.