Understanding Lipschitz Continuity

Lipschitz continuity doesn't say that if you take any old $x,y$ and plot $|f(x)-f(y)|\leq M|x-y|$. It says that $M$ is fixed and the inequality holds for all $x,y \in \mathbb R$ . This is much stronger of a condition. If all you needed to do was pick some $M$ for every choice of $x,y$, the condition would mean nothing.

If your function is differentiable, Lipschitz continuity just says that the function has bounded derivative. I think of this as a "wiggling" and "stretching" bound.

For the stretching, it says that a function $f:\mathbb R \to \mathbb R$ can't grow too fast. For example, if the Lipschitz constant were $1$ and $f(0)=0$, then it is trapped between the two lines $y=x-1,x+1$. This essentially means that it can't grow too quickly, or wiggle too much.

If the constant $M <1$, a function is said to be a contraction which has many nice properties as well.

--

You can not pick K sufficiently enough for a function to be Lipschitz continuous if they are not. That's the main point of that kind of continuity. If $f$ is not lipschitz continuous, and you say that $K = 10^6$, I can find an pair of points $x_1$ and $x_2$ such that $|f(x_1) - f(x_2)| \geq 10^6|x_1 - x_2|$.

Think about the mean value theorem and Lipschitz continuity.

Mean value theorem says if $f$ is continuous at [a,b] and differentiable at (a,b), then

$\exists c \in (a,b)$ such that $\displaystyle\frac{f(b) - f(a)}{b-a} = f'(c)$.

Lipschitz says that

$\exists K > 0, \forall a,b \in D_f,\mbox{ such that } \displaystyle\frac{|f(b) - f(a)|}{|b - a|} \leq K$.

Then if the derivative of $f$ as a function is bounded, then $f$ will be Lipschitz.

Consider the case

$f(x) = \sqrt{x}$ for $x \in [0,1]$, then $f$ is not Lipschitz, since $\displaystyle\sup_{x \in [0,1]}f'(x) = \displaystyle\lim_{x\to 0} f'(x) = +\infty$.

Also, as an additional note if a function $f$ defined on $S \subseteq \mathbb R$ is Lipschitz continuous then $f$ is uniformly continuous on $S$.

Definition: f is Lipschitz iff there exists $K>0$ such that for all $x,y\in dom(f)$ we have $$(1)\quad |f(x)-f(y)|<K |x-y|.$$ For $S\subset dom(f)$ we say $f$ is Lipschitz on $S$ to mean that $f$ restricted to the domain $S$ is a Lipschitz function. This is most commonly done when $S$ is an interval or a half-line.

The value of $K$ is not unique, for if (1) holds for all $x,y\in dom (f)$ then it will hold with $K$ replaced by $42K.$ What matters is whether $at$ $least$ $one$ $K>0$ exists such that (1) holds for all $x,y \in dom(f).$