Quantile function properties

I am confused by "Inverse distribution function (quantile function)" section of the wikipedia page on CDFs . It says that $$F^{-1}(F(x)) \leq x\text{ and }F(F^{-1}(y)) \geq y$$

However, I thought that the very definition of the inverse function was that $$F^{-1}(F(x)) = x\text{ and } F(F^{-1}(y))=y$$ Now I am starting to wonder if I've understood everything incorrectly. Can anyone explain to me why there is an inequality instead of just an equals sign?

As the Wikipedia page states, "Unfortunately, the distribution does not, in general, have an inverse. One may define, for $y \in [0,1]$, the generalized inverse distribution function $F^{-1}(y)= \inf \{x: F(x) \geq y\}$."

Since not all distribution functions $F$ are strictly increasing, not all $F$ have (proper) inverses. When $F$ does has a proper inverse, then the two definitions coincide. When $F$ does not have an inverse (example?) then the definition given still retains many useful properties used in probabilistic arguments.

$F^{-1}$ is simply the notation used in this context to be suggestive of an inverse. In some books, the notation $F^{\leftarrow}$ is used to distinguish it or make clear that a (proper) inverse may not exist and the generalized version is being used. However, this alternate notation is not all that common.

Let's briefly prove some of the facts on the Wikipedia webpage.

Fact 1: $F^{-1}(y)$ is nondecreasing.

Proof: Let $y_1 < y_2$. Then $F(x) \geq y_2$ implies $F(x) \geq y_1$, and so $\{x: F(x) \geq y_1\} \supset \{x:F(x) \geq y_2\}$. Hence, clearly, $\inf \{x: F(x) \geq y_1\} \leq \inf \{x : F(x) \geq y_2\}$.

Fact 2: $F^{-1}(F(x)) \leq x$.

Proof: $F^{-1}(F(x)) = \inf\{z: F(z) \geq F(x)\}$ and $x \in \{z: F(z) \geq F(x)\}$, so the result follows.

Fact 3: $F( F^{-1}(y) ) \geq y$.

Proof: Let $x_n \in \{x : F(x) \geq y\}$ such that $x_n \to x_0$. Then $\liminf_n F(x_n) \geq y$, but since $F$ is monotone nondecreasing and continuous from the right, $\liminf_n F(x_n) \leq F(x_0)$. Hence $F(x_0) \geq y$, that is, $x_0 \in \{x: F(x) \geq y\}$. So, $\{x: F(x) \geq y\}$ is closed and hence contains its infimum. Thus, $F(F^{-1}(y)) \geq y$.

Fact 4: $F^{-1}(y) \leq x$ iff $y \leq F(x)$.

Proof: $F^{-1}(y) \leq x$ implies $x \in \{z: F(z) \geq y\}$ and so $y \leq F(x)$. On the other hand, if $y \leq F(x)$ then $x \in \{z : F(z) \geq y\}$ and so $F^{-1}(y) \leq x$ since $F^{-1}(y)$ is the infimum.

I have uses this before, but for a visualisation try this with the CDF and its reflection

and note the vertical green line segments which strictly are not parts of the functions but appear as horizontal line segments in the reflections.

The following example would illustrate the point:

$y < F(x)$ only implies $F^{-1}(y) \le x$. Consider an example of discrete random variate with $\mathbb{P}(X=0) = \frac{1}{3}$, $\mathbb{P}(X=1) = \frac{1}{2}$ and $\mathbb{P}(X=2) = \frac{1}{6}$.

Consider $y=\frac{1}{2}$ and $x=1$, then $F(1) = \mathbb{P}(X \le 1) = \frac{1}{3} + \frac{1}{2} = \frac{5}{6}$, so $y < F(x)$, yet

$$ F^{-1}(y) = \operatorname{argmin}_{x \in \{0,1,2\}} \left( F(x) \ge y \right) = 1 $$ Now $F(F^{-1}(y)) = \frac{5}{6} > y=\frac{1}{2}$. Likewise, take $x=\frac{3}{2}$. $F(\frac{3}{2}) = \mathbb{P}(X \le \frac{3}{2}) = \frac{5}{6}$, and $F^{-1}(F(\frac{3}{2}))=1 < x=\frac{3}{2}$.