What exactly is the Probability Integral Transform?

Your understanding looks basically correct to me.

As far as purpose, I've seen it used mostly to generate random variables from continuous distributions. For instance, if $X$ has a $U(0,1)$ distribution, then $F_X(x) = x$. Thus the requirement $F_X(x) = F_Y(y)$ in the probability integral transform reduces to $x = F_Y(y)$ or $y = F_Y^{-1}(x)$. Since $y$ is an observation from the probability distribution $Y$, this means that we can generate observations from the distribution $Y$ by generating $U(0,1)$ random variables (which most software programs can do easily) and applying the $F_Y^{-1}$ transformation.

For example, suppose you want to generate instances of an exponential$(\lambda)$ random variable. The cdf is $$F(y) = \int_0^y \lambda e^{-\lambda t} dt = 1 - e^{-\lambda y}.$$ Solving for $y$, we have $$F(y) - 1 = - e^{-\lambda y} \Rightarrow -\lambda y = \ln (1- F(y)) \Rightarrow y = F^{-1}(x) = -\ln(1-x)/\lambda.$$

Thus if $x$ is an observation from a $U(0,1)$ distribution, then $y = -\ln(1-x)/\lambda$ is an observation from an exponential$(\lambda)$ distribution. Moreover, $x$ having a $U(0,1)$ distribution is equivalent to $1-x$ having a $U(0,1)$ distribution, so we often express the transformation as $y = -\ln x/\lambda$.

As far as a general procedure for performing the transformation, what I've done here with the uniform and exponential distributions should give you a guide. Unfortunately, though, there aren't that many commonly-used distributions for which the cdf can be inverted analytically.