Möbius transformation with infinity in both the $w$-plane and $z$-plane.

The formula $$ \frac{(f(z)-w_1)(w_2-w_3)}{(f(z)-w_3)(w_2-w_1)}=\frac{(z-z_1)(z_2-z_3)}{(z-z_3)(z_2-z_1)} $$ can only be applied if all $z_j$ and $w_j$ are complex numbers, i.e. not $\infty$. The general formula is $$ (f(z), w_1, w_2, w_3) = (z, z_1, z_2, z_3) $$ where $(z, z_1, z_2, z_3)$ is the cross-ratio of the numbers $z, z_1, z_2, z_3$, that is the value of $z$ under the Möbius transformation which maps $z_1, z_2, z_3$ to $0, 1, \infty$, respectively.

If $z, z_1, z_2, z_3$ are all (distinct) complex numbers then their cross-ratio is $$ (z, z_1, z_2, z_3) = \frac{(z-z_1)(z_2-z_3)}{(z-z_3)(z_2-z_1)} \, . $$ If one of the $z_j$ is equal to $\infty$ then the corresponding formula can be obtained by a limiting process. As an example, for $z_1 \to \infty$ we get $$ (z, \infty, z_2, z_3) = \frac{z_2-z_3}{z-z_3} \, . $$ Similarly, $$ (z, z_1, \infty, z_3) = \frac{z-z_1}{z-z_3} $$ and $$ (z, z_1, z_2, \infty) = \frac{z-z_1}{z_2-z_1} \, . $$

In your case we need the solution for $$ (f(z), \infty, 0, i) = (z, 0, \infty, 5) \, . $$ Using the above formulae this translates to $$ \frac{0-i}{f(z)-i} = \frac{z-0}{z-5} \iff \boxed{f(z) = \frac{5i}{z}} \, . $$

Of course one could have obtained that result with an educated guess. A Möbius transformation exchanges the points $0$ and $\infty$ if it is of the form $T(z) = a/z$ for some non-zero constant $a$, and the condition $f(5)=i$ gives $a=5i$.