Is there a continuous function $f(x)$ such that the inverse function is $1/f(x)$?
Solution 1:
For this answer i will assume that the function is defined on an open interval (otherwise see the answer from @Elliot G ) and real.
Because $\frac1{f(x)}$ is the inverse function, we have $x = f^{-1}(f(x)) = \frac1{f(f(x))} $ and therefore $f(f(x)) = \frac 1 x$.
Note that because $f$ is bijective, it has to be either monotone falling or increasing. if $f$ is increasing, so is $f(f(x))$, but $\frac 1 x$ is not increasing, so it is not possible.
if $f$ is decreasing, the composition $f \circ f$ is still increasing (because the composition of decreasing functions is increasing). So there is no way to do it on an open interval.
Solution 2:
You might not like this, but consider $f:\{1\}\to\{1\}$ where $f(1)=1$. This satisfies the conditions since $f$ is technically continuous on the domain.
Solution 3:
Given
$$ f^{-1}(x) = \frac{1}{f(x)}. \tag 1 $$
Whence
$$ x = f^{-1}(f(x)) = \frac{1}{f(f(x))}. \tag 2 $$
Thus
$$ f(f(x)) = \frac{1}{x}.\tag 3 $$
Consider the function
$$ f(x) = \frac{1 + a x}{b + c x}. \tag 4 $$
Then
$$ f(f(x)) = \frac {\displaystyle 1 + a \frac{1 + a x}{b + c x}} {\displaystyle b + c \frac{1 + a x}{b + c x}} = \frac {[b + a] + [c + a^2] x} {[b^2 + c] + [bc + ac] x}. \tag 5 $$
So we get $f(f(x)) = x^{-1}$ for
$$ \left[ \begin{array}{rcl} c + a^2 &=& 0\\ b^2 + c &=& 0\\ \displaystyle \frac{b+a}{bc+ac} &=& 1 \end{array} \tag 6 \right. $$
Therefore $c=1$, $a=b=\pm \mathbf{i}$. Whence
$$ \bbox[16px,border:2px solid #800000] { f(x) = \frac{1 \pm \mathbf{i} x}{x \pm \mathbf{i}}.} \tag 7 $$
Consider the function
$$ f(x) = x^\alpha. \tag 8 $$
Then
$$ f(f(x)) = \big( x^\alpha \big)^\alpha = x^{\alpha^2}. \tag 9 $$
So we get $f(f(x)) = x^{-1}$ for $\alpha = \pm \mathbf{i}$. Whence
$$ \bbox[16px,border:2px solid #800000] { f(x) = x^{\displaystyle \pm \mathbf{i}}. } \tag {10} $$
I have no idea if there are other possible function.