Dual and adjoint operator
More generally, if $A : X \to Y$ is a linear operator, then we have $I_X \circ A^\dagger=A^\ast\circ I_Y$, as in the following commutative diagram:
$$\begin{array}{c} Y & \xrightarrow{A^\dagger} & X \\ I_Y \downarrow ~~~~~&& ~~~~\downarrow I_X \\ Y^* & \xrightarrow{A^*} & X^* \end{array}$$
This is just a reformulation of the definition of $A^\dagger$: $$A^*(I_Y(y))(x)=I_Y(y)(A(x))=\langle A(x),y \rangle = \langle x,A^\dagger(y)\rangle=I_X(A^\dagger(y))(x).$$
And the diagram tells us that $I$ is a natural isomorphism from the functor $\mathrm{Hilb}^{\mathrm{op}} \to \mathrm{Hilb}$ mapping $X \mapsto X$ and $A \mapsto A^\dagger$ to the functor $\mathrm{Hilb}^{\mathrm{op}} \to \mathrm{Hilb}$ mapping $X \mapsto X^*$ and $A \mapsto A^*$.
Strictly speaking, if we work over $\mathbb{C}$, we have to take the complex-transpose $\overline{X}$ to define an isomorphism $I_X : \overline{X} \cong X^*$, and the first functor has to be modified accordingly.