What is the inverse of a square matrix with orthogonal columns?
A matrix with orthogonal columns (which is easily seen to be invertible) may be written as $QD$ where $Q$ is orthogonal and $D$ is invertible diagonal. Its inverse is then $D^{-1}Q^T$.
If the columns of $A$ are orthogonal then $$A^TA=D^2$$ where the diagonal elements of $D$ are the lengths of the columns. Thus $$A^{-1}=D^{-2}A^T=(AD^{-2})^T$$ The recipe is therefore to divide the column vectors of $A$ by their lengths squared and then take the transpose.