What is the geometric interpretation of the transpose?
To answer your second question first: an orthogonal matrix $O$ satisfies $O^TO=I$, so $\det(O^TO)=(\det O)^2=1$, and hence $\det O = \pm 1$. The determinant of a matrix tells you by what factor the (signed) volume of a parallelipiped is multipled when you apply the matrix to its edges; therefore hitting a volume in $\mathbb{R}^n$ with an orthogonal matrix either leaves the volume unchanged (so it is a rotation) or multiplies it by $-1$ (so it is a reflection).
To answer your first question: the action of a matrix $A$ can be neatly expressed via its singular value decomposition, $A=U\Lambda V^T$, where $U$, $V$ are orthogonal matrices and $\Lambda$ is a matrix with non-negative values along the diagonal (nb. this makes sense even if $A$ is not square!) The values on the diagonal of $\Lambda$ are called the singular values of $A$, and if $A$ is square and symmetric they will be the absolute values of the eigenvalues.
The way to think about this is that the action of $A$ is first to rotate/reflect to a new basis, then scale along the directions of your new (intermediate) basis, before a final rotation/reflection.
With this in mind, notice that $A^T=V\Lambda^T U^T$, so the action of $A^T$ is to perform the inverse of the final rotation, then scale the new shape along the canonical unit directions, and then apply the inverse of the original rotation.
Furthermore, when $A$ is symmetric, $A=A^T\implies V\Lambda^T U^T = U\Lambda V^T \implies U = V $, therefore the action of a symmetric matrix can be regarded as a rotation to a new basis, then scaling in this new basis, and finally rotating back to the first basis.
yoyo has succinctly described my intuition for orthogonal transformations in the comments: from polarization you know that you can recover the inner product from the norm and vice versa, so knowing that a linear transformation preserves the inner product ($\langle x, y \rangle = \langle Ax, Ay \rangle$) is equivalent to knowing that it preserves the norm, hence the orthogonal transformations are precisely the linear isometries.
I'm a little puzzled by your comment about rotations and reflections because for me a rotation is, by definition, an orthogonal transformation of determinant $1$. (I say this not because I like to dogmatically stick to definitions over intuition but because this definition is elegant, succinct, and agrees with my intuition.) So what intuitive definition of a rotation are you working with here?
As for the transpose and symmetric matrices in general, my intuition here is not geometric. First, here is a comment which may or may not help you. If $A$ is, say, a stochastic matrix describing the transitions in some Markov chain, then $A^T$ is the matrix describing what happens if you run all of those transitions backwards. Note that this is not at all the same thing as inverting the matrix in general.
A slightly less naive comment is that the transpose is a special case of a structure called a dagger category, which is a category in which every morphism $f : A \to B$ has a dagger $f^{\dagger} : B \to A$ (here the adjoint). The example we're dealing with here is implicitly the dagger category of Hilbert spaces, which is relevant to quantum mechanics, but there's another dagger category relevant to a different part of physics: the $3$-cobordism category describes how space can change with time in relativity, and here the dagger corresponds to just flipping a cobordism upside-down. (Note the similarity to the Markov chain example.) Since relativity and quantum mechanics are both supposed to describe the time evolution of physical systems, it's natural to ask for ways to relate the two dagger categories I just described, and this is (roughly) part of topological quantum field theory.
The punchline is that for me, "adjoint" is intuitively "time reversal." (Unfortunately, what this has to do with self-adjoint operators as observables in quantum mechanics I'm not sure.)
Overview If a matrix A acting on vectors tells you how a vector is transformed, the matrix $A^T$ tells you how linear measurement of this vector are transformed.
If E is your vector space, the "space of linear measurements", or dual space is the space of all linear transformation $E \rightarrow \mathbb{R}$. Said differently, it is the space of all linear functions that take in a vector and output a number.
Example Let's work in $\mathbb{R}^2$.
- We have a generic vector $X = \begin{pmatrix} x_1 \\ x_2 \end{pmatrix}$
- A sample matrix $A = $$ \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix} $
- A sample linear measurement $h(X)=2x_1$, we note it $\begin{pmatrix} 2 \\ 0 \end{pmatrix}$
Maybe this measurement has some meaning physically (total mass or momentum...). We want to know how the transformation A affects h.
The familiar product $AX = \begin{pmatrix} x_1 + x_2 \\ x_2 \end{pmatrix}$ tells you how X is transformed. Now that we have seen that X has changed we might ask, what is the new measurement of X by h? It is obtained by computing $h(AX)$:
$h(Ax) = h(\begin{pmatrix} x_1 + x_2 \\ x_2 \end{pmatrix}) = 2x_1 + 2x_2$ which we note as $\begin{pmatrix} 2 \\ 2 \end{pmatrix}$
So when we transform by A, h goes from $\begin{pmatrix} 2 \\ 0 \end{pmatrix}$ to $\begin{pmatrix} 2 \\ 2 \end{pmatrix}$
It turns out that this transformation is precisely what the transpose is doing, as you can verify: $A^T\begin{pmatrix} 2 \\ 0 \end{pmatrix} = \begin{pmatrix} 2 \\ 2 \end{pmatrix}$
This is kind of hand-wavy, if you want more details look up dual space, which makes all of this precise.
We better interpret the geometric meaning of transpose from the view point of projective geometry. Because only in projective geometry, it is possible to interpret that of all square matrices.
It would be difficult for OP to understand and grasp the necessary premises or mathematical basis when interpreting the meaning in projective geometry of a square matrix's transpose only from a not too much lengthy answer posted here. So I would recommend you to read the article in this link first.
After read article in the above link, use the basic principles below:
Some basic or elementary geometric transformations are equivalent to Househodler's elementary matrices in homogeneous coordinates in projective geometry; that is what stereohomology is defined;
All square matrices can be represented by concatenated multiplication of elementary matrices;
a transpose of any elementary geometric transformation means exchanging the "center" and the "interface" of it per the definition of stereohomology.
then: the transpose of any square matrices have their own geometric meaning in projective geometry, though for complicated cases, the geometric interpretation of transpose might not be unique.