Is it misleading to think of rank-2 tensors as matrices?

Solution 1:

It's not misleading as long as you change your notion of equivalence. When a matrix represents a linear transformation $V \to V$, the correct notion of equivalence is similarity: $M \simeq B^{-1} MB$ where $B$ is invertible. When a matrix represents a bilinear form $V \times V \to \mathbb{R}$, the correct notion of equivalence is congruence: $M \simeq B^TMB$ where $B$ is invertible. As long as you keep this distinction in mind, you're fine.

Solution 2:

You're absolutely right.

Maybe someone will find useful a couple of remarks telling the same story in coordinate-free way:

• What happens here is indeed identification of space with its dual: so a bilinear map $T\colon V\times V\to\mathbb{R}$ is rewritten as $V\times V^*\to\mathbb{R}$ — which is exactly the same thing as a linear operator $A\colon V\to V$;

• An identification of $V$ and $V^*$ is exactly the same thing as a scalar product on $V$, and using this scalar product one can write $T(v,w)=(v,Aw)$;

• So orthogonal change of basis preserves this identification — in terms of Qiaochu Yuan's answer one can see this from the fact that for orthogonal matrix $B^T=B^{-1}$ (moral of the story: if you have a canonical scalar product, there is no difference between $T$ and $A$ whatsoever; and if you don't have one — see Qiaochu Yuan's answer.)