Confusion regarding real inner-products and the spectral theorem for symmetric matrices

Solution 1:

Why should the vectors $v_i$ coincide with the eigenvectors $p_i$ of $A$?

More explicitly, consider an inner product such that $e_1$ and $e_1+e_2$ are an orthonormal basis. You can check that this corresponds to $x^tAy$ where $A=\left(\begin{array}{cc} 1 & -1\\ -1 & 2\end{array}\right)$. But neither $e_1$ nor $e_1+e_2$ are eigenvectors of this matrix.

Solution 2:

Ok, I've realized a couple of mistakes. I'll leave this here in case anyone else has similar confusion regarding this.

The first problem is that we need to distinguish between orthogonality according to the dot-product and orthogonality according to $g$. For the sake of clear notation, we'll write $e(\vec{v}, \vec{w}) := \vec{v}\cdot\vec{w}$, so $e$ will be the standard Euclidean inner-product. We should avoid confusing $g$-orthogonality with $e$-orthogonality.

The second problem is that just because $(\vec{v}_{1}, \ldots, \vec{v}_{n})$ is $g$-orthogonal, it does not mean this is the only $g$-orthogonal basis. In particular, it's not true that $\vec{v}_{1}, \ldots, \vec{v}_{n}$ and $\vec{p}_{1}, \ldots, \vec{p}_{n}$ are the same (and when I say "not the same" I also mean there's no rescaling that can make any one into any of the other).

Given a matrix $A$ representing $g$ in the standard basis, the spectral theorem guarantees $A$ has eigenvectors $\vec{p}_{1}, \ldots, \vec{p}_{n}$ with eigenvalues $\lambda_{1}, \ldots, \lambda_{n}$. These eigenvectors have a special meaning: these are the vectors that are both $g$-orthogonal and $e$-orthogonal. If we normalize them, then the matrix $P = (\vec{p}_{1} \;\cdots\; \vec{p}_{n})$ is an orthogonal matrix in the sense that it satisfies $P^{T}P = PP^{T} = I$, and at the same time $P$ allows you to transform to the basis in which the matrix of $g$ is diagonalized. This is okay because basis $(\vec{p}_{1}, \ldots, \vec{p}_{n})$ is simultaneously $g$-orthogonal and $e$-orthogonal.

Example. I'll prove an example with $n=2$. Suppose I construct a real inner-product in which the vectors $$ \vec{v}_{1} = \begin{pmatrix} 1 \\ 3 \end{pmatrix} \quad\text{ and }\quad \vec{v}_{2} = \begin{pmatrix} -1 \\ 1 \end{pmatrix} $$ are $g$-orthonormal but not $e$-orthonormal (or even $e$-orthogonal). By setting $\vec{v}\,^{T}_{i}g\,\vec{v}_{j} = \delta_{ij}$ and assuming $g$ is symmetric I can solve for entries of $g$ to get $$ g = \begin{pmatrix} \tfrac{5}{8} & -\tfrac{1}{8} \\ -\tfrac{1}{8} & \tfrac{1}{8} \end{pmatrix}. $$ Now to diagonalize $g$, I find eigenvectors $$ \vec{p}_{1} = \frac{1}{\sqrt{10 + 4\sqrt{5}}}\begin{pmatrix} -2-\sqrt{5} \\ 1 \end{pmatrix} \quad\text{ and }\quad \vec{p}_{2} = \frac{1}{\sqrt{10 - 4\sqrt{5}}}\begin{pmatrix} -2+\sqrt{5} \\ 1 \end{pmatrix} $$ with eigenvalues $\lambda_{1} = (3+\sqrt{5})/8$ and $\lambda_{2} = (3-\sqrt{5})/8$, respectively. By taking $P = (\vec{p}_{1} \;\; \vec{p}_{2})$, we indeed find $P^{T}P = I$ and $P^{T}gP = L$ where $L = \text{diag}(\lambda_{1}, \lambda_{2})$. The eigenvectors $\vec{p}_{1}, \vec{p}_{2}$ are not what we started out with, which was $\vec{v}_{1}, \vec{v}_{2}$, and it is only $\vec{p}_{1}, \vec{p}_{2}$ that are simultaneously $g$-orthogonal and $e$-orthogonal whereas $\vec{v}_{1}, \vec{v}_{2}$ are only $g$-orthogonal. Hence $P$ corresponds to a rotation+reflection in the 2D plane with respect to inner-product $e$, and it diagonalizes $g$ at the same time.

Last Note: I thought I'd point out one more thing that I think led to me be confused. Given a basis $\mathcal{B}$ you can construct a inner-product $g$ such that $\mathcal{B}$ is $g$-orthonormal (as I did in the example), and such an inner-product is unique. Thus, $\mathcal{B}$ uniquely determines $g$. However, given $g$ you can't uniquely specify a $g$-orthonormal basis (because there's more than one). Thus, $g$ does not uniquely determine $\mathcal{B}$ (contrary to what I thought).

Last Last Note: The fact that $\mathcal{B}$ uniquely determines $g$ does not contradict the possibility that $\mathcal{P} = (\vec{p}_{1}, \ldots, \vec{p}_{n})$ is both $e$-orthogonal and $g$-orthogonal, because the Last Note talks about ortho-normality rather than ortho-gonality. The basis $\mathcal{P}$ may be both $e$-orthogonal and $g$-orthogonal, but it is orthonormal in only one inner-product.