what is the geometrical interpretation to positive definite matrix

What is the geometrical interpretation of positive definite matrix ? (not necessarily symmetric)

if $A$ is positive definite, what does it do to a vector $x$ (i.e. $Ax$)?

Solution 1:

Matrix being positive definite is like a (real) number being positive.

Say you have a positive (real) number $a$. It has a property such that you can multiply any real number $b \in R$ by $a$ and the result of multiplication $a \cdot b$ will preserve $b$'s sign. In another words, $a$ scales $b$ but does not reflect it (or change its sign).

Now let's apply this concept to matrices.

First, recall that for any vectors $a, b$: $$ a^T b = |a||b|cos\theta $$ where $\theta$ is the angle between $a$ and $b$. The value of $a^T b$ is positive iff $cos\theta > 0$ (i.e. $-\frac{\pi}{2} \leq \theta \leq \frac{\pi}{2}$)

Second, recall the definition of a positive-definite matrix $A \in R^{n \times n}$: $$ x^T A x > 0 $$ for all $x \in R^n \backslash \{ 0 \}$.

The meaning of $x^T A x$ is: Take any vector $x$ and transform it into $Ax$ and then take a dot product of $x$ with its "transformed self". Since the result is always positive this must mean that the angle between $x$ and $Ax$ must have been such that $-\frac{\pi}{2} \leq \theta \leq \frac{\pi}{2}$. This means that $A$ does not "reflect" any vector, thus behaving like a real (positive) number $a$.

Solution 2:

If the matrix is not symmetric, there is no notion of positivity other than having eigenvalues all positive. For symmetric matrices, you get a partial ordering such as $A\succ B$ which means $A-B$ is positive definite.

Moreover, every symmetric positive definite matrix defines an ellipsoid. The principal axes are given by the eigenvectors of $A$ define and the square root of the eigenvalues are the radii of the corresponding axes. The usual convention is to use the inverse of the matrix to define the ellipsoid. This is to define the unit ball as the image of the mapping $x^TA^{-1}x$.