If $A$ and $B$ are positive definite, then is $B^{-1} - A^{-1}$ positive semidefinite?

I've found this while googling some properties of positive semidefinite matrices. (Unfortunately, I cannot remember where I've discovered it.) If this is true, it'll greatly save my time in my work. Is it true? How can you prove it?

Let's say I have two real symmetric and positive definite matrices $\mathbf A$ and $\mathbf B$ of the same size. Also, assume that $\mathbf A - \mathbf B$ is positive semidefinite too. Then, $\mathbf B^{-1} - \mathbf A^{-1}$ is also positive semidefinite.

Solution 1:

Yes. Here is one way to prove it:

Since $A-B\geq0$, we have (conjugating with $B^{-1/2}$) that $B^{-1/2}AB^{-1/2}-I\geq0$. This tells you that all the eigenvalues of the positive definite matrix $B^{-1/2}AB^{-1/2}$ are $\geq1$.

Now note that $B^{-1/2}AB^{-1/2}=(B^{-1/2}A^{1/2})(A^{1/2}B^{-1/2})$. Since commuting the product of two matrices does not change the eigenvalues, we get that $A^{1/2}B^{-1/2}B^{-1/2}A^{1/2}=A^{1/2}B^{-1}A^{1/2}$ has also all eigenvalues $\geq1$. So $$ A^{1/2}B^{-1}A^{1/2}-I\geq0. $$ Now conjugating with $A^{-1/2}$ we get $B^{-1}-A^{-1}\geq0$.

Solution 2:

I like Martin Argerami's answer better, but here is a slightly different approach which does not use that the nonzero eigenvalues of $CD$ are the same as those of $DC$.

Note: this approach applies verbatim to the more general case of a $C^*$-algebra. Note also that in view of the notion of operator monotone function, we could interpret this result as saying that $x\longmapsto x^{-1}$ is operator nonincreasing.

The rather lengthy following paragraph shows that we can assume $B=I_n$ without loss of generality. Then it is obvious.

Since $B$ is positive definite, we have its square root $B^{1/2}$ which is also positive definite with positive definite inverse $B^{-1/2}$. Now since $A-B$ is positive semidefinite, so is $B^{-1/2}(A-B)B^{-1/2}=B^{-1/2}AB^{-1/2}-I_n=A_0-I_n$. And to prove $B^{-1}-A^{-1}$ positive semidefinite is equivalent to proving $B^{1/2}(B^{-1}-A^{-1})B^{1/2}=I_n-A_0^{-1}$ is positive semidefinite. Since $A_0=B^{-1/2}AB^{-1/2}$ is still positive definite, this shows that we can assume $B=I_n$ withtout loss of generality.

Now using the usual notation $C\geq D$ for symmetric matrices to mean that $C-D$ is positive semidefinite, or equivalently that the spectrum of $C-D$ is nonnegative, we get $$ A-I_n\geq 0\Rightarrow A^{-1/2}(A-I_n)A^{-1/2}=I_n-A^{-1}\geq 0. $$ So the result you mentioned holds, together, of course, with its converse.

Solution 3:

Let me offer a constructive proof.

Lemma 1. For invertible and symmetric matrix $A$, and vectors $x$ and $y$, we have

$$ x^{'}A^{-1}x = \underset{y}{max\, \, } 2{x}'y - {y}'Ay $$

Lemma 2. For invertable matrices $A$ and $B$, we have

$$ (A + C) ^{-1} = A^{-1} - A^{-1} (A ^{-1} + C ^{-1}) ^{-1} A ^{-1} $$

Theorem. If $A > 0$, $B > 0$, $A$ and $B$ are symmetric matricex, and $A > B$, then $ B ^{-1} > A ^{-1} $.

Proof. By Lemma 1, we have

$$x ^{'} ( B ^{-1} - A ^{-1} ) x = \underset{y}{max\, \, } 2{x}'y - {y}' ( B ^{-1} - A ^{-1} ) ^{-1} y$$

Let $ y = A ^{-1} x $, we have $$ x ^{'} ( B ^{-1} - A ^{-1} ) x \ge 2x ^{'} A ^{-1} x - x ^{'} A ^{-1} ( B ^{-1} - A ^{-1} ) ^ {-1} A ^{-1} x $$

Then by Lemma 2, we have

$$ x ^{'} ( B ^{-1} - A ^{-1} ) x \ge x ^{'} A ^{-1} x + x ^{'} ( A - B ) ^{-1} x $$

By assumption, $ A > 0 $, so $ A ^{-1} > 0 $. Also $ A > B $, i.e. $ A - B > 0 $, so $ (A - B ) ^ {-1} > 0$. Thus the LHS of above equation is greater than 0, so $ B ^{-1} - A ^{-1} > 0 $, i.e. $ B ^{-1} > A ^{-1} $.

EDIT: typos.