How to find the minimum value of $f(x)=x^TAx+b^Tx+c$?

Solution 1:

Here is a solution with no calculus at all. It's the same process of completing the square that we learn in the first algebra course in high school with some linear algebra thrown in.

Let $q=\frac12 A^{-1}b$. Note that $$f(x) = (x+q)^\top A (x+q) + (c-q^\top Aq).$$ Notice that we use symmetry of $A$ to get $q^\top Ax= \frac12 b^\top (A^{-1}Ax) = \frac12 b^\top x$ and similarly for $x^\top Aq$. Since $A$ is positive definite, $y^\top Ay\ge 0$, with equality holding if and only if $y=0$. Thus, $f$ attains its minimum when $x+q=0$, i.e., when $x=-q=-\frac12 A^{-1}b$.