How to calculate the cost of Cholesky decomposition?
Taken from http://www.cs.utexas.edu/users/flame/Notes/NotesOnCholReal.pdf
To analyze complexity for Cholesky decomposition of $n \times n$ matrix, let $f(n)$ be the cost of decomposition of $n \times n$ matrix
Then,
$f(n) = 2(n-1)^2 + (n-1) + 1 + f(n-1)$ , if we use rank 1 update for $A_{22} - L_{12}L_{12}^T$.
But, since we are only interested in lower triangular matrix, only lower triangular part need to be updated which requires approximately $\frac{n}{2} flops$.
So, $f(n) = (n-1)^2 + (n-1) + 1 + f(n-1)$
$f(1) = 1$
After solving this you get $f(n) = \frac{1}{3}n^3 + \frac{2}{3}n$ , which is approximately the total cost of Cholesky decomposition.
We want to factor $A$ into $R^TR$ where $R$ is upper triangular. The algorithm proceeds as follows.
R = A
for k=1 to m
for j=k+1 to m
R_{j,j:m} = R_{j,j:m} - R_{k,j:m} \bar{R}_{kj}/R_{kk}
R_{k,k:m} = R_{k,k:m}/\sqrt{R_{kk}}
The line inside the innermost for-loop
R_{j,j:m} = R_{j,j:m} - R_{k,j:m} \bar{R}_{kj}/R_{kk}
requires $1$ division, $m-j+1$ multiplications and $m-j+1$ subtractions. Since $j$ runs from $k+1$ to $m$, the cost will be
$m-k$ divisions,
$\left(\displaystyle (m+1)(m-k) - \sum_{j=k+1}^m j \right) = \dfrac{(m-k)(m+k+1)}2$ multiplications and
$\left(\displaystyle (m+1)(m-k) - \sum_{j=k+1}^m j \right) = \dfrac{(m-k)(m+k+1)}2$ subtractions.
Now in the outer loop $k$ runs from $1$ to $m$ costing $(m-1)$ divisions, $\dfrac13m(m^2-1)$ multiplications and $\dfrac13m(m^2-1)$ subtractions.
Hence, the total cost for Cholesky is
- $(m-1)$ divisions
- $\dfrac13m(m^2-1)$ multiplications
- $\dfrac13m(m^2-1)$ subtractions
I found Section 1.6 ("Serial complexity of the algorithm") of the following webpage to be useful for this topic: https://algowiki-project.org/en/Cholesky_decomposition
Edit: Here's the info from that page.
1.6 Serial complexity of the algorithm
The following number of operations should be performed to decompose a matrix of order $n$ using a serial version of the Cholesky algorithm:
- $n$ square roots
- $\frac{n(n-1)}{2}$ divisions
- $\frac{n^3-n}{6}$ multiplications and $\frac{n^3-n}{6}$ additions (subtractions): the main amount of computational work.
In the accumulation mode, the multiplication and subtraction operations should be made in double precision (or by using the corresponding function, like the DPROD function in Fortran), which increases the overall computation time of the Cholesky algorithm.
Thus, a serial version of the Cholesky algorithm is of cubic complexity.