Efficient diagonal update of matrix inverse
Turns out this was very simple. I'll just write it here if someone else has need for it: $$(kI+A)^{-1}=(kI+PDP^{-1})^{-1}=(P(D + kI)P^{-1})^{-1}=P(D + kI)^{-1}P^{-1}$$ where $A=PDP^{-1}$ is the eigenvalue decomposition. And the inverse of a diagonal matrix is quickly computed as the matrix with diagonal elements the reciprocal of the diagonal elements of the original matrix. This is roughly $50$ times faster than the original solution for my data on my computer.
-- Tommy L