Inverse of Symmetric Matrix Plus Diagonal Matrix if Square Matrix's Inverse Is Known

Let $A$ be an $n \times n$ symmetric matrix of rank $n$ with known inverse $A^{-1}$. Let $D$ be a diagonal matrix with the same dimensions and rank. What is the fastest way to compute $(A+D)^{-1}$? Assume all diagonal elements of $A+D$ are positive and that $A+D$ is invertible.

Since $(A+D)^{-1} = (I+ A^{-1} D)^{-1} A^{-1}$, and $A^{-1}$ is known, we just need to find out how to calculate $B ^{-1}$ where $B$ is the (known) matrix $1+A^{-1} D$.

Following Greg Martin's observation, consider first the case where the only nonzero entry of $D$ is in the upper left corner. In this case, all columns of $A^{-1} D$ are zero except the first one so $B$ has the form
$$ B = \begin{pmatrix} v_1 & 0 & 0 &\cdots & 0\\ v_2 & 1 & 0 & \cdots & 0 \\ v_3 & 0 & 1 & \cdots & 0 \\ \vdots & & & & \vdots \\ v_n & 0 & 0 & \cdots & 1 \\ \end{pmatrix} = \begin{pmatrix} v_1 & 0 & 0 &\cdots & 0\\ 0 & 1 & 0 & \cdots & 0 \\ 0 & 0 & 1 & \cdots & 0 \\ \vdots & & & & \vdots \\ 0 & 0 & 0 & \cdots & 1 \\ \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 &\cdots & 0\\ v_2 & 1 & 0 & \cdots & 0 \\ v_3 & 0 & 1 & \cdots & 0 \\ \vdots & & & & \vdots \\ v_n & 0 & 0 & \cdots & 1 \\ \end{pmatrix}$$ Invertibility of $B$ is equivalent to the condition $v_1 \neq 0$. In this case both factors above are invertible. The inverse of the diagonal factor on the left is trivial to compute, and the inverse of the right factor happens to be $$ \begin{pmatrix} 1 & 0 & 0 &\cdots & 0\\ -v_2 & 1 & 0 & \cdots & 0 \\ -v_3 & 0 & 1 & \cdots & 0 \\ \vdots & & & & \vdots \\ -v_n & 0 & 0 & \cdots & 1 \\ \end{pmatrix}$$ so the inverse of $B$ is easy to recover. Namely: $$B^{-1} = \begin{pmatrix} \frac{1}{v_1} & 0 & 0 &\cdots & 0\\ -\frac{v_2}{v_1} & 1 & 0 & \cdots & 0 \\ -\frac{v_3}{v_1} & 0 & 1 & \cdots & 0 \\ \vdots & & & & \vdots \\ -\frac{v_n}{v_1} & 0 & 0 & \cdots & 1 \\ \end{pmatrix}.$$