Inverse of a block matrix with singular diagonal blocks
Solution 1:
If you are looking for a blockwise closed-form formula in terms of $A,B$ and $C$, I am very skeptical about its usefulness. Yet that doesn't mean there isn't one: since $X$ is invertible, $$ X^{-1} = (X^TX)^{-1}X^T ={\underbrace{\begin{bmatrix}A^TA+C^TC & A^TB\\ B^TA & B^TB\end{bmatrix}}_M}^{-1} \begin{bmatrix}A^T & C^T\\ B^T & 0\end{bmatrix}. $$ As $X$ is invertible, $M=X^TX$ is positive definite. Hence the diagonal sub-block $B^TB$ is invertible (and in fact, positive definite). In turn, its Schur complement in $M$ is also invertible. By applying the matrix inversion formula in terms of Schur complement, we get $$ X^{-1}= \begin{bmatrix} S^{-1} & -S^{-1} A^TB (B^TB)^{-1} \\ -(B^TB)^{-1} B^TA S^{-1} & (B^TB)^{-1} + (B^TB)^{-1} B^TA S^{-1} A^TB (B^TB)^{-1} \end{bmatrix} \begin{bmatrix}A^T & C^T\\ B^T & 0\end{bmatrix}, $$ where $S=A^TA+C^TC-A^TB(B^TB)^{-1}B^TA$ is the Schur complement of $B^TB$ in $M$.
Edit. The above idea can be generalised to any invertible complex matrix $X=\begin{bmatrix}A&B\\ C&D\end{bmatrix}$ with square diagonal sub-blocks $A$ and $D$ of possibly different sizes. We have \begin{aligned} X^{-1} = (X^\ast X)^{-1}X^\ast &={\begin{bmatrix}A^\ast A+C^\ast C & A^\ast B+C^\ast D\\ B^\ast A+D^\ast C & B^\ast B+D^\ast D\end{bmatrix}}^{-1} \begin{bmatrix}A^\ast & C^\ast\\ B^\ast & D^\ast\end{bmatrix}\\ &=:{\underbrace{\begin{bmatrix}P&Q\\ Q^\ast&R\end{bmatrix}}_M}^{-1} \begin{bmatrix}A^\ast & C^\ast\\ B^\ast & D^\ast\end{bmatrix} \end{aligned} where $P= A^\ast A+C^\ast C,\ Q=A^\ast B+C^\ast D$ and $R=B^\ast B+D^\ast D$.
Since $X$ is invertible, $M=X^\ast X$ is positive definite. Thus the principal submatrices $R$ and its Schur complement $S=P-QR^{-1}Q^\ast$ in $M$ are also positive definite and we may apply the usual block matrix inverse formula to $M$ to obtain $$ X^{-1}= \begin{bmatrix} S^{-1} & -S^{-1} QR^{-1} \\ -R^{-1}Q^\ast S^{-1} & R^{-1} + R^{-1}Q^\ast S^{-1}QR^{-1} \end{bmatrix} \begin{bmatrix}A^\ast & C^\ast\\ B^\ast & D^\ast\end{bmatrix}. $$