You don't work my friend. Consider a few thousand random $10\times 10$ matrices and you can get $\sum_{i,j}A^{-1}_{i,j}\geq 16$.

EDIT. If I understand correctly your second point, you consider $0-1$ matrices with diagonal $1,\cdots, 1$ and $\det(A)=\pm 1$.

With a random test, I find the following $10\times 10$ matrix $A$ with $\det(A)=1$ and $\sum_{i,j}A^{-1}_{i,j}=21$.

enter image description here